Compute-efficient pretraining and scaling to trillion-parameter models

100M Token Context Windows