1 citations · 2 across the 8 of their papers we have counts for
3 papers · 1 filter
A Large Scale Investigation of Scaling Limits in Chemical Language Models
Kamran Chitsaz, Roshan Balaji, Quentin Fournier +2
Chemical Language Models (CLMs) are increasingly used in de novo drug design, driven by recent growth in model scale, compute, and dataset size. However, the relationship between d…
Exploring Quantization for Efficient Pre-Training of Transformer Language Models
Kamran Chitsaz, Quentin Fournier, Gonçalo Mordido +1
The increasing scale of Transformer models has led to an increase in their pre-training computational requirements. While quantization has proven to be effective after pre-training…
Manifold Metric: A Loss Landscape Approach for Predicting Model Performance
Pranshu Malviya, Jerry Huang, Aristide Baratin +2
Determining the optimal model for a given task often requires training multiple models from scratch, which becomes impractical as dataset and model sizes grow. A more efficient alt…