3 papers
cs.CL2024
Dynamic layer selection in decoder-only transformers
Theodore Glavas, Joud Chataoui, Florence Regol +4
The vast size of Large Language Models (LLMs) has prompted a search to optimize inference. One effective approach is dynamic inference, which adapts the architecture to the sample-…
cs.LG2024
Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEE
Florence Regol, Joud Chataoui, Bertrand Charpentier +3
Machine learning models can solve complex tasks but often require significant computational resources during inference. This has led to the development of various post-training com…
cs.LG2024
Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNN
Florence Regol, Joud Chataoui, Mark Coates
Large pretrained models, coupled with fine-tuning, are slowly becoming established as the dominant architecture in machine learning. Even though these models offer impressive perfo…