9 citations · 15 across the 11 of their papers we have counts for
5 papers · 1 filter
Mixing Times of Glauber Dynamics on Masked Language Models
Suvadip Sana, Sami Wolf, Neer Mehta +4
Masked language models (MLMs) define local conditional distributions over tokens but do not, in general, correspond to any consistent joint distribution over sequences. This raises…
MI-to-Mid Distilled Compression (M2M-DC): An Hybrid-Information-Guided-Block Pruning with Progressive Inner Slicing Approach to Model Compression
Lionel Levine, Haniyeh Ehsani Oskouie, Sajjad Ghiasvand +1
We introduce MI-to-Mid Distilled Compression (M2M-DC), a two-scale, shape-safe compression framework that interleaves information-guided block pruning with progressive inner slicin…
Exploring the Impact of Dataset Statistical Effect Size on Model Performance and Data Sample Size Sufficiency
Arya Hatamian, Lionel Levine, Haniyeh Ehsani Oskouie +1
Having a sufficient quantity of quality data is a critical enabler of training effective machine learning models. Being able to effectively determine the adequacy of a dataset prio…
Exploring Cross-model Neuronal Correlations in the Context of Predicting Model Performance and Generalizability
Haniyeh Ehsani Oskouie, Sajjad Ghiasvand, Lionel Levine +1
As Artificial Intelligence (AI) models are increasingly integrated into critical systems, the need for a robust framework to establish the trustworthiness of AI is increasingly par…
Do language models plan ahead for future tokens?
Wilson Wu, John X. Morris, Lionel Levine
Do transformers "think ahead" during inference at a given position? It is known transformers prepare information in the hidden states of the forward pass at time step that is t…