164 citations · 171 across the 4 of their papers we have counts for
4 papers
GRIN: GRadient-INformed MoE
Liyuan Liu, Young Jin Kim, Shuohang Wang +14
Mixture-of-Experts (MoE) models scale more effectively than dense models due to sparse computation through expert routing, selectively activating only a small subset of expert modu…
Implicit Chain of Thought Reasoning via Knowledge Distillation
Yuntian Deng, Kiran Prasad, Roland Fernandez +3
To augment language models with the ability to reason, researchers usually prompt or finetune them to produce chain of thought reasoning steps before producing the final answer. Ho…
Language Is Not All You Need: Aligning Perception with Language Models
Shaohan Huang, Li Dong, Wenhui Wang +15
A big convergence of language, multimodal perception, action, and world modeling is a key step toward artificial general intelligence. In this work, we introduce Kosmos-1, a Multim…
Understanding the Effectiveness of Very Large Language Models on Dialog Evaluation
Jessica Huynh, Cathy Jiao, Prakhar Gupta +4
Language models have steadily increased in size over the past few years. They achieve a high level of performance on various natural language processing (NLP) tasks such as questio…