192 citations · 585 across the 33 of their papers we have counts for
4 papers · 1 filter
GRIN: GRadient-INformed MoE
Liyuan Liu, Young Jin Kim, Shuohang Wang +14
Mixture-of-Experts (MoE) models scale more effectively than dense models due to sparse computation through expert routing, selectively activating only a small subset of expert modu…
Simply Trainable Nearest Neighbour Machine Translation with GPU Inference
Hossam Amer, Abdelrahman Abouelenin, Mohamed Maher +3
Nearest neighbor machine translation is a successful approach for fast domain adaption, which interpolates the pre-trained transformers with domain-specific token-level k-nearest-n…
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Marah Abdin, Jyoti Aneja, Hany Awadalla +126
We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal test…
Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation
Juan Manuel Zambrano Chaves, Shih-Cheng Huang, Yanbo Xu +24
The scaling laws and extraordinary performance of large foundation models motivate the development and utilization of such models in biomedicine. However, despite early promising r…