23.6k citations
- Carnegie Mellon UniversityUS49 papers
- Google (United States)US37 papers
- California Institute of TechnologyUS33 papers
- Amazon (Germany)DE32 papers
- University of Illinois Urbana-ChampaignUS22 papers
- Microsoft (United States)US20 papers
- Stanford UniversityUS20 papers
- The University of Texas at AustinUS20 papers
- University of California, Los AngelesUS20 papers
- University of California, Santa BarbaraUS19 papers
- University of WashingtonUS19 papers
- Georgia Institute of TechnologyUS17 papers
4 papers · 2 filters
ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration
Mengting Ai, Tianxin Wei, Yifan Chen +7
Mixture-of-Experts (MoE) Transformer, the backbone architecture of multiple phenomenal language models, leverages sparsity by activating only a fraction of model parameters for eac…
Mitigating Attrition: Data-Driven Approach Using Machine Learning and Data Engineering
Naveen Edapurath Vijayan
This paper presents a novel data-driven approach to mitigating employee attrition using machine learning and data engineering techniques. The proposed framework integrates data fro…
LDC-MTL: Balancing Multi-Task Learning through Scalable Loss Discrepancy Control
Peiyao Xiao, Chaosheng Dong, Shaofeng Zou +1
Multi-task learning (MTL) has been widely adopted for its ability to simultaneously learn multiple tasks. While existing gradient manipulation methods often yield more balanced sol…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1144
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…