Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024
Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast
Chufan Shi, Cheng Yang, Xinyu Zhu +6
Mixture-of-Experts (MoE) has emerged as a prominent architecture for scaling model size while maintaining computational efficiency. In MoE, each token in the input sequence activat…
cs.CL2024
A Thorough Examination of Decoding Methods in the Era of LLMs
Chufan Shi, Haoran Yang, Deng Cai +4
Decoding methods play an indispensable role in converting language models from next-token predictors into practical task solvers. Prior research on decoding methods, primarily focu…