2 citations · 3 across the 18 of their papers we have counts for
6 papers · 1 filter
OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
Liujianfu Wang, Yuyang Du, Yuchen Pan +3
Mixture-of-Experts (MoE), while offering significant advantages as a Large Language Model (LLM) architecture, faces substantial challenges when deployed on low-cost edge devices wi…
Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Data
Jiacheng Liu, Mayi Xu, Qiankun Pi +5
Large Language Models (LLMs) are increasingly employed in applications that require processing information from heterogeneous formats, including texts, tables, infoboxes, and knowl…
DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration
Tianteng Gu, Bei Liu, Bo Xiao +3
Pruning is a widely used technique to compress large language models (LLMs) by removing unimportant weights, but it often suffers from significant performance degradation - especia…
ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data
Tong Chen, Faeze Brahman, Jiacheng Liu +5
Language models (LMs) can memorize and reproduce segments from their pretraining data verbatim even in non-adversarial settings, raising concerns about copyright, plagiarism, priva…
DataDecide: How to Predict Best Pretraining Data with Small Experiments
Ian Magnusson, Nguyen Tai, Ben Bogin +10
Because large language models are expensive to pretrain on different datasets, using smaller-scale experiments to decide on data is crucial for reducing costs. Which benchmarks and…
Efficient Test-Time Scaling via Self-Calibration
Chengsong Huang, Langlin Huang, Jixuan Leng +2
Increasing test-time computation is a straightforward approach to enhancing the quality of responses in Large Language Models (LLMs). While Best-of-N sampling and Self-Consistency…