18 papers
Mergeable Model-Side Aggregation States for Long-Context Language Models
Dachuan Song, Junyu Yin, Zechen Hu +1
The paper proposes a model-side aggregation interface that uses compact HyperLogLog sketches to maintain set-based aggregation states alongside frozen language models, enabling acc…
EffGen: Enabling Small Language Models as Capable Autonomous Agents
Gaurav Srivastava, Aafiya Hussain, Chi Wang +2
Most existing language model agentic systems today are built and optimized for large language models (e.g., GPT, Claude, Gemini) via API calls; while powerful, this approach faces…
BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models
Yi Fang, Haoran Xu, Jiaxin Han +4
Foundation models have revolutionized various fields such as natural language processing (NLP) and computer vision (CV). While efforts have been made to transfer the success of the…
Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Models
Gaurav Srivastava, Aafiya Hussain, Sriram Srinivasan +1
Large language models (LLMs) achieve impressive performance on complex mathematical benchmarks yet sometimes fail on basic math reasoning while generating unnecessarily verbose res…
Elastic Spectral State Space Models for Budgeted Inference
Dachuan Song, Xuan Wang
Foundation models are typically trained at a fixed computational capacity, while real-world applications require deployment across platforms with different resource constraints. Cu…
MEDVISTAGYM: A Scalable Training Environment for Thinking with Medical Images via Tool-Integrated Reinforcement Learning
Meng Lu, Yuxing Lu, Yuchen Zhuang +6
Vision language models (VLMs) achieve strong performance on general image understanding but struggle to think with medical images, especially when performing multi-step reasoning t…