6 papers
Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL
Juliette Decugis, Sean O'Brien, Francis Bach +2
Reinforcement learning post-training dramatically improves LLM reasoning, but suffers from training instability and diversity collapse. Advantage functions offer an appealing fix:…
A Few Bad Neurons: Isolating and Surgically Correcting Sycophancy
Claire O'Brien, Jessica Seto, Dristi Roy +6
Behavioral alignment in large language models (LLMs) is often achieved through broad fine-tuning, which can result in undesired side effects like distributional shift and low inter…
Separate the Wheat from the Chaff: Winnowing Down Divergent Views in Retrieval Augmented Generation
Song Wang, Zihan Chen, Peng Wang +5
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by integrating external knowledge sources to address their limitations in accessing up-to-date or special…
Semantic Convergence: Investigating Shared Representations Across Scaled LLMs
Daniel Son, Sanjana Rathore, Andrew Rufail +6
We investigate feature universality in Gemma-2 language models (Gemma-2-2B and Gemma-2-9B), asking whether models with a four-fold difference in scale still converge on comparable…
Introducing MAPO: Momentum-Aided Gradient Descent Prompt Optimization
Anthony Cui, Pranav Nandyalam, Andrew Rufail +4
Momentum-Aided Prompt Optimization (MAPO) enhances the efficiency and efficacy of prompt optimization for Large Language Models (LLMs). Building on ProTeGi, MAPO uses positive natu…
Are you really listening? Boosting Perceptual Awareness in Music-QA Benchmarks
Yongyi Zang, Sean O'Brien, Taylor Berg-Kirkpatrick +2
Large Audio Language Models (LALMs), where pretrained text LLMs are finetuned with audio input, have made remarkable progress in music understanding. However, current evaluation me…