6 papers
Estimating the Empowerment of Language Model Agents
Jinyeop Song, Jeff Gore, Max Kleiman-Weiner
As language model (LM) agents become increasingly capable and adopted in real-world applications, there is a growing need for scalable evaluation frameworks beyond costly, manually…
Efficient and Transferable Agentic Knowledge Graph RAG via Reinforcement Learning
Junhong Lin, Shicheng Liu, Jinyeop Song +3
Knowledge-graph retrieval-augmented generation (KG-RAG) couples large language models (LLMs) with structured, verifiable knowledge graphs (KGs) to reduce hallucination and provide…
The Blessing of Dimensionality in LLM Fine-tuning: A Variance-Curvature Perspective
Qiyao Liang, Jinyeop Song, Yizhou Liu +4
Weight-perturbation evolution strategies (ES) can fine-tune billion-parameter language models with surprisingly small populations (e.g., ), contradicting classical…
Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective
Seungwook Han, Jinyeop Song, Jeff Gore +1
Autoregressive transformers exhibit adaptive learning through in-context learning (ICL), which begs the question of how. Prior works have shown that transformers represent the ICL…
When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research
Guijin Son, Jiwoo Hong, Honglu Fan +8
Recent advances in large language models (LLMs) have fueled the vision of automated scientific discovery, often called AI Co-Scientists. To date, prior work casts these systems as…
Reconciling Kaplan and Chinchilla Scaling Laws
Tim Pearce, Jinyeop Song
Kaplan et al. [2020] (`Kaplan') and Hoffmann et al. [2022] (`Chinchilla') studied the scaling behavior of transformers trained on next-token language prediction. These studies prod…