5 citations · 10 across the 16 of their papers we have counts for
8 papers · 1 filter
Curriculum Learning as Transport: Understanding Curricula with Wasserstein Geodesics
Changho Shin, David Alvarez-Melis
Curriculum learning is governed by several coupled design choices---how difficulty is defined, how examples are ordered, how much exposure each level receives, and how quickly trai…
Bridging Compute- and Data-Optimal Pretraining
Tian Qin, Kimia Hamidieh, David Alvarez-Melis
Classical compute-optimal scaling laws assume an unbounded supply of fresh pretraining data, yet pretraining is increasingly entering a regime in which compute grows faster than th…
Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry
Jennifer Meng Lu, Ruochen Zhang, Isabelle Lee +3
Humans cannot always intuit what scenarios are most challenging to LLMs. Hoping to capture challenging edge cases, developers either design problems to be difficult for humans or c…
RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training
Rachit Bansal, Clara Mohri, Tian Qin +2
The standard LLM training pipeline applies reinforcement learning (RL) only after pre-training and supervised fine-tuning (SFT). We question this status quo by training a LLM from…
Low-Frequency Shortcuts in Texture-Driven Visual Learning
Utku Şirin, Cathy Hou, David Alvarez-Melis +2
Neural networks suffer from shortcut learning, where learned features generalize well to the training set but not to in-distribution (ID) or out-of-distribution (OOD) test sets. Ex…
Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention
Jing Huang, Daniel Wurgaft, Rachit Bansal +6
Larger models learn tasks smaller models do not. What drives this phenomenon? We develop a simple phenomenological argument that power-law scaling already suggests that a larger mo…
Reliable and Responsible Foundation Models: A Comprehensive Survey
Xinyu Yang, Junlin Han, Rishi Bommasani +49
Foundation models, including Large Language Models (LLMs), Multimodal Large Language Models (MLLMs), Image Generative Models (i.e, Text-to-Image Models and Image-Editing Models), a…
Beyond Explainable AI (XAI): An Overdue Paradigm Shift and Post-XAI Research Directions
Saleh Afroogh, Syed Ishtiaque Ahmed, Petra Ahrweiler +46
This study provides a cross-disciplinary examination of Explainable Artificial Intelligence (XAI) approaches-focusing on deep neural networks (DNNs) and large language models (LLMs…