1 citations · 1 across the 10 of their papers we have counts for
11 papers
Language Models Are "Insecure" Reporters
Jenny Y. Huang, Jiameng Fan, Ahmed Imtiaz Humayun +5
As large language models are deployed in increasingly autonomous long-horizon tasks, manually auditing and verifying the actions, artifacts, and outputs of models becomes more diff…
Bridging Compute- and Data-Optimal Pretraining
Tian Qin, Kimia Hamidieh, David Alvarez-Melis
Classical compute-optimal scaling laws assume an unbounded supply of fresh pretraining data, yet pretraining is increasingly entering a regime in which compute grows faster than th…
RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training
Rachit Bansal, Clara Mohri, Tian Qin +2
The standard LLM training pipeline applies reinforcement learning (RL) only after pre-training and supervised fine-tuning (SFT). We question this status quo by training a LLM from…
Do We Really Need External Tools to Mitigate Hallucinations? SIRA: Shared-Prefix Internal Reconstruction of Attribution
Tian Qin, Junzhe Chen, Yuqing Shi +3
Large vision-language models (LVLMs) often hallucinate when language priors dominate weak or ambiguous visual evidence. Existing contrastive decoding methods mitigate this problem…
Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power
Yuzhu Chen, Tian Qin, Xinmei Tian +2
Equivariant neural networks encode the intrinsic symmetry of data as an inductive bias, which has achieved impressive performance in wide domains. However, the understanding to the…
Decomposing Elements of Problem Solving: What "Math" Does RL Teach?
Tian Qin, Core Francisco Park, Mujin Kwun +5
Mathematical reasoning tasks have become prominent benchmarks for assessing the reasoning capabilities of LLMs, especially with reinforcement learning (RL) methods such as GRPO sho…
To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoning
Tian Qin, David Alvarez-Melis, Samy Jelassi +1
Recent advancements in large language models (LLMs) have significantly improved their reasoning abilities, particularly through techniques involving search and backtracking. Backtr…
Random Scaling of Emergent Capabilities
Rosie Zhao, Tian Qin, David Alvarez-Melis +2
Language models famously improve under a smooth scaling law, but some specific capabilities exhibit sudden breakthroughs in performance. Advocates of "emergence" view these capabil…
Sometimes I am a Tree: Data Drives Unstable Hierarchical Generalization
Tian Qin, Naomi Saphra, David Alvarez-Melis
Early in training, LMs can behave like n-gram models, but eventually they often learn tree-based syntactic rules and generalize hierarchically out of distribution (OOD). We study t…
A Label is Worth a Thousand Images in Dataset Distillation
Tian Qin, Zhiwei Deng, David Alvarez-Melis
Data is a crucial factor in the performance of machine learning models, a principle that dataset distillation methods exploit by compressing training datasets in…
Distributional Dataset Distillation with Subtask Decomposition
Tian Qin, Zhiwei Deng, David Alvarez-Melis
What does a neural network learn when training from a task-specific dataset? Synthesizing this knowledge is the central idea behind Dataset Distillation, which recent work has show…