4 papers
ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models
Nikhil Anand, Shwetha Somasundaram, Anirudh Phukan +2
Large Language Models (LLMs) encode vast amounts of parametric knowledge during pre-training. As world knowledge evolves, effective deployment increasingly depends on their ability…
Dataset Difficulty and the Role of Inductive Bias
Devin Kwok, Nikhil Anand, Jonathan Frankle +2
Motivated by the goals of dataset pruning and defect identification, a growing body of methods have been developed to score individual examples within a dataset. These methods, whi…
Comprehensive Benchmarking of Entropy and Margin Based Scoring Metrics for Data Selection
Anusha Sabbineni, Nikhil Anand, Maria Minakova
While data selection methods have been studied extensively in active learning, data pruning, and data augmentation settings, there is little evidence for the efficacy of these meth…
Influence Scores at Scale for Efficient Language Data Sampling
Nikhil Anand, Joshua Tan, Maria Minakova
Modern ML systems ingest data aggregated from diverse sources, such as synthetic, human-annotated, and live customer traffic. Understanding \textit{which} examples are important to…