most citedLatent Multi-Head Attention for Small Language Models

1 citations · 1 across the 18 of their papers we have counts for

collaborators
Showing cs.LGShow all

5 papers · 1 filter

cs.LG2026

Cross-Benchmark Transfer from RL on Agentic Coding Tasks

Sushant Mehta, Logan Ritchie, Edwin Chen

Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was su…

cs.LG2025

Beyond Surface-Level Similarity: Hierarchical Contamination Detection for Synthetic Training Data in Foundation Models

Sushant Mehta

Synthetic data has become essential for training foundation models, yet benchmark contamination threatens evaluation integrity. Although existing detection methods identify token-l…

cs.LG2025

When Are Learning Biases Equivalent? A Unifying Framework for Fairness, Robustness, and Distribution Shift

Sushant Mehta

Machine learning systems exhibit diverse failure modes: unfairness toward protected groups, brittleness to spurious correlations, poor performance on minority sub-populations, whic…

cs.LG2025

Scaling Laws and In-Context Learning: A Unified Theoretical Framework

Sushant Mehta, Ishan Gupta

In-context learning (ICL) enables large language models to adapt to new tasks from demonstrations without parameter updates. Despite extensive empirical studies, a principled under…

cs.LG2025

Muon: Training and Trade-offs with Latent Attention and MoE

Sushant Mehta, Raj Dandekar, Rajat Dandekar +1

We present a comprehensive theoretical and empirical study of the Muon optimizer for training transformers only with a small to medium decoder (30M - 200M parameters), with an emph…