9 papers
Dist2ill: Distributional Distillation for One-Pass Uncertainty Estimation in Large Language Models
Yicong Zhao, King Yeung Tsang, Harshil Vejendla +9
Large Language Models (LLMs) often exhibit misalignment between the quality of their generated responses and the confidence estimates they assign to them. Bayesian treatments, such…
RewriteNets: End-to-End Trainable String-Rewriting for Generative Sequence Modeling
Harshil Vejendla
Dominant sequence models like the Transformer represent structure implicitly through dense attention weights, incurring quadratic complexity. We propose RewriteNets, a novel neural…
Teaching by Failure: Counter-Example-Driven Curricula for Transformer Self-Improvement
Harshil Vejendla
Transformer models often exhibit brittle extrapolation, failing on inputs that are longer or structurally more complex than those seen during training. We introduce Counter-Example…
LATTA: Langevin-Anchored Test-Time Adaptation for Enhanced Robustness and Stability
Harshil Vejendla
Test-time adaptation (TTA) aims to adapt a pretrained model to distribution shifts using only unlabeled test data. While promising, existing methods like Tent suffer from instabili…
H1B-KV: Hybrid One-Bit Caches for Memory-Efficient Large Language Model Inference
Harshil Vejendla
Autoregressive decoding in large language models (LLMs) requires caching a growing list of past key-value (KV) pairs, making long-context inference a memory-bound problem. While re…
Learning to Predict Chaos: Curriculum-Driven Training for Robust Forecasting of Chaotic Dynamics
Harshil Vejendla
Forecasting chaotic systems is a cornerstone challenge in many scientific fields, complicated by the exponential amplification of even infinitesimal prediction errors. Modern machi…