9 papers
RewriteNets: End-to-End Trainable String-Rewriting for Generative Sequence Modeling
Harshil Vejendla
Dominant sequence models like the Transformer represent structure implicitly through dense attention weights, incurring quadratic complexity. We propose RewriteNets, a novel neural…
Teaching by Failure: Counter-Example-Driven Curricula for Transformer Self-Improvement
Harshil Vejendla
Transformer models often exhibit brittle extrapolation, failing on inputs that are longer or structurally more complex than those seen during training. We introduce Counter-Example…
LATTA: Langevin-Anchored Test-Time Adaptation for Enhanced Robustness and Stability
Harshil Vejendla
Test-time adaptation (TTA) aims to adapt a pretrained model to distribution shifts using only unlabeled test data. While promising, existing methods like Tent suffer from instabili…
H1B-KV: Hybrid One-Bit Caches for Memory-Efficient Large Language Model Inference
Harshil Vejendla
Autoregressive decoding in large language models (LLMs) requires caching a growing list of past key-value (KV) pairs, making long-context inference a memory-bound problem. While re…
Learning to Predict Chaos: Curriculum-Driven Training for Robust Forecasting of Chaotic Dynamics
Harshil Vejendla
Forecasting chaotic systems is a cornerstone challenge in many scientific fields, complicated by the exponential amplification of even infinitesimal prediction errors. Modern machi…
Wave-PDE Nets: Trainable Wave-Equation Layers as an Alternative to Attention
Harshil Vejendla
We introduce Wave-PDE Nets, a neural architecture whose elementary operation is a differentiable simulation of the second-order wave equation. Each layer propagates its hidden stat…
SliceMoE: Routing Embedding Slices Instead of Tokens for Fine-Grained and Balanced Transformer Scaling
Harshil Vejendla
Mixture-of-Experts (MoE) layers scale transformers by routing tokens to a sparse subset of feed-forward experts. Token-level routing, however, assigns an entire semantic spectrum t…
Drift-Adapter: A Practical Approach to Near Zero-Downtime Embedding Model Upgrades in Vector Databases
Harshil Vejendla
Upgrading embedding models in production vector databases typically requires re-encoding the entire corpus and rebuilding the Approximate Nearest Neighbor (ANN) index, leading to s…
Dist2ill: Distributional Distillation for One-Pass Uncertainty Estimation in Large Language Models
Yicong Zhao, King Yeung Tsang, Harshil Vejendla +9
Large Language Models (LLMs) often exhibit misalignment between the quality of their generated responses and the confidence estimates they assign to them. Bayesian treatments, such…