24 citations · 30 across the 9 of their papers we have counts for
9 papers
EMAFusion: A Self-Optimizing System for Seamless LLM Selection and Integration
Soham Shah, Kumar Shridhar, Surojit Chatterjee +1
While recent advances in large language models (LLMs) have significantly enhanced performance across diverse natural language tasks, the high computational and financial costs asso…
UNDO: Understanding Distillation as Optimization
Kushal Jain, Piyushi Goyal, Kumar Shridhar
Knowledge distillation has emerged as an effective strategy for compressing large language models' (LLMs) knowledge into smaller, more efficient student models. However, standard o…
Beyond Pattern Recognition: Probing Mental Representations of LMs
Moritz Miller, Kumar Shridhar
Language Models (LMs) have demonstrated impressive capabilities in solving complex reasoning tasks, particularly when prompted to generate intermediate explanations. However, it re…
Distilling LLMs' Decomposition Abilities into Compact Language Models
Denis Tarasov, Kumar Shridhar
Large Language Models (LLMs) have demonstrated proficiency in their reasoning abilities, yet their large size presents scalability challenges and limits any further customization.…
First-Step Advantage: Importance of Starting Right in Multi-Step Math Reasoning
Kushal Jain, Moritz Miller, Niket Tandon +1
Language models can solve complex reasoning tasks better by learning to generate rationales for their predictions. Often these models know how to solve a task but their auto-regres…
SCREWS: A Modular Framework for Reasoning with Revisions
Kumar Shridhar, Harsh Jhamtani, Hao Fang +3
Large language models (LLMs) can improve their accuracy on various tasks through iteratively refining and revising their output based on feedback. We observe that these revisions c…
Learning to Drop Out: An Adversarial Approach to Training Sequence VAEs
Đorđe Miladinović, Kumar Shridhar, Kushal Jain +4
In principle, applying variational autoencoders (VAEs) to sequential data offers a method for controlled sequence generation, manipulation, and structured representation learning.…
Translational Equivariance in Kernelizable Attention
Max Horn, Kumar Shridhar, Elrich Groenewald +1
While Transformer architectures have show remarkable success, they are bound to the computation of all pairwise interactions of input element and thus suffer from limited scalabili…
ProbAct: A Probabilistic Activation Function for Deep Neural Networks
Kumar Shridhar, Joonho Lee, Hideaki Hayashi +6
Activation functions play an important role in training artificial neural networks. The majority of currently used activation functions are deterministic in nature, with their fixe…