13 citations · 18 across the 14 of their papers we have counts for
17 papers
Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency
Parsa Hosseini, Akasha Tigalappanavara, Sumit Nawathe +6
Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency either through inference-ti…
ContextWeaver: Selective and Dependency-Structured Memory Construction for LLM Agents
Yating Wu, Yuhao Zhang, Sayan Ghosh +4
Large language model (LLM) agents often struggle in long-context interactions. As the history grows, sliding windows and prompt compression may drop earlier information that later…
Know Thy Reasoner: Not All Language Models Explore Alike
Moulik Choraria, Argyrios Gerogiannis, Anirban Das +4
Compute scaling for LLM reasoning trades off exploring solution approaches (\emph{breadth}) against refining promising ones (\emph{depth}), yet why a given trade-off works, and why…
G-RepsNet: A Fast and General Construction of Equivariant Networks for Arbitrary Matrix Groups
Sourya Basu, Suhas Lohit, Matthew Brand
Group equivariance is a strong inductive bias useful in a wide range of deep learning tasks. However, constructing efficient equivariant networks for general groups and domains is…
Efficient Model-Agnostic Multi-Group Equivariant Networks
Razan Baltaji, Sourya Basu, Lav R. Varshney
Constructing model-agnostic group equivariant networks, such as equitune (Basu et al., 2023b) and its generalizations (Kim et al., 2023), can be computationally expensive for large…
Transformers are Universal Predictors
Sourya Basu, Moulik Choraria, Lav R. Varshney
We find limits to the Transformer architecture for language modeling and show it has a universal prediction property in an information-theoretic sense. We further analyze performan…
Efficient Equivariant Transfer Learning from Pretrained Models
Sourya Basu, Pulkit Katdare, Prasanna Sattigeri +4
Efficient transfer learning algorithms are key to the success of foundation models on diverse downstream tasks even with limited data. Recent works of Basu et al. (2023) and Kaba e…
Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models
Sourya Basu, Prasanna Sattigeri, Karthikeyan Natesan Ramamurthy +4
We introduce equi-tuning, a novel fine-tuning method that transforms (potentially non-equivariant) pretrained models into group equivariant models while incurring minimum los…
Equivariant Mesh Attention Networks
Sourya Basu, Jose Gallego-Posada, Francesco Viganò +2
Equivariance to symmetries has proven to be a powerful inductive bias in deep learning research. Recent works on mesh processing have concentrated on various kinds of natural symme…
Hypergraph-based Source Codes for Function Computation Under Maximal Distortion
Sourya Basu, Daewon Seo, Lav R. Varshney
This work investigates functional source coding problems with maximal distortion, motivated by approximate function computation in many modern applications. The maximal distortion…
Advanced Methods for Connectome-Based Predictive Modeling of Human Intelligence: A Novel Approach Based on Individual Differences in Cortical Topography
Evan D. Anderson, Ramsey Wilcox, Anuj Nayak +5
Individual differences in human intelligence can be modeled and predicted from in vivo neurobiological connectivity. Many established modeling frameworks for predicting intelligenc…
Autoequivariant Network Search via Group Decomposition
Sourya Basu, Akshayaa Magesh, Harshit Yadav +1
Recent works show that group equivariance as an inductive bias improves neural network performance for both classification and generation. However, designing group-equivariant neur…
The Twelvefold Way of Non-Sequential Lossless Compression
Taha Ameen ur Rahman, Alton S. Barbehenn, Xinan Chen +27
Many information sources are not just sequences of distinguishable symbols but rather have invariances governed by alternative counting paradigms such as permutations, combinations…
Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity
Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar +1
Neural text decoding is important for generating high-quality texts using language models. To generate high-quality text, popular decoding algorithms like top-k, top-p (nucleus), a…
Functional Epsilon Entropy
Sourya Basu, Daewon Seo, Lav R. Varshney
We consider the problem of coding for computing with maximal distortion, where the sender communicates with a receiver, which has its own private data and wants to compute a functi…
Coding for Computing Arbitrary Functions Unknown to the Encoder
Sourya Basu, Lav R. Varshney
In this paper we consider point-to-point and distributed source coding problems where the receiver is only interested in a function of the data sent by the source encoder(s), while…
Universal and Succinct Source Coding of Deep Neural Networks
Sourya Basu, Lav R. Varshney
Deep neural networks have shown incredible performance for inference tasks in a variety of domains. Unfortunately, most current deep networks are enormous cloud-based structures th…