1 citations · 2 across the 14 of their papers we have counts for
11 papers · 1 filter
Conditional Flow Models Convert Epistemic Uncertainty into Apparent Aleatoric Variance
Navodita Sharma, Abhishek Shetty, Max Simchowitz +1
Conditional Flow Models (CFMs) are increasingly being used in place of deterministic predictors across machine learning, including for tasks that were classically formulated as reg…
Dynamic Compression in Recurrent Networks
Jyothish Pari, Ryan Bahlous-Boldi, Pulkit Agrawal
Recurrent models process long contexts efficiently by compressing their history into a fixed-size state, but modern architectures typically do so in a single causal pass over the s…
H2LooP Spark Preview: Continual Pretraining of Large Language Models for Low-Level Embedded Systems Code
Amit Singh, Vedant Nipane, Pulkit Agrawal +2
Large language models (LLMs) demonstrate strong code generation abilities in general-purpose programming languages but remain limited in specialized domains such as low-level embed…
Training Language Models via Neural Cellular Automata
Dan Lee, Seungwook Han, Akarsh Kumar +1
Pre-training is crucial for large language models (LLMs), as it is when most representations and capabilities are acquired. However, natural language pre-training has problems: hig…
Self-Distillation Enables Continual Learning
Idan Shenfeld, Mehul Damani, Jonas Hübotter +1
Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-pol…
RL's Razor: Why Online Reinforcement Learning Forgets Less
Idan Shenfeld, Jyothish Pari, Pulkit Agrawal
Comparison of fine-tuning models with reinforcement learning (RL) and supervised fine-tuning (SFT) reveals that, despite similar performance at a new task, RL preserves prior knowl…
Self-Adapting Language Models
Adam Zweiger, Jyothish Pari, Han Guo +3
Large language models (LLMs) are powerful but static; they lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples. We introduce Self-Adapting LLMs…
Known Unknowns: Out-of-Distribution Property Prediction in Materials and Molecules
Nofit Segal, Aviv Netanyahu, Kevin P. Greenman +2
Discovery of high-performance materials and molecules requires identifying extremes with property values that fall outside the known distribution. Therefore, the ability to extrapo…
General Intelligence Requires Reward-based Pretraining
Seungwook Han, Jyothish Pari, Samuel J. Gershman +1
Large Language Models (LLMs) have demonstrated impressive real-world utility, exemplifying artificial useful intelligence (AUI). However, their ability to reason adaptively and rob…
Efficient Diffusion Transformer Policies with Mixture of Expert Denoisers for Multitask Learning
Moritz Reuss, Jyothish Pari, Pulkit Agrawal +1
Diffusion Policies have become widely used in Imitation Learning, offering several appealing properties, such as generating multimodal and discontinuous behavior. As models are bec…
Collective Model Intelligence Requires Compatible Specialization
Jyothish Pari, Samy Jelassi, Pulkit Agrawal
In this work, we explore the limitations of combining models by averaging intermediate features, referred to as model merging, and propose a new direction for achieving collective…