5 papers
Skip-It? Theoretical Conditions for Layer Skipping in Vision-Language Models
Max Hartman, Vidhata Jayaraman, Moulik Choraria +2
Vision-language models achieve incredible performance across a wide range of tasks, but their large size makes inference costly. Recent work has shown that multimodal processing co…
Energy-Aware Routing to Large Reasoning Models
Austin R. Ellis-Mohr, Max Hartman, Lav R. Varshney
Large reasoning models (LRMs) have heterogeneous inference energy costs based on which model is used and how much it reasons. To reduce energy, it is important to choose the right…
Federated Nonlinear System Identification
Omkar Tupe, Max Hartman, Lav R. Varshney +1
We consider federated learning of linearly-parameterized nonlinear systems. We establish theoretical guarantees on the effectiveness of federated nonlinear system identification co…
SparseJEPA: Sparse Representation Learning of Joint Embedding Predictive Architectures
Max Hartman, Lav Varshney
Joint Embedding Predictive Architectures (JEPA) have emerged as a powerful framework for learning general-purpose representations. However, these models often lack interpretability…
SwitchCIT: Switching for Continual Instruction Tuning
Xinbo Wu, Max Hartman, Vidhata Arjun Jayaraman +1
Large language models (LLMs) and multimodal models (MMs) have exhibited impressive capabilities in various domains, particularly in general language understanding and visual reason…