8 papers
Latent Denoising Improves Visual Alignment in Large Multimodal Models
Dhruv Parikh, Jacob Fein-Ashley, Rajgopal Kannan +1
Large Multimodal Models (LMMs) such as LLaVA are typically trained with an autoregressive language modeling objective, providing only indirect supervision to visual tokens. This of…
Mixture of Thoughts: Learning to Aggregate What Experts Think, Not Just What They Say
Jacob Fein-Ashley, Dhruv Parikh, Rajgopal Kannan +1
Open-source Large Language Models (LLMs) increasingly specialize by domain (e.g., math, code, general reasoning), motivating systems that leverage complementary strengths across mo…
SPECTRE: An FFT-Based Efficient Drop-In Replacement to Self-Attention for Long Contexts
Jacob Fein-Ashley, Neelesh Gupta, Rajgopal Kannan +1
Long-context transformers face significant efficiency challenges due to the quadratic cost of self-attention. However, many modern applications-from multi-turn dialogue to high-res…
Contextual Feedback Loops: Amplifying Deep Reasoning with Iterative Top-Down Feedback
Jacob Fein-Ashley, Rajgopal Kannan, Viktor Prasanna
Conventional deep networks rely on one-way backpropagation that overlooks reconciling high-level predictions with lower-level representations. We propose \emph{Contextual Feedback…
ClusterViG: Efficient Globally Aware Vision GNNs via Image Partitioning
Dhruv Parikh, Jacob Fein-Ashley, Tian Ye +2
Convolutional Neural Networks (CNN) and Vision Transformers (ViT) have dominated the field of Computer Vision (CV). Graph Neural Networks (GNN) have performed remarkably well acros…
Linear Diffusion Networks
Jacob Fein-Ashley
We present Linear Diffusion Networks (LDNs), a novel architecture that reinterprets sequential data processing as a unified diffusion process. Our model integrates adaptive diffusi…