2 citations · 5 across the 12 of their papers we have counts for
12 papers
Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations
Zizhao Hu, Nathan Elijah Segura, Mohammad Rostami +1
Human input reaches language models by typing or speaking, and each channel leaves a distinct signature: orthographic noise for keyboards; for voice, disfluency from conventional t…
In-Context Collapse in Vision-Language Models and How to Mitigate it?
Mohammad Rostami
Many-shot in-context learning (ICL) lets vision-language models (VLMs) adapt from image--label demonstrations without weight updates, and is widely assumed to improve as more demon…
Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction
Priyashree Roy, Sujitha Martin, Mohammad Rostami +6
Intelligent document processing (IDP) with vision-language models (VLMs) hinges on confidence scores trustworthy enough to route extractions between automation and human review. Ex…
SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion
Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei +3
Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full…
Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISM
Zizhao Hu, Mohammad Rostami, Jesse Thomason
Persona prompting can steer LLM generation towards a domain-specific tone and pattern. This behavior enables use cases in multi-agent systems where diverse interactions are crucial…
Multi-modal Synthetic Data Training and Model Collapse: Insights from VLMs and Diffusion Models
Zizhao Hu, Mohammad Rostami, Jesse Thomason
Recent research has highlighted the risk of generative model collapse, where performance progressively degrades when continually trained on self-generated data. However, existing e…
Static Key Attention in Vision
Zizhao Hu, Xiaolin Zhou, Mohammad Rostami
The success of vision transformers is widely attributed to the expressive power of their dynamically parameterized multi-head self-attention mechanism. We examine the impact of sub…
Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning
Amin Banayeeanzade, Mahdi Soltanolkotabi, Mohammad Rostami
Multi-task learning (MTL) is a machine learning paradigm that aims to improve the generalization performance of a model on multiple related tasks by training it simultaneously on t…
Lateralization MLP: A Simple Brain-inspired Architecture for Diffusion
Zizhao Hu, Mohammad Rostami
The Transformer architecture has dominated machine learning in a wide range of tasks. The specific characteristic of this architecture is an expensive scaled dot-product attention…
An Intermediate Fusion ViT Enables Efficient Text-Image Alignment in Diffusion Models
Zizhao Hu, Shaochong Jia, Mohammad Rostami
Diffusion models have been widely used for conditional data cross-modal generation tasks such as text-to-image and text-to-video. However, state-of-the-art models still fail to ali…
Efficient Multimodal Diffusion Models Using Joint Data Infilling with Partially Shared U-Net
Zizhao Hu, Shaochong Jia, Mohammad Rostami
Recently, diffusion models have been used successfully to fit distributions for cross-modal data translation and multimodal data generation. However, these methods rely on extensiv…
Class-Incremental Learning Using Generative Experience Replay Based on Time-aware Regularization
Zizhao Hu, Mohammad Rostami
Learning new tasks accumulatively without forgetting remains a critical challenge in continual learning. Generative experience replay addresses this challenge by synthesizing pseud…