2 citations · 4 across the 12 of their papers we have counts for
12 papers
Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations
Zizhao Hu, Nathan Elijah Segura, Mohammad Rostami +1
Human input reaches language models by typing or speaking, and each channel leaves a distinct signature: orthographic noise for keyboards; for voice, disfluency from conventional t…
SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion
Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei +3
Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full…
Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISM
Zizhao Hu, Mohammad Rostami, Jesse Thomason
Persona prompting can steer LLM generation towards a domain-specific tone and pattern. This behavior enables use cases in multi-agent systems where diverse interactions are crucial…
Multi-modal Synthetic Data Training and Model Collapse: Insights from VLMs and Diffusion Models
Zizhao Hu, Mohammad Rostami, Jesse Thomason
Recent research has highlighted the risk of generative model collapse, where performance progressively degrades when continually trained on self-generated data. However, existing e…
Static Key Attention in Vision
Zizhao Hu, Xiaolin Zhou, Mohammad Rostami
The success of vision transformers is widely attributed to the expressive power of their dynamically parameterized multi-head self-attention mechanism. We examine the impact of sub…
Lateralization MLP: A Simple Brain-inspired Architecture for Diffusion
Zizhao Hu, Mohammad Rostami
The Transformer architecture has dominated machine learning in a wide range of tasks. The specific characteristic of this architecture is an expensive scaled dot-product attention…
An Intermediate Fusion ViT Enables Efficient Text-Image Alignment in Diffusion Models
Zizhao Hu, Shaochong Jia, Mohammad Rostami
Diffusion models have been widely used for conditional data cross-modal generation tasks such as text-to-image and text-to-video. However, state-of-the-art models still fail to ali…
Efficient Multimodal Diffusion Models Using Joint Data Infilling with Partially Shared U-Net
Zizhao Hu, Shaochong Jia, Mohammad Rostami
Recently, diffusion models have been used successfully to fit distributions for cross-modal data translation and multimodal data generation. However, these methods rely on extensiv…
Class-Incremental Learning Using Generative Experience Replay Based on Time-aware Regularization
Zizhao Hu, Mohammad Rostami
Learning new tasks accumulatively without forgetting remains a critical challenge in continual learning. Generative experience replay addresses this challenge by synthesizing pseud…
Cognitively Inspired Cross-Modal Data Generation Using Diffusion Models
Zizhao Hu, Mohammad Rostami
Most existing cross-modal generative methods based on diffusion models use guidance to provide control over the latent space to enable conditional generation across different modal…
Evaluating NLP Systems On a Novel Cloze Task: Judging the Plausibility of Possible Fillers in Instructional Texts
Zizhao Hu, Ravikiran Chanumolu, Xingyu Lin +2
Cloze task is a widely used task to evaluate an NLP system's language understanding ability. However, most of the existing cloze tasks only require NLP systems to give the relative…
GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL
Sumedh A Sontakke, Stephen Iota, Zizhao Hu +3
Out-of-distribution (OOD) detection is a well-studied topic in supervised learning. Extending the successes in supervised learning methods to the reinforcement learning (RL) settin…