3 citations · 4 across the 11 of their papers we have counts for
12 papers
Distilling Visual Reasoning into Text Space
Wenhan Yang, Nilay Naharas, Ali Payani +1
Large Vision-Language Models (LVLMs) have shown strong promise for multimodal reasoning, yet often struggle with tasks requiring concepts beyond what is directly observable in the…
AdaMEM: Test-Time Adaptive Memory for Language Agents
Yunxiang Zhang, Yiheng Li, Ali Payani +1
A central challenge for language agents is utilizing past experience to adapt to dynamic test-time conditions. While recent work demonstrates the promise of agentic memory mechanis…
Predicting Plasticity in Deep Continual Learning: A Theoretical Perspective
Jiuqi Wang, Jayanth Srinivasa, Claire Chen +3
Deep continual learning requires models to adapt to new tasks without retraining from scratch. However, neural networks can lose their ability to adapt to new tasks after training…
Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation
Haiyan Zhao, Zirui He, Guanchu Wang +3
Activation verbalization explains hidden representations in natural language, but existing methods are mostly limited to self-explanation, where each model explains only its own ac…
D3-Gym: Constructing Real-World Verifiable Environments for Data-Driven Discovery
Hanane Nour Moussa, Yifei Li, Zhuoyang Li +7
Despite recent progress in language models and agents for scientific data-driven discovery, advancing their capabilities is held back by the absence of verifiable environments repr…
Halo: Domain-Aware Query Optimization for Long-Context Question Answering
Pramod Chunduri, Francisco Romero, Ali Payani +2
Long-context question answering (QA) over lengthy documents is critical for applications such as financial analysis, legal review, and scientific research. Current approaches, such…
Balancing Faithfulness and Performance in Reasoning via Multi-Listener Soft Execution
Nithin Sivakumaran, Shoubin Yu, Hyunji Lee +4
Chain-of-thought (CoT) reasoning sometimes fails to faithfully reflect the true computation of a large language model (LLM), hampering its utility in explaining how LLMs arrive at…
Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity
Dang Nguyen, Ali Payani, Baharan Mirzasoleiman
Hallucination in large language models (LLMs) can be detected by assessing the uncertainty of model outputs, typically measured using entropy. Semantic entropy (SE) enhances tradit…
Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gap
Wenhan Yang, Spencer Stice, Ali Payani +1
Ensuring Vision-Language Models (VLMs) generate safe outputs is crucial for their reliable deployment. However, LVLMs suffer from drastic safety degradation compared to their LLM b…
Effective Guidance for Model Attention with Simple Yes-no Annotations
Seongmin Lee, Ali Payani, Duen Horng Chau
Modern deep learning models often make predictions by focusing on irrelevant areas, leading to biased performance and limited generalization. Existing methods aimed at rectifying m…
Data-Efficient Contrastive Language-Image Pretraining: Prioritizing Data Quality over Quantity
Siddharth Joshi, Arnav Jain, Ali Payani +1
Contrastive Language-Image Pre-training (CLIP) on large-scale image-caption datasets learns representations that can achieve remarkable zero-shot generalization. However, such mode…
Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddings
Yihao Xue, Ali Payani, Yu Yang +1
Pretrained machine learning models need to be adapted to distribution shifts when deployed in new target environments. When obtaining labeled data from the target distribution is e…