274 citations · 891 across the 147 of their papers we have counts for
149 papers
Teaching Language Models To Gather Information Proactively
Tenghao Huang, Sihao Chen, Muhao Chen +4
Large language models (LLMs) are increasingly expected to function as collaborative partners, engaging in back-and-forth dialogue to solve complex, ambiguous problems. However, cur…
Agentic Web: Weaving the Next Web with AI Agents
Yingxuan Yang, Mulei Ma, Yuxuan Huang +15
The emergence of AI agents powered by large language models (LLMs) marks a pivotal shift toward the Agentic Web, a new phase of the internet defined by autonomous, goal-driven inte…
DiscoSum: Discourse-aware News Summarization
Alexander Spangher, Tenghao Huang, Jialiang Gu +2
Recent advances in text summarization have predominantly leveraged large language models to generate concise summaries. However, language models often do not maintain long-term dis…
Is Extending Modality The Right Path Towards Omni-Modality?
Tinghui Zhu, Kai Zhang, Muhao Chen +1
Omni-modal language models (OLMs) aim to integrate and reason over diverse input modalities--such as text, images, video, and audio--while maintaining strong language capabilities.…
Code Execution as Grounded Supervision for LLM Reasoning
Dongwon Jung, Wenxuan Zhou, Muhao Chen
Training large language models (LLMs) with chain-of-thought (CoT) supervision has proven effective for enhancing their reasoning abilities. However, obtaining reliable and accurate…
QA-LIGN: Aligning LLMs through Constitutionally Decomposed QA
Jacob Dineen, Aswin RRV, Qin Liu +8
Alignment of large language models (LLMs) with principles like helpfulness, honesty, and harmlessness typically relies on scalar rewards that obscure which objectives drive the tra…
MetamatBench: Integrating Heterogeneous Data, Computational Tools, and Visual Interface for Metamaterial Discovery
Jianpeng Chen, Wangzhi Zhan, Haohui Wang +10
Metamaterials, engineered materials with architected structures across multiple length scales, offer unprecedented and tunable mechanical properties that surpass those of conventio…
LayerIF: Estimating Layer Quality for Large Language Models using Influence Functions
Hadi Askari, Shivanshu Gupta, Fei Wang +2
Pretrained Large Language Models (LLMs) achieve strong performance across a wide range of tasks, yet exhibit substantial variability in the various layers' training quality with re…
Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation
Hongxiang Zhang, Hao Chen, Muhao Chen +1
Recent decoding methods improve the factuality of large language models (LLMs) by refining how the next token is selected during generation. These methods typically operate at the…
QLIP: A Dynamic Quadtree Vision Prior Enhances MLLM Performance Without Retraining
Kyle R. Chickering, Bangzheng Li, Muhao Chen
Multimodal Large Language Models (MLLMs) encode images into visual tokens, aligning visual and textual signals within a shared latent space to facilitate crossmodal representation…
Diagnosing and Mitigating Modality Interference in Multimodal Large Language Models
Rui Cai, Bangzheng Li, Xiaofei Wen +2
Multimodal Large Language Models demonstrate strong performance on multimodal benchmarks, yet often exhibit poor robustness when exposed to spurious modality interference, such as…
MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
Qin Liu, Wenxuan Zhou, Nan Xu +5
One critical challenge for large language models (LLMs) for making complex reasoning is their reliance on matching reasoning patterns from training data, instead of proactively sel…
Semantic-Clipping: Efficient Vision-Language Modeling with Semantic-Guidedd Visual Selection
Bangzheng Li, Fei Wang, Wenxuan Zhou +5
Vision-Language Models (VLMs) leverage aligned visual encoders to transform images into visual tokens, allowing them to be processed similarly to text by the backbone large languag…
Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization
Shuyang Hao, Yiwei Wang, Bryan Hooi +4
In the realm of large vision-language models (LVLMs), adversarial jailbreak attacks serve as a red-teaming approach to identify safety vulnerabilities of these models and their ass…
MapQA: Open-domain Geospatial Question Answering on Map Data
Zekun Li, Malcolm Grossman, Eric +4
Geospatial question answering (QA) is a fundamental task in navigation and point of interest (POI) searches. While existing geospatial QA datasets exist, they are limited in both s…
BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
Terry Tong, Fei Wang, Zhe Zhao +1
This paper proposes a novel backdoor threat attacking the LLM-as-a-Judge evaluation regime, where the adversary controls both the candidate and evaluator model. The backdoored eval…
AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection
Weidi Luo, Shenghong Dai, Xiaogeng Liu +4
The rapid advancements in Large Language Models (LLMs) have enabled their deployment as autonomous agents for handling complex tasks in dynamic environments. These LLMs demonstrate…
VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gap
Qin Liu, Fei Wang, Chaowei Xiao +1
The emergence of vision language models (VLMs) comes with increased safety concerns, as the incorporation of multiple modalities heightens vulnerability to attacks. Although VLMs c…
ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails
Xiaofei Wen, Wenxuan Zhou, Wenjie Jacky Mo +1
Ensuring the safety of large language models (LLMs) is critical as they are deployed in real-world applications. Existing guardrails rely on rule-based filtering or single-pass cla…
R2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memory
Tenghao Huang, Kinjal Basu, Ibrahim Abdelaziz +3
The proliferation of web agents necessitates advanced navigation and interaction strategies within complex web environments. Current models often struggle with efficient navigation…
Unraveling Indirect In-Context Learning Using Influence Functions
Hadi Askari, Shivanshu Gupta, Terry Tong +3
In this work, we introduce a novel paradigm for generalized In-Context Learning (ICL), termed Indirect In-Context Learning. In Indirect ICL, we explore demonstration selection stra…
MetaScientist: A Human-AI Synergistic Framework for Automated Mechanical Metamaterial Design
Jingyuan Qi, Zian Jia, Minqian Liu +15
The discovery of novel mechanical metamaterials, whose properties are dominated by their engineered structures rather than chemical composition, is a knowledge-intensive and resour…
SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Models
Zilan Wang, Junfeng Guo, Jiacheng Zhu +4
Recent advances in large-scale text-to-image (T2I) diffusion models have enabled a variety of downstream applications, including style customization, subject-driven personalization…
FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attacks
Jiongxiao Wang, Fangzhou Wu, Wendi Li +5
Large language models (LLMs) have been widely deployed as the backbone with additional tools and text information for real-world applications. However, integrating external informa…