256 citations · 280 across the 16 of their papers we have counts for
16 papers
MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
Qin Liu, Wenxuan Zhou, Nan Xu +5
One critical challenge for large language models (LLMs) for making complex reasoning is their reliance on matching reasoning patterns from training data, instead of proactively sel…
A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models
Zihao Lin, Samyadeep Basu, Mohammad Beigi +18
The rise of foundation models has transformed machine learning research, prompting efforts to uncover their inner workings and develop more efficient and reliable applications for…
VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gap
Qin Liu, Fei Wang, Chaowei Xiao +1
The emergence of vision language models (VLMs) comes with increased safety concerns, as the incorporation of multiple modalities heightens vulnerability to attacks. Although VLMs c…
MetaScientist: A Human-AI Synergistic Framework for Automated Mechanical Metamaterial Design
Jingyuan Qi, Zian Jia, Minqian Liu +15
The discovery of novel mechanical metamaterials, whose properties are dominated by their engineered structures rather than chemical composition, is a knowledge-intensive and resour…
Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models
Qin Liu, Chao Shang, Ling Liu +7
The safety alignment ability of Vision-Language Models (VLMs) is prone to be degraded by the integration of the vision module compared to its LLM backbone. We investigate this phen…
Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models
Tinghui Zhu, Qin Liu, Fei Wang +2
Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities for capturing and reasoning over multimodal inputs. However, these models are prone to parametric kno…
SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignment
Qin Liu, Fei Wang, Chaowei Xiao +1
Existing preference alignment is a one-size-fits-all alignment mechanism, where the part of the large language model (LLM) parametric knowledge with non-preferred features is unifo…
Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges
Qin Liu, Wenjie Mo, Terry Tong +4
The advancement of Large Language Models (LLMs) has significantly impacted various domains, including Web search, healthcare, and software development. However, as these models sca…
Familiarity-Aware Evidence Compression for Retrieval-Augmented Generation
Dongwon Jung, Qin Liu, Tenghao Huang +2
Retrieval-augmented generation (RAG) improves large language models (LMs) by incorporating non-parametric knowledge through evidence retrieved from external sources. However, it of…
MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding
Fei Wang, Xingyu Fu, James Y. Huang +18
We introduce MuirBench, a comprehensive benchmark that focuses on robust multi-image understanding capabilities of multimodal LLMs. MuirBench consists of 12 diverse multi-image tas…
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing
Jiangshu Du, Yibo Wang, Wenting Zhao +37
This work is motivated by two key trends. On one hand, large language models (LLMs) have shown remarkable versatility in various generative tasks such as writing, drawing, and ques…
Two Heads are Better than One: Nested PoE for Robust Defense Against Multi-Backdoors
Victoria Graf, Qin Liu, Muhao Chen
Data poisoning backdoor attacks can cause undesirable behaviors in large language models (LLMs), and defending against them is of increasing importance. Existing defense mechanisms…
Monotonic Paraphrasing Improves Generalization of Language Model Prompting
Qin Liu, Fei Wang, Nan Xu +3
Performance of large language models (LLMs) may vary with different prompts or instructions of even the same task. One commonly recognized factor for this phenomenon is the model's…
The Rise and Potential of Large Language Model Based Agents: A Survey
Zhiheng Xi, Wenxiang Chen, Xin Guo +26
For a long time, humanity has pursued artificial intelligence (AI) equivalent to or surpassing the human level, with AI agents considered a promising vehicle for this pursuit. AI a…
Secrets of RLHF in Large Language Models Part I: PPO
Rui Zheng, Shihan Dou, Songyang Gao +24
Large language models (LLMs) have formulated a blueprint for the advancement of artificial general intelligence. Its primary objective is to function as a human-centric (helpful, h…
From Shortcuts to Triggers: Backdoor Defense with Denoised PoE
Qin Liu, Fei Wang, Chaowei Xiao +1
Language models are often at risk of diverse backdoor attacks, especially data poisoning. Thus, it is important to investigate defense solutions for addressing them. Existing backd…