2 citations · 7 across the 37 of their papers we have counts for
25 papers · 1 filter
Are Your Agents Upward Deceivers?
Dadi Guo, Qingyu Liu, Dongrui Liu +13
Large Language Model (LLM)-based agents are increasingly used as autonomous subordinates that carry out tasks for users. This raises the question of whether they may also engage in…
AI Deception: Risks, Dynamics, and Controls
Boyuan Chen, Sitong Fang, Jiaming Ji +56
As intelligence increases, so does its shadow. AI deception, in which systems induce false beliefs to secure self-beneficial outcomes, has evolved from a speculative concern to an…
VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models
Borong Zhang, Jiahao Li, Jiachen Shen +8
While Vision-Language-Action models (VLAs) are rapidly advancing toward generalist robot policies, quantitatively characterizing their capability boundaries and failure modes remai…
Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Models
Sitong Fang, Shiyi Hou, Kaile Wang +6
Are frontier AI systems becoming more capable? Certainly. Yet such progress is not an unalloyed blessing but rather a Trojan horse: behind their performance leaps lie more insidiou…
SafeMT: Multi-turn Safety for Multimodal Language Models
Han Zhu, Juntao Dai, Jiaming Ji +8
With the widespread use of multi-modal Large Language models (MLLMs), safety issues have become a growing concern. Multi-turn dialogues, which are more common in everyday interacti…
RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbations
Jianing Guo, Zhenhong Wu, Chang Tu +13
In Vision-Language-Actionf(VLA) models, robustness to real-world perturbations is critical for deployment. Existing methods target simple visual disturbances, overlooking the broad…
A Good Plan is Hard to Find: Aligning Models with Preferences is Misaligned with What Helps Users
Nishant Balepur, Matthew Shu, Yoo Yeon Sung +5
To assist users in complex tasks, LLMs generate plans: step-by-step instructions towards a goal. While alignment methods aim to ensure LLM plans are helpful, they train (RLHF) or e…
A Game-Theoretic Negotiation Framework for Cross-Cultural Consensus in LLMs
Guoxi Zhang, Jiawei Chen, Tianzhuo Yang +3
The increasing prevalence of large language models (LLMs) is influencing global value systems. However, these models frequently exhibit a pronounced WEIRD (Western, Educated, Indus…
LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning
Weijie Shi, Han Zhu, Jiaming Ji +7
Legal judgment prediction (LJP) aims to function as a judge by making final rulings based on case claims and facts, which plays a vital role in the judicial domain for supporting c…
SafeLawBench: Towards Safe Alignment of Large Language Models
Chuxue Cao, Han Zhu, Jiaming Ji +7
With the growing prevalence of large language models (LLMs), the safety of LLMs has raised significant concerns. However, there is still a lack of definitive standards for evaluati…
FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation
Junyu Luo, Zhizhuo Kou, Liming Yang +10
Multimodal Large Language Models (MLLMs) have experienced rapid development in recent years. However, in the financial domain, there is a notable lack of effective and specialized…
Mitigating Deceptive Alignment via Self-Monitoring
Jiaming Ji, Wenqi Chen, Kaile Wang +8
Modern large language models rely on chain-of-thought (CoT) reasoning to achieve impressive performance, yet the same mechanism can amplify deceptive alignment, situations in which…
Generative RLHF-V: Learning Principles from Multi-modal Human Preference
Jiayi Zhou, Jiaming Ji, Boyuan Chen +6
Training multi-modal large language models (MLLMs) that align with human intentions is a long-term challenge. Traditional score-only reward models for alignment suffer from low acc…
J1: Exploring Simple Test-Time Scaling for LLM-as-a-Judge
Chi-Min Chan, Chunpu Xu, Jiaming Ji +7
The current focus of AI research is shifting from emphasizing model training towards enhancing evaluation quality, a transition that is crucial for driving further advancements in…
InterMT: Multi-Turn Interleaved Preference Alignment with Human Feedback
Boyuan Chen, Donghai Hong, Jiaming Ji +12
As multimodal large models (MLLMs) continue to advance across challenging tasks, a key question emerges: What essential capabilities are still missing? A critical aspect of human l…
When Slower Isn't Truer: Inverse Scaling Law of Truthfulness in Multimodal Reasoning
Sitong Fang, Wenjing Cao, Jiahao Li +7
Reasoning models have attracted increasing attention for their ability to tackle complex tasks, embodying the System II (slow thinking) paradigm in contrast to System I (fast, intu…
RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models
Bang An, Shiyue Zhang, Mark Dredze
Efforts to ensure the safety of large language models (LLMs) include safety fine-tuning, evaluation, and red teaming. However, despite the widespread use of the Retrieval-Augmented…
Benchmarking Multi-National Value Alignment for Large Language Models
Weijie Shi, Chengyi Ju, Chengzhong Liu +8
Do Large Language Models (LLMs) hold positions that conflict with your country's values? Occasionally they do! However, existing works primarily focus on ethical reviews, failing t…
A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
Kun Wang, Guibin Zhang, Zhenhong Zhou +100
The remarkable success of Large Language Models (LLMs) has illuminated a promising pathway toward achieving Artificial General Intelligence for both academic and industrial communi…
Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback
Jiaming Ji, Xinyu Chen, Rui Pan +13
Multimodal large language models (MLLMs) are essential for building general-purpose AI assistants; however, they pose increasing safety risks. How can we ensure safety alignment of…
ThinkPatterns-21k: A Systematic Study on the Impact of Thinking Patterns in LLMs
Pengcheng Wen, Jiaming Ji, Chi-Min Chan +5
Large language models (LLMs) have demonstrated enhanced performance through the \textit{Thinking then Responding} paradigm, where models generate internal thoughts before final res…
SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning
Borong Zhang, Yuhao Zhang, Jiaming Ji +5
Vision-language-action models (VLAs) show potential as generalist robot policies. However, these models pose extreme safety challenges during real-world deployment, including the r…
SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Hantao Lou, Changye Li, Jiaming Ji +1
With the integration of image modality, the semantic space of multimodal large language models (MLLMs) is more complex than text-only models, making their interpretability more cha…
RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?
Haotian Xu, Xing Wu, Weinong Wang +11
Can scaling transform reasoning? In this work, we explore the untapped potential of scaling Long Chain-of-Thought (Long-CoT) data to 1000k samples, pioneering the development of a…