6 papers
HarnessTTT: Learn to Optimize Harness at Test-Time for Scientific Discovery
Yingzi Ma, Yulong Cao, Nanxi Li +5
Scientific discovery with language models requires finding the best valid solution within a finite computational budget. We study how feedback from this search can improve the proc…
EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents
Nanxi Li, Yingzi Ma, Yulong Cao +7
Large language model (LLM) agents can translate natural-language requests into real-world actions, creating safety risks from both indirect prompt injections and directly harmful r…
LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails
Nanxi Li, Zhengyue Zhao, Chaowei Xiao
Guardrails are a critical safety layer for modern AI systems, but their operating regime is changing. As LLMs are deployed as customized assistants, safety policies are increasingl…
Beyond Static Vision: Scene Dynamic Field Unlocks Intuitive Physics Understanding in Multi-modal Large Language Models
Nanxi Li, Xiang Wang, Yuanjie Chen +3
While Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in image and video understanding, their ability to comprehend the physical world has become…
PRISM: Robust VLM Alignment with Principled Reasoning for Integrated Safety in Multimodality
Nanxi Li, Zhengyue Zhao, G. Edward Suh +2
Safeguarding vision-language models (VLMs) is a critical challenge, as existing methods often suffer from over-defense, which harms utility, or rely on shallow alignment, failing t…
The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMs
Hong Li, Nanxi Li, Yuanjie Chen +4
Multi-modal Large Language Models (MLLMs) have exhibited impressive capability. However, recently many deficiencies of MLLMs have been found compared to human intelligence, $\texti…