13 papers
Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges
Tuo Liang, Zhe Hu, Disheng Liu +2
Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, shared cultural knowledge, and communic…
ARGUS: Theory-of-Mind Guided Argument Generation with Strategy-Aware Planning and Knowledge Grounding
Zhe Hu
Persuasive argument generation requires modeling audience beliefs, rhetorical strategies, and factual grounding. Despite recent advancements, existing methods remain largely audien…
VIABLE: A Visually Impaired Assistance Benchmark for VLM-as-a-Judge Evaluation
Yi Zhao, Siqi Wang, Zhe Hu +2
AI-based Visually Impaired Assistance (VIA) remains challenging, largely due to the high cost of human evaluation. The VLM-as-a-Judge paradigm may offer a promising alternative, al…
ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response
Xiaomeng Zhu, Fengming Zhu, Weijie Zhou +8
While passive agents merely follow instructions, proactive agents align with higher-level objectives, such as assistance and safety by continuously monitoring the environment to de…
VIVA+: Human-Centered Situational Decision-Making
Zhe Hu, Yixiao Ren, Guanzhong Liu +2
Multimodal Large Language Models (MLLMs) show promising results for embodied agents in operating meaningfully in complex, human-centered environments. Yet, evaluating their capacit…
Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use
Yabo Zhang, Yihan Zeng, Qingyun Li +3
Large language models (LLMs) have demonstrated strong capabilities in language understanding and reasoning, yet they remain limited when tackling real-world tasks that require up-t…
When 'YES' Meets 'BUT': Can Large Models Comprehend Contradictory Humor Through Comparative Reasoning?
Tuo Liang, Zhe Hu, Jing Li +8
Understanding humor-particularly when it involves complex, contradictory narratives that require comparative reasoning-remains a significant challenge for large vision-language mod…
Praxis-VLM: Vision-Grounded Decision Making via Text-Driven Reinforcement Learning
Zhe Hu, Jing Li, Zhongzhu Pu +2
Vision Language Models exhibit impressive performance for various tasks, yet they often lack the sophisticated situational reasoning required for complex decision-making. This pape…
Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data
Shuhao Gu, Jialing Zhang, Siyuan Zhou +23
Recently, Vision-Language Models (VLMs) have achieved remarkable progress in multimodal tasks, and multimodal instruction data serves as the foundation for enhancing VLM capabiliti…
Language-Augmented Symbolic Planner for Open-World Task Planning
Guanqi Chen, Lei Yang, Ruixing Jia +5
Enabling robotic agents to perform complex long-horizon tasks has been a long-standing goal in robotics and artificial intelligence (AI). Despite the potential shown by large langu…
VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Values
Zhe Hu, Yixiao Ren, Jing Li +1
Large vision language models (VLMs) have demonstrated significant potential for integration into daily life, making it crucial for them to incorporate human values when making deci…
Debate-to-Write: A Persona-Driven Multi-Agent Framework for Diverse Argument Generation
Zhe Hu, Hou Pong Chan, Jing Li +1
Writing persuasive arguments is a challenging task for both humans and machines. It entails incorporating high-level beliefs from various perspectives on the topic, along with deli…
Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictions
Zhe Hu, Tuo Liang, Jing Li +5
Recent advancements in large multimodal language models have demonstrated remarkable proficiency across a wide range of tasks. Yet, these models still struggle with understanding t…