11 citations · 16 across the 24 of their papers we have counts for
25 papers
MemWM: Memory-Augmented Text-Based World Model
Yujun Wang, Tao Zhang, Jinhe Bi +9
World models are increasingly used to support planning in agents by predicting how environment states evolve in response to agent actions. Yet fluent next-state predictions can sti…
NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics
Jian Lan, Zhicheng Liu, Xinpeng Wang +5
The ability to derive precise spatial and physical insights is a cornerstone of vision-language models (VLMs), yet their poor performances in related spatial intelligence tasks suc…
AUVIC: Adversarial Unlearning of Visual Concepts for Multi-modal Large Language Models
Haokun Chen, Jianing Li, Yao Zhang +4
Multimodal Large Language Models (MLLMs) achieve impressive performance once optimized on massive datasets. Such datasets often contain sensitive or copyrighted content, raising si…
GroundedPRM: Tree-Guided and Fidelity-Aware Process Reward Modeling for Step-Level Reasoning
Yao Zhang, Yu Wu, Haowei Zhang +6
Process Reward Models (PRMs) aim to improve multi-step reasoning in Large Language Models (LLMs) by supervising intermediate steps and identifying errors. However, building effecti…
Deep Research Agents Brings Deeper Harm
Shuo Chen, Zonggen Li, Xingyu Jin +8
We reveal that Deep Research (DR) agents systematically expose safety risks: simply submitting harmful queries that a standalone LLM would reject outright can elicit detailed and d…
Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrails
Shuo Chen, Zhen Han, Haokun Chen +6
Open-weight Large Reasoning Models (LRMs) are approaching the capabilities of their frontier counterparts but pose significant safety concerns, as they are difficult to patch or mo…
DreamPoster: A Unified Framework for Image-Conditioned Generative Poster Design
Xiwei Hu, Haokun Chen, Zhongqi Qi +4
We present DreamPoster, a Text-to-Image generation framework that intelligently synthesizes high-quality posters from user-provided images and text prompts while maintaining conten…
SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence
Yao Zhang, Chenyang Lin, Shijie Tang +4
The rapid progress of Large Language Models has advanced agentic systems in decision-making, coordination, and task execution. Yet, existing agentic system generation frameworks la…
FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
Yao Zhang, Hewei Gao, Haokun Chen +3
Multimodal Large Language Models (MLLMs) excel in tasks like multimodal reasoning and cross-modal retrieval but face deployment challenges in real-world scenarios due to distribute…
CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Process
Jinhe Bi, Danqi Yan, Yifan Wang +8
Recent Large Reasoning Models significantly improve the reasoning ability of Large Language Models by learning to reason, exhibiting the promising performance in solving complex ta…
Unveiling the "Fairness Seesaw": Discovering and Mitigating Gender and Race Bias in Vision-Language Models
Jian Lan, Udo Schlegel, Tanveer Hannan +3
Although Vision-Language Models (VLMs) have achieved remarkable success, the knowledge mechanisms underlying their social biases remain a black box, where fairness- and ethics-rela…
Does Machine Unlearning Truly Remove Knowledge?
Haokun Chen, Yueqi Zhang, Yuan Bi +9
In recent years, Large Language Models (LLMs) have achieved remarkable advancements, drawing significant attention from the research community. Their capabilities are largely attri…
EmoAgent: A Multi-Agent Framework for Diverse Affective Image Manipulation
Qi Mao, Haobo Hu, Yujie He +3
Affective Image Manipulation (AIM) aims to alter visual elements within an image to evoke specific emotional responses from viewers. However, existing AIM approaches rely on rigid…
Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Models
Haokun Chen, Sebastian Szyller, Weilin Xu +1
Large language models (LLMs) are trained using massive datasets, which often contain undesirable content such as harmful texts, personal information, and copyrighted material. To a…
LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering
Jinhe Bi, Yujun Wang, Haokun Chen +4
Multimodal Large Language Models (MLLMs) have significantly advanced visual tasks by integrating visual representations into large language models (LLMs). The textual modality, inh…
FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models
Haokun Chen, Hang Li, Yao Zhang +7
One-Shot Federated Learning (OSFL), a special decentralized machine learning paradigm, has recently gained significant attention. OSFL requires only a single round of client data o…
Towards Artwork Explanation in Large-scale Vision Language Models
Kazuki Hayashi, Yusuke Sakai, Hidetaka Kamigaito +2
Large-scale Vision-Language Models (LVLMs) output text from images and instructions, demonstrating capabilities in text generation and comprehension. However, it has not been clari…
Transformer as Linear Expansion of Learngene
Shiyu Xia, Miaosen Zhang, Xu Yang +3
We propose expanding the shared Transformer module to produce and initialize Transformers of varying depths, enabling adaptation to diverse resource constraints. Drawing an analogy…
Building Variable-sized Models via Learngene Pool
Boyu Shi, Shiyu Xia, Xu Yang +3
Recently, Stitchable Neural Networks (SN-Net) is proposed to stitch some pre-trained networks for quickly building numerous networks with different complexity and performance trade…
Manipulating the Label Space for In-Context Classification
Haokun Chen, Xu Yang, Yuhang Huang +3
After pre-training by generating the next word conditional on previous words, the Language Model (LM) acquires the ability of In-Context Learning (ICL) that can learn a new task co…
FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning
Haokun Chen, Yao Zhang, Denis Krompass +2
Recently, foundation models have exhibited remarkable advancements in multi-modal learning. These models, equipped with millions (or billions) of parameters, typically require a su…
FedPop: Federated Population-based Hyperparameter Tuning
Haokun Chen, Denis Krompass, Jindong Gu +1
Federated Learning (FL) is a distributed machine learning (ML) paradigm, in which multiple clients collaboratively train ML models without centralizing their local data. Similar to…
Exploring Diverse In-Context Configurations for Image Captioning
Xu Yang, Yongliang Wu, Mingzhuo Yang +2
After discovering that Language Models (LMs) can be good in-context few-shot learners, numerous strategies have been proposed to optimize in-context sequence configurations. Recent…
CL-CrossVQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering
Yao Zhang, Haokun Chen, Ahmed Frikha +5
Visual Question Answering (VQA) is a multi-discipline research task. To produce the right answer, it requires an understanding of the visual content of images, the natural language…