12 citations · 23 across the 8 of their papers we have counts for
11 papers
Action-Consequence Alignment for Reliable Planning and Self-Improving in Latent World Models
Jinping Wang1, Zhiqiang Gao, Xiantong Zhen +1
Latent world models learn to predict observed transitions, yet low prediction error alone does not guarantee reliable planning. Inspired by self tickling experiments in neuroscienc…
Beyond One-Step Accuracy: State-Affine Latent Transition for Reliable Visual Planning
Boyuan Zhang, Yingjun Du, Xiantong Zhen +1
Joint-embedding world models enable visual planning by learning action-conditioned dynamics in latent space. Yet they are commonly trained for one-step prediction on encoded states…
See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs
Yuqing Lei, Wenbo Lyu, Yingjun Du +3
Large Vision-Language Models (LVLMs) excel at multimodal tasks but remain prone to object hallucinations. Prior training-free remedies often uniformly strengthen visual signals, wh…
MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models
Yuqing Lei, Yingjun Du, Yawen Huang +2
Vision-language models (VLMs) such as CLIP exhibit strong zero-shot generalization but remain sensitive to domain shifts at test time. Test-time prompt tuning (TPT) mitigates this…
VisChainBench: A Benchmark for Multi-Turn, Multi-Image Visual Reasoning Beyond Language Priors
Wenbo Lyu, Yingjun Du, Jinglin Zhao +2
Understanding multi-image, multi-turn scenarios is a critical yet underexplored capability for Large Vision-Language Models (LVLMs). Existing benchmarks predominantly focus on stat…
Variational Task Vector Composition
Boyuan Zhang, Yingjun Du, Xiantong Zhen +1
Task vectors capture how a model changes during fine-tuning by recording the difference between pre-trained and task-specific weights. The composition of task vectors, a key operat…
Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation
Yuchen Yang, Yingdong Shi, Cheems Wang +3
Fine-tuning pretrained large models to downstream tasks is an important problem, which however suffers from huge memory overhead due to large-scale parameters. This work strives to…
Episodic Multi-Task Learning with Heterogeneous Neural Processes
Jiayi Shen, Xiantong Zhen, Qi +2
This paper focuses on the data-insufficiency problem in multi-task learning within an episodic training setup. Specifically, we explore the potential of heterogeneous information a…
Knowledge Graph Embeddings for Multi-Lingual Structured Representations of Radiology Reports
Tom van Sonsbeek, Xiantong Zhen, Marcel Worring
The way we analyse clinical texts has undergone major changes over the last years. The introduction of language models such as BERT led to adaptations for the (bio)medical domain l…
Variational Multi-Task Learning with Gumbel-Softmax Priors
Jiayi Shen, Xiantong Zhen, Marcel Worring +1
Multi-task learning aims to explore task relatedness to improve individual tasks, which is of particular significance in the challenging scenario that only limited data is availabl…
Learning to Learn Kernels with Variational Random Features
Xiantong Zhen, Haoliang Sun, Yingjun Du +4
In this work, we introduce kernels with random Fourier features in the meta-learning framework to leverage their strong few-shot learning ability. We propose meta variational rando…