103 citations · 122 across the 18 of their papers we have counts for
19 papers
What Makes an Efficient VLA? Navigating Action-Head Design, Scaling, and Latency
Luoyang Sun, Guoyang Xia, Fengfa Li +9
Vision-Language-Action (VLA) models combine a pretrained vision encoder, a language backbone, and an action head, but their relative contribution has not been established under con…
VANE: Reliable Test-Time Training for Vision-Language-Action Models via Future Visual Representation Prediction
Hongjin Ji, Guoyang Xia, Luoyang Sun +2
Test-time training (TTT) offers a lightweight way to adapt vision--language--action (VLA) policies from unlabeled deployment streams, but it remains difficult to use reliably in cl…
VLAFlow: A Unified Training Framework for Vision-Language-Action Models via Co-training and Future Latent Alignment
Guoyang Xia, Fengfa Li, Hongjin Ji +4
Vision-language-action models (VLAs) have recently advanced robotic manipulation, yet the effects of different robot-data pre-training paradigms remain difficult to compare because…
FastMMoE: Accelerating Multimodal Large Language Models through Dynamic Expert Activation and Routing-Aware Token Pruning
Guoyang Xia, Yifeng Ding, Fengfa Li +4
Multimodal large language models (MLLMs) have achieved impressive performance, but high-resolution visual inputs result in long sequences of visual tokens and substantial inference…
ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference
Haojie Ouyang, Jianwei Lv, Lei Ren +3
Transformer-based large models excel in natural language processing and computer vision, but face severe computational inefficiencies due to the self-attention's quadratic complexi…
SMAR: Soft Modality-Aware Routing Strategy for MoE-based Multimodal Large Language Models Preserving Language Capabilities
Guoyang Xia, Yifeng Ding, Fengfa Li +4
Mixture of Experts (MoE) architectures have become a key approach for scaling large language models, with growing interest in extending them to multimodal tasks. Existing methods t…
Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark
Ziming Cheng, Binrui Xu, Lisheng Gong +14
With enhanced capabilities and widespread applications, Multimodal Large Language Models (MLLMs) are increasingly required to process and reason over multiple images simultaneously…
FreeGraftor: Training-Free Cross-Image Feature Grafting for Subject-Driven Text-to-Image Generation
Zebin Yao, Lei Ren, Huixing Jiang +4
Subject-driven image generation aims to synthesize novel scenes that faithfully preserve subject identity from reference images while adhering to textual guidance. However, existin…
Concept Conductor: Orchestrating Multiple Personalized Concepts in Text-to-Image Synthesis
Zebin Yao, Fangxiang Feng, Ruifan Li +1
The customization of text-to-image models has seen significant advancements, yet generating multiple personalized concepts remains a challenging task. Current methods struggle with…
DiffHarmony: Latent Diffusion Model Meets Image Harmonization
Pengfei Zhou, Fangxiang Feng, Xiaojie Wang
Image harmonization, which involves adjusting the foreground of a composite image to attain a unified visual consistency with the background, can be conceptualized as an image-to-i…
Whether you can locate or not? Interactive Referring Expression Generation
Fulong Ye, Yuxing Long, Fangxiang Feng +1
Referring Expression Generation (REG) aims to generate unambiguous Referring Expressions (REs) for objects in a visual scene, with a dual task of Referring Expression Comprehension…
GR-GAN: Gradual Refinement Text-to-image Generation
Bo Yang, Fangxiang Feng, Xiaojie Wang
A good Text-to-Image model should not only generate high quality images, but also ensure the consistency between the text and the generated image. Previous models failed to simulta…
Question-Driven Graph Fusion Network For Visual Question Answering
Yuxi Qian, Yuncong Hu, Ruonan Wang +2
Existing Visual Question Answering (VQA) models have explored various visual relationships between objects in the image to answer complex questions, which inevitably introduces irr…
Co-VQA : Answering by Interactive Sub Question Sequence
Ruonan Wang, Yuxi Qian, Fangxiang Feng +2
Most existing approaches to Visual Question Answering (VQA) answer questions directly, however, people usually decompose a complex question into a sequence of simple sub questions…
Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scene
Duo Zheng, Fandong Meng, Qingyi Si +5
Visual dialog has witnessed great progress after introducing various vision-oriented goals into the conversation, especially such as GuessWhich and GuessWhat, where the only image…
Multi-stage Pre-training over Simplified Multimodal Pre-training Models
Tongtong Liu, Fangxiang Feng, Xiaojie Wang
Multimodal pre-training models, such as LXMERT, have achieved excellent results in downstream tasks. However, current pre-trained models require large amounts of training data and…
Answer-Driven Visual State Estimator for Goal-Oriented Visual Dialogue
Zipeng Xu, Fangxiang Feng, Xiaojie Wang +3
A goal-oriented visual dialogue involves multi-turn interactions between two agents, Questioner and Oracle. During which, the answer given by Oracle is of great significance, as it…
Constructing Hierarchical Image-tags Bimodal Representations for Word Tags Alternative Choice
Fangxiang Feng, Ruifan Li, Xiaojie Wang
This paper describes our solution to the multi-modal learning challenge of ICML. This solution comprises constructing three-level representations in three consecutive stages and ch…
Challenges in Representation Learning: A report on three machine learning contests
Ian J. Goodfellow, Dumitru Erhan, Pierre Luc Carrier +25
The ICML 2013 Workshop on Challenges in Representation Learning focused on three challenges: the black box learning challenge, the facial expression recognition challenge, and the…