11 citations · 50 across the 25 of their papers we have counts for
28 papers
Long-Term Memory-Guided Enhancement for Target Perception in Audio-Language Models
Zhenhong Zhou, Xuanyue Zhao, Youji Liu +4
Audio large language models (ALLMs) can reason about the content of audio recordings to perform complex tasks. However, these capabilities usually collapse in real-world environmen…
COMET: Contrastive Motion-Enhanced Temporal Reasoning for Video Multimodal Large Language Models
Chenghua Zhu, Zhaolu Kang, Qifan Shi +8
Video multimodal large language models have advanced significantly, yet fine-grained motion-temporal understanding remains fragile. The core bottleneck is not only sparse frame sam…
CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughts
Lianyu Hu, Shengqian Qin, Zeqin Liao +4
Chain-of-thought (CoT) reasoning has enabled multi-modal large language models (MLLMs) to tackle complex visual reasoning tasks by generating explicit intermediate reasoning steps…
PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning
Wenyuan Wang, Lianyu Hu, Hao Wang +1
Video-language models (VLMs) have achieved remarkable performance on video understanding and visual question answering, yet they remain unreliable in reasoning about physical plaus…
TacReasoner: A Dynamic Tactile-Language Framework for Interactive Reasoning in Real-World Scenarios
Kailin Lyu, Di Wu, Long Xiao +7
Among the five primary human senses, tactile is arguably the most fundamental to survival, as it enables the perception of physical contact and interaction in real-world environmen…
TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding
Lianyu Hu, Xiaoyu Ma, Zeqin Liao +1
Chain-of-thought (CoT) reasoning has proven effective for enhancing problem-solving in large language models. However, when applied to multimodal LLMs (MLLMs), existing CoT approac…
PhotoCraft: Agentic Reasoning with Hierarchical Self-Evolving Memory for Deep Image Search
Kailin Lyu, Zhiqiang Yuan, Jianwei He +9
Deep Image Search requires multi-step reasoning over rich contextual cues, such as time, location, and event relations. However, most existing LLM-based agents are stateless and re…
TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representation
Kailin Lyu, Di Wu, Pengwei Zhang +12
Touch is a key modality for embodied agents to understand the physical world. Although recent work has incorporated tactile signals into language systems for tactile commonsense re…
VLAMotor: Test-Guided Enhancement of Vision-Language-Action Models via Agent-BasedData Synthesis
Zeqin Liao, Peifan Ren, Zixu Gao +6
Vision-Language-Action (VLA) models follow a data-driven paradigm and are constrained by the coverage of training data, making them prone to failure on edge-case configurations aft…
Reading Between the Pixels: An Inscriptive Jailbreak Attack on Text-to-Image Models
Zonghao Ying, Haowen Dai, Lianyu Hu +5
Modern text-to-image (T2I) models can now render legible, paragraph-length text, enabling a fundamentally new class of misuse. We identify and formalize the inscriptive jailbreak,…
TennisExpert: Towards Expert-Level Analytical Sports Video Understanding
Zhaoyu Liu, Xi Weng, Lianyu Hu +4
Tennis is one of the most widely followed sports, generating extensive broadcast footage with strong potential for professional analysis, automated coaching, and real-time commenta…
BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planning
Xiaoyu Ma, Lianyu Hu, Wenbing Tang +4
Embodied task planning requires agents to execute long-horizon, goal-directed actions in complex 3D environments, where success depends on both immediate perception and accumulated…
HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory System
Kailin Lyu, Kangyi Wu, Pengna Li +9
LLM-based agents have demonstrated impressive zero-shot performance in vision-language navigation (VLN) tasks. However, most zero-shot methods primarily rely on closed-source LLMs…
SSL-SSAW: Self-Supervised Learning with Sigmoid Self-Attention Weighting for Question-Based Sign Language Translation
Zekang Liu, Wei Feng, Fanhua Shang +3
Sign Language Translation (SLT) bridges the communication gap between deaf people and hearing people, where dialogue provides crucial contextual cues to aid in translation. Buildin…
LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compression
Lianyu Hu, Fanhua Shang, Wei Feng +1
In this paper, we introduce LightVLM, a simple but effective method that can be seamlessly deployed upon existing Vision-Language Models (VLMs) to greatly accelerate the inference…
iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models
Lianyu Hu, Liqing Gao, Fanhua Shang +2
Recent methods have made notable progress in accelerating Large Vision-Language Models (LVLMs) by exploiting the inherent redundancy in visual inputs. Most existing approaches, how…
Deep Correlated Prompting for Visual Recognition with Missing Modalities
Lianyu Hu, Tongkai Shi, Wei Feng +2
Large-scale multimodal models have shown excellent performance over a series of tasks powered by the large corpus of paired multimodal training data. Generally, they are always ass…
Pose-Guided Fine-Grained Sign Language Video Generation
Tongkai Shi, Lianyu Hu, Fanhua Shang +3
Sign language videos are an important medium for spreading and learning sign language. However, most existing human image synthesis methods produce sign language images with detail…
CorrNet+: Sign Language Recognition and Translation via Spatial-Temporal Correlation
Lianyu Hu, Wei Feng, Liqing Gao +2
In sign language, the conveyance of human body trajectories predominantly relies upon the coordinated movements of hands and facial expressions across successive frames. Despite th…
Improving Continuous Sign Language Recognition with Adapted Image Models
Lianyu Hu, Tongkai Shi, Liqing Gao +2
The increase of web-scale weakly labelled image-text pairs have greatly facilitated the development of large-scale vision-language models (e.g., CLIP), which have shown impressive…
Dynamic Spatial-Temporal Aggregation for Skeleton-Aware Sign Language Recognition
Lianyu Hu, Liqing Gao, Zekang Liu +1
Skeleton-aware sign language recognition (SLR) has gained popularity due to its ability to remain unaffected by background information and its lower computational requirements. Cur…
COMMA: Co-Articulated Multi-Modal Learning
Lianyu Hu, Liqing Gao, Zekang Liu +2
Pretrained large-scale vision-language models such as CLIP have demonstrated excellent generalizability over a series of downstream tasks. However, they are sensitive to the variat…
AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognition
Lianyu Hu, Liqing Gao, Zekang Liu +2
Raw videos have been proven to own considerable feature redundancy where in many cases only a portion of frames can already meet the requirements for accurate recognition. In this…
Continuous Sign Language Recognition with Correlation Network
Lianyu Hu, Liqing Gao, Zekang Liu +1
Human body trajectories are a salient cue to identify actions in the video. Such body trajectories are mainly conveyed by hands and face across consecutive frames in sign language.…