11 citations · 50 across the 25 of their papers we have counts for
20 papers · 1 filter
COMET: Contrastive Motion-Enhanced Temporal Reasoning for Video Multimodal Large Language Models
Chenghua Zhu, Zhaolu Kang, Qifan Shi +8
Video multimodal large language models have advanced significantly, yet fine-grained motion-temporal understanding remains fragile. The core bottleneck is not only sparse frame sam…
CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughts
Lianyu Hu, Shengqian Qin, Zeqin Liao +4
Chain-of-thought (CoT) reasoning has enabled multi-modal large language models (MLLMs) to tackle complex visual reasoning tasks by generating explicit intermediate reasoning steps…
TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding
Lianyu Hu, Xiaoyu Ma, Zeqin Liao +1
Chain-of-thought (CoT) reasoning has proven effective for enhancing problem-solving in large language models. However, when applied to multimodal LLMs (MLLMs), existing CoT approac…
Reading Between the Pixels: An Inscriptive Jailbreak Attack on Text-to-Image Models
Zonghao Ying, Haowen Dai, Lianyu Hu +5
Modern text-to-image (T2I) models can now render legible, paragraph-length text, enabling a fundamentally new class of misuse. We identify and formalize the inscriptive jailbreak,…
TennisExpert: Towards Expert-Level Analytical Sports Video Understanding
Zhaoyu Liu, Xi Weng, Lianyu Hu +4
Tennis is one of the most widely followed sports, generating extensive broadcast footage with strong potential for professional analysis, automated coaching, and real-time commenta…
HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory System
Kailin Lyu, Kangyi Wu, Pengna Li +9
LLM-based agents have demonstrated impressive zero-shot performance in vision-language navigation (VLN) tasks. However, most zero-shot methods primarily rely on closed-source LLMs…
LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compression
Lianyu Hu, Fanhua Shang, Wei Feng +1
In this paper, we introduce LightVLM, a simple but effective method that can be seamlessly deployed upon existing Vision-Language Models (VLMs) to greatly accelerate the inference…
iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models
Lianyu Hu, Liqing Gao, Fanhua Shang +2
Recent methods have made notable progress in accelerating Large Vision-Language Models (LVLMs) by exploiting the inherent redundancy in visual inputs. Most existing approaches, how…
Deep Correlated Prompting for Visual Recognition with Missing Modalities
Lianyu Hu, Tongkai Shi, Wei Feng +2
Large-scale multimodal models have shown excellent performance over a series of tasks powered by the large corpus of paired multimodal training data. Generally, they are always ass…
Pose-Guided Fine-Grained Sign Language Video Generation
Tongkai Shi, Lianyu Hu, Fanhua Shang +3
Sign language videos are an important medium for spreading and learning sign language. However, most existing human image synthesis methods produce sign language images with detail…
CorrNet+: Sign Language Recognition and Translation via Spatial-Temporal Correlation
Lianyu Hu, Wei Feng, Liqing Gao +2
In sign language, the conveyance of human body trajectories predominantly relies upon the coordinated movements of hands and facial expressions across successive frames. Despite th…
Improving Continuous Sign Language Recognition with Adapted Image Models
Lianyu Hu, Tongkai Shi, Liqing Gao +2
The increase of web-scale weakly labelled image-text pairs have greatly facilitated the development of large-scale vision-language models (e.g., CLIP), which have shown impressive…
Dynamic Spatial-Temporal Aggregation for Skeleton-Aware Sign Language Recognition
Lianyu Hu, Liqing Gao, Zekang Liu +1
Skeleton-aware sign language recognition (SLR) has gained popularity due to its ability to remain unaffected by background information and its lower computational requirements. Cur…
COMMA: Co-Articulated Multi-Modal Learning
Lianyu Hu, Liqing Gao, Zekang Liu +2
Pretrained large-scale vision-language models such as CLIP have demonstrated excellent generalizability over a series of downstream tasks. However, they are sensitive to the variat…
AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognition
Lianyu Hu, Liqing Gao, Zekang Liu +2
Raw videos have been proven to own considerable feature redundancy where in many cases only a portion of frames can already meet the requirements for accurate recognition. In this…
Continuous Sign Language Recognition with Correlation Network
Lianyu Hu, Liqing Gao, Zekang Liu +1
Human body trajectories are a salient cue to identify actions in the video. Such body trajectories are mainly conveyed by hands and face across consecutive frames in sign language.…
Self-Emphasizing Network for Continuous Sign Language Recognition
Lianyu Hu, Liqing Gao, Zekang liu +1
Hand and face play an important role in expressing sign language. Their features are usually especially leveraged to improve system performance. However, to effectively extract vis…
Spatial Temporal Graph Attention Network for Skeleton-Based Action Recognition
Lianyu Hu, Shenglan Liu, Wei Feng
It's common for current methods in skeleton-based action recognition to mainly consider capturing long-term temporal dependencies as skeleton sequences are typically long (>128 fra…
Temporal Lift Pooling for Continuous Sign Language Recognition
Lianyu Hu, Liqing Gao, Zekang Liu +1
Pooling methods are necessities for modern neural networks for increasing receptive fields and lowering down computational costs. However, commonly used hand-crafted pooling approa…
FSD-10: A Dataset for Competitive Sports Content Analysis
Shenlan Liu, Xiang Liu, Gao Huang +6
Action recognition is an important and challenging problem in video analysis. Although the past decade has witnessed progress in action recognition with the development of deep lea…