4 citations · 9 across the 14 of their papers we have counts for
14 papers
Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflicts
Zhihao Zhu, Jiafeng Liang, Shixin Jiang +5
Large Multimodal Models (LMMs) have demonstrated impressive capabilities in video reasoning via Chain-of-Thought (CoT). However, the robustness of their reasoning chains remains qu…
HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understanding
Haowei Zhang, Shudong Yang, Jinlan Fu +2
Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated significant improvement in offline video understanding. However, extending these capabilities to s…
FutureOmni: Evaluating Future Forecasting from Omni-Modal Context for Multimodal LLMs
Qian Chen, Jinlan Fu, Changsong Li +3
Although Multimodal Large Language Models (MLLMs) demonstrate strong omni-modal perception, their ability to forecast future events from audio-visual cues remains largely unexplore…
AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agents
Jiafeng Liang, Hao Li, Chang Li +12
Memory serves as the pivotal nexus bridging past and future, providing both humans and AI systems with invaluable concepts and experience to navigate complex tasks. Recent research…
MCM-DPO: Multifaceted Cross-Modal Direct Preference Optimization for Alt-text Generation
Jinlan Fu, Shenzhen Huangfu, Hao Fei +4
The alt-text generation task produces concise, context-relevant descriptions of images, enabling blind and low-vision users to access online images. Despite the capabilities of lar…
Investigating and Enhancing the Robustness of Large Multimodal Models Against Temporal Inconsistency
Jiafeng Liang, Shixin Jiang, Xuan Dong +7
Large Multimodal Models (LMMs) have recently demonstrated impressive performance on general video comprehension benchmarks. Nevertheless, for broader applications, the robustness o…
CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs
Jinlan Fu, Shenzhen Huangfu, Hao Fei +4
Multimodal Large Language Models (MLLMs) still struggle with hallucinations despite their impressive capabilities. Recent studies have attempted to mitigate this by applying Direct…
From 2D to 3D: AISG-SLA Visual Localization Challenge
Jialin Gao, Bill Ong, Darld Lwi +13
Research in 3D mapping is crucial for smart city applications, yet the cost of acquiring 3D data often hinders progress. Visual localization, particularly monocular camera position…
PetalView: Fine-grained Location and Orientation Extraction of Street-view Images via Cross-view Local Search with Supplementary Materials
Wenmiao Hu, Yichen Zhang, Yuxuan Liang +5
Satellite-based street-view information extraction by cross-view matching refers to a task that extracts the location and orientation information of a given street-view image query…
Chain of Thought Explanation for Dialogue State Tracking
Lin Xu, Ningxin Peng, Daquan Zhou +2
Dialogue state tracking (DST) aims to record user queries and goals during a conversational interaction achieved by maintaining a predefined set of slots and their corresponding va…
CET2: Modelling Topic Transitions for Coherent and Engaging Knowledge-Grounded Conversations
Lin Xu, Qixian Zhou, Jinlan Fu +1
Knowledge-grounded dialogue systems aim to generate coherent and engaging responses based on the dialogue contexts and selected external knowledge. Previous knowledge selection met…
Hint-before-Solving Prompting: Guiding LLMs to Effectively Utilize Encoded Knowledge
Jinlan Fu, Shenzhen Huangfu, Hang Yan +2
Large Language Models (LLMs) have recently showcased remarkable generalizability in various domains. Despite their extensive knowledge, LLMs still face challenges in efficiently ut…
OmniDialog: An Omnipotent Pre-training Model for Task-Oriented Dialogue System
Mingtao Yang, See-Kiong Ng, Jinlan Fu
Pre-trained conversation models (PCMs) have demonstrated remarkable results in task-oriented dialogue (TOD) systems. Many PCMs focus predominantly on dialogue management tasks like…
Extending JumpProcess.jl for fast point process simulation with time-varying intensities
Guilherme Augusto Zagatti, Samuel A. Isaacson, Christopher Rackauckas +3
Point processes model the occurrence of a countable number of random points over some support. They can model diverse phenomena, such as chemical reactions, stock market transactio…