Publications (24)
TextShield-R1: Reinforced Reasoning for Tampered Text Detection
Chenfan Qu, Yiwu Zhong, Jian Liu +3
The growing prevalence of tampered images poses serious security threats, highlighting the urgent need for reliable detection methods. Multimodal large language models (MLLMs) demo…
Multimodal Segmentation for Vocal Tract Modeling
Rishi Jain, Bohan Yu, Peter Wu +2
Accurate modeling of the vocal tract is necessary to construct articulatory representations for interpretable speech processing and linguistics. However, vocal tract modeling is ch…
From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
Chenxi Zhou, Pengfei Cao, Jiang Li +4
Post-Training Quantization (PTQ) is critical for the efficient deployment of Large Language Models (LLMs). While 4-bit quantization is widely regarded as an optimal trade-off, redu…
Fast, High-Quality and Parameter-Efficient Articulatory Synthesis using Differentiable DSP
Yisi Liu, Bohan Yu, Drake Lin +3
Articulatory trajectories like electromagnetic articulography (EMA) provide a low-dimensional representation of the vocal tract filter and have been used as natural, grounded featu…
OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models
Keda Tao, Kele Shao, Bohan Yu +3
Omnimodal large language models (OmniLLMs) have attracted increasing research attention of late towards unified audio-video understanding. However, the high computational cost of p…
Towards Streaming Speech-to-Avatar Synthesis
Tejas S. Prabhune, Peter Wu, Bohan Yu +1
Streaming speech-to-avatar synthesis creates real-time animations for a virtual character from audio data. Accurate avatar representations of speech are important for the visualiza…
Doctor Imitator: Hand-Radiography-based Bone Age Assessment by Imitating Scoring Methods
Jintai Chen, Bohan Yu, Biwen Lei +3
Bone age assessment is challenging in clinical practice due to the complicated bone age assessment process. Current automatic bone age assessment methods were designed with rare co…
IESR:Efficient MCTS-Based Modular Reasoning for Text-to-SQL with Large Language Models
Tao Liu, Jiafan Lu, Bohan Yu +11
Text-to-SQL is a key natural language processing task that maps natural language questions to SQL queries, enabling intuitive interaction with web-based databases. Although current…
From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search
Junlin Liu, Jiangwang Chen, Zixin Song +7
Agentic search enables large language models to solve knowledge-intensive tasks by interleaving multi-step reasoning with retrieval, yet optimizing this with outcome-based reinforc…
EvolKV: Evolutionary KV Cache Compression for LLM Inference
Bohan Yu, Yekun Chai
Existing key-value (KV) cache compression methods typically rely on heuristics, such as uniform cache allocation across layers or static eviction policies, however, they ignore the…
E2VIDiff: Perceptual Events-to-Video Reconstruction using Diffusion Priors
Jinxiu Liang, Bohan Yu, Yixin Yang +2
Event cameras, mimicking the human retina, capture brightness changes with unparalleled temporal resolution and dynamic range. Integrating events into intensities poses a highly il…
NTIRE 2024 Quality Assessment of AI-Generated Content Challenge
Xiaohong Liu, Xiongkuo Min, Guangtao Zhai +111
This paper reports on the NTIRE 2024 Quality Assessment of AI-Generated Content Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancemen…
Active Perception Agent for Omnimodal Audio-Video Understanding
Keda Tao, Wenjie Du, Bohan Yu +3
Omnimodal large language models have made significant strides in unifying audio and visual modalities; however, they often face challenges in fine-grained cross-modal understanding…
Towards an Interpretable Representation of Speaker Identity via Perceptual Voice Qualities
Robin Netzorg, Bohan Yu, Andrea Guzman +3
Unlike other data modalities such as text and vision, speech does not lend itself to easy interpretation. While lay people can understand how to describe an image or sentence via p…
TableEval: A Real-World Benchmark for Complex, Multilingual, and Multi-Structured Table Question Answering
Junnan Zhu, Jingyi Wang, Bohan Yu +4
LLMs have shown impressive progress in natural language processing. However, they still face significant challenges in TableQA, where real-world complexities such as diverse table…
LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs
Keda Tao, Yuhua Zheng, Jia Xu +13
Recent advancements in omnimodal large language models (OmniLLMs) have significantly improved the comprehension of audio and video inputs. However, current evaluations primarily fo…
Towards EMG-to-Speech with a Necklace Form Factor
Peter Wu, Ryan Kaveh, Raghav Nautiyal +8
Electrodes for decoding speech from electromyography (EMG) are typically placed on the face, requiring adhesives that are inconvenient and skin-irritating if used regularly. We exp…
PReM: Learning What to Preserve and When to Refresh for Context Compression
Bohan Yu, Lei Shen, Chenxi Zhou +5
The paper proposes PReM, a framework that lets language models dynamically decide which parts of a long context to keep and when to refresh stored information, improving efficiency…
STAGE-Claw: Automated State-based Agent Benchmarking for Realistic Scenarios
Sirui Liang, Bohan Yu, Peiyu Wang +8
Large language models are increasingly used to power personal agents for everyday applications, but evaluating these agents remains a challenge. Existing benchmarks still rely on s…
Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models
Chenxi Zhou, Pengfei Cao, Jiang Li +4
Post-Training Quantization (PTQ) is a critical strategy for efficient Large Language Models (LLMs) deployment. However, existing scaling laws primarily focus on general performance…
Conformity Dynamics in LLM Multi-Agent Systems: The Roles of Topology and Self-Social Weighting
Chen Han, Jin Tan, Bohan Yu +2
Large Language Models (LLMs) are increasingly instantiated as interacting agents in multi-agent systems (MAS), where collective decisions emerge through social interaction rather t…
Deep Speech Synthesis from Multimodal Articulatory Representations
Peter Wu, Bohan Yu, Kevin Scheck +6
The amount of articulatory data available for training deep learning models is much less compared to acoustic speech data. In order to improve articulatory-to-acoustic synthesis pe…
StreamingAssistant: Efficient Visual Token Pruning for Accelerating Online Video Understanding
Xinqi Jin, Hanxun Yu, Bohan Yu +8
Online video understanding is essential for applications like public surveillance and AI glasses. However, applying Multimodal Large Language Models (MLLMs) to this domain is chall…
SR-KI: Scalable and Real-Time Knowledge Integration into LLMs via Supervised Attention
Bohan Yu, Wei Huang, Kang Liu
This paper proposes SR-KI, a novel approach for integrating real-time and large-scale structured knowledge bases (KBs) into large language models (LLMs). SR-KI begins by encoding K…