3 citations · 11 across the 17 of their papers we have counts for
19 papers
The Evolution of Attention in Large Language Models: Mechanisms, Trade-offs, and Emerging Trends
Zhentao Tan, Jingyi Shen, Yanbo Li +3
Self-attention gives LLMs fine-grained, query-dependent access to context, but dense token interactions incur quadratic prefill cost and a key--value cache growing with context len…
SPIDER: Multi-Layer Semantic Token Pruning and Adaptive Sub-Layer Skipping in Multimodal Large Language Models
Tianxiang Chen, Zhentao Tan, Zi Ye +9
Multimodal Large Language Models face significant efficiency challenges that stem from two distinct yet coupled sources: data redundancy and computational redundancy. While most me…
From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs
Zhentao Tan, Chang Liu, Yao Liu +2
As large language models (LLMs) scale rapidly, dense full-parameter adaptation becomes increasingly expensive, motivating sparse and modular architectures such as Mixture-of-Expert…
HydraHead: From Head-Level Functional Heterogeneity to Specialized Attention Hybridization
Zhentao Tan, Wei Chen, Jingyi Shen +4
The quadratic complexity of attention poses a critical bottleneck for long-context processing, spurring interest in hybrid attention designs. Most open-source hybrid models adopt a…
CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answering
Yuyang Hong, Jiaqi Gu, Yujin Lou +7
Knowledge-based visual question answering (KB-VQA) demonstrates significant potential for handling knowledge-intensive tasks. However, conflicts arise between static parametric kno…
Interact-RAG: Reason and Interact with the Corpus, Beyond Black-Box Retrieval
Yulong Hui, Chao Chen, Zhihang Fu +3
Retrieval-Augmented Generation (RAG) has significantly enhanced LLMs by incorporating external information. However, prevailing agentic RAG approaches are constrained by a critical…
Flora: Effortless Context Construction to Arbitrary Length and Scale
Tianxiang Chen, Zhentao Tan, Xiaofan Bo +5
Effectively handling long contexts is challenging for Large Language Models (LLMs) due to the rarity of long texts, high computational demands, and substantial forgetting of short-…
ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL
Yang Qin, Chao Chen, Zhihang Fu +4
Despite the significant advancements in Text-to-SQL (Text2SQL) facilitated by large language models (LLMs), the latest state-of-the-art techniques are still trapped in the in-conte…
Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection
Tianxiang Chen, Zhentao Tan, Tao Gong +5
As a manner to augment pre-trained large language models (LLM), knowledge injection is critical to develop vertical domain large models and has been widely studied. Although most c…
Category-Extensible Out-of-Distribution Detection via Hierarchical Context Descriptions
Kai Liu, Zhihang Fu, Chao Chen +5
The key to OOD detection has two aspects: generalized feature representation and precise category description. Recently, vision-language models such as CLIP provide significant adv…
Structure-aware Domain Knowledge Injection for Large Language Models
Kai Liu, Ze Chen, Zhihang Fu +6
This paper introduces a pioneering methodology, termed StructTuning, to efficiently transform foundation Large Language Models (LLMs) into domain specialists. It significantly redu…
Enhancing LLM's Cognition via Structurization
Kai Liu, Zhihang Fu, Chao Chen +6
When reading long-form text, human cognition is complex and structurized. While large language models (LLMs) process input contexts through a causal and sequential perspective, thi…
Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution
Kai Liu, Zhihang Fu, Sheng Jin +6
Detecting and rejecting unknown out-of-distribution (OOD) samples is critical for deployed neural networks to void unreliable predictions. In real-world scenarios, however, the eff…
ESOD: Efficient Small Object Detection on High-Resolution Images
Kai Liu, Zhihang Fu, Sheng Jin +5
Enlarging input images is a straightforward and effective approach to promote small object detection. However, simple image enlargement is significantly expensive on both computati…
ROPO: Robust Preference Optimization for Large Language Models
Xize Liang, Chao Chen, Shuang Qiu +6
Preference alignment is pivotal for empowering large language models (LLMs) to generate helpful and harmless responses. However, the performance of preference alignment is highly s…
MiM-ISTD: Mamba-in-Mamba for Efficient Infrared Small Target Detection
Tianxiang Chen, Zi Ye, Zhentao Tan +6
Recently, infrared small target detection (ISTD) has made significant progress, thanks to the development of basic models. Specifically, the models combining CNNs with transformers…
Bootstrapping Audio-Visual Segmentation by Strengthening Audio Cues
Tianxiang Chen, Zhentao Tan, Tao Gong +6
How to effectively interact audio with vision has garnered considerable interest within the multi-modality research field. Recently, a novel audio-visual segmentation (AVS) task ha…
Optimal Parameter and Neuron Pruning for Out-of-Distribution Detection
Chao Chen, Zhihang Fu, Kai Liu +3
For a machine learning model deployed in real world scenarios, the ability of detecting out-of-distribution (OOD) samples is indispensable and challenging. Most existing OOD detect…
INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection
Chao Chen, Kai Liu, Ze Chen +5
Knowledge hallucination have raised widespread concerns for the security and reliability of deployed LLMs. Previous efforts in detecting hallucinations have been employed at logit-…