Publications (108)
Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization
Xiachong Feng, Xiaocheng Feng, Libo Qin +2
Current dialogue summarization systems usually encode the text with a number of general semantic features (e.g., keywords and topics) to gain more powerful dialogue modeling capabi…
X-WebAgentBench: A Multilingual Interactive Web Benchmark for Evaluating Global Agentic System
Peng Wang, Ruihan Tao, Qiguang Chen +2
Recently, large language model (LLM)-based agents have achieved significant success in interactive environments, attracting significant academic and industrial attention. Despite t…
Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Models
Qiguang Chen, Hanjing Li, Libo Qin +7
Recently, Diffusion Large Language Models (DLLMs) have offered high throughput and effective sequential reasoning, making them a competitive alternative to autoregressive LLMs (ALL…
Temporal Self-Rewarding Language Models: Decoupling Chosen-Rejected via Past-Future
Yidong Wang, Xin Wang, Cunxiang Wang +9
Self-Rewarding Language Models propose an architecture in which the Large Language Models(LLMs) both generates responses and evaluates its own outputs via LLM-as-a-Judge prompting,…
CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning
Yangfan Ye, Xiaocheng Feng, Zekun Yuan +11
Current large language models (LLMs) often exhibit imbalanced multilingual capabilities due to their English-centric training corpora. To address this, existing fine-tuning approac…
Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts
Xianzhen Luo, Qingfu Zhu, Zhiming Zhang +5
Program of Thoughts (PoT) is an approach characterized by its executable intermediate steps, which ensure the accuracy of the logical calculations in the reasoning process. Current…
Synergistic Dual Spatial-aware Generation of Image-to-Text and Text-to-Image
Yu Zhao, Hao Fei, Xiangtai Li +6
In the visual spatial understanding (VSU) area, spatial image-to-text (SI2T) and spatial text-to-image (ST2I) are two fundamental tasks that appear in dual form. Existing methods f…
FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark
Liang Xu, Xiaojing Lu, Chenyang Yuan +8
Pretrained Language Models (PLMs) have achieved tremendous success in natural language understanding tasks. While different learning schemes -- fine-tuning, zero-shot, and few-shot…
A Survey on Spoken Language Understanding: Recent Advances and New Frontiers
Libo Qin, Tianbao Xie, Wanxiang Che +1
Spoken Language Understanding (SLU) aims to extract the semantics frame of user queries, which is a core component in a task-oriented dialog system. With the burst of deep neural n…
OMIBench: Benchmarking Olympiad-Level Multi-Image Reasoning in Large Vision-Language Model
Qiguang Chen, Chengyu Luan, Jiajun Wu +7
Large vision-language models (LVLMs) have made substantial advances in reasoning tasks at the Olympiad level. Nevertheless, current Olympiad-level multimodal reasoning benchmarks f…
Beyond Correctness: Evaluating Subjective Writing Preferences Across Cultures
Shuangshuang Ying, Yunwen Li, Xingwei Qu +21
Current preference learning methods achieve high accuracy on standard benchmarks but exhibit significant performance degradation when objective quality signals are removed. We intr…
Can Large Language Models Understand You Better? An MBTI Personality Detection Dataset Aligned with Population Traits
Bohan Li, Jiannan Guan, Longxu Dou +12
The Myers-Briggs Type Indicator (MBTI) is one of the most influential personality theories reflecting individual differences in thinking, feeling, and behaving. MBTI personality de…
AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling
Libo Qin, Xiao Xu, Wanxiang Che +1
In real-world scenarios, users usually have multiple intents in the same utterance. Unfortunately, most spoken language understanding (SLU) models either mainly focused on the sing…
Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State Representation
Haoyang Wen, Yijia Liu, Wanxiang Che +2
Classic pipeline models for task-oriented dialogue system require explicit modeling the dialogue states and hand-crafted action spaces to query a domain-specific knowledge base. Co…
Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retriever
Libo Qin, Yijia Liu, Wanxiang Che +3
Querying the knowledge base (KB) has long been a challenge in the end-to-end task-oriented dialogue system. Previous sequence-to-sequence (Seq2Seq) dialogue generation work treats…
Task-Specific Data Selection for Instruction Tuning via Monosemantic Neuronal Activations
Da Ma, Gonghu Shang, Zhi Chen +6
Instruction tuning improves the ability of large language models (LLMs) to follow diverse human instructions, but achieving strong performance on specific target tasks remains chal…
Improving Language Model Reasoning with Self-motivated Learning
Yunlong Feng, Yang Xu, Libo Qin +2
Large-scale high-quality training data is important for improving the performance of models. After trained with data that has rationales (reasoning steps), models gain reasoning ca…
Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers
Libo Qin, Qiguang Chen, Yuhang Zhou +6
Multilingual Large Language Models are capable of using powerful Large Language Models to handle and respond to queries in multiple languages, which achieves remarkable success in…
OpenSLU: A Unified, Modularized, and Extensible Toolkit for Spoken Language Understanding
Libo Qin, Qiguang Chen, Xiao Xu +2
Spoken Language Understanding (SLU) is one of the core components of a task-oriented dialogue system, which aims to extract the semantic meaning of user queries (e.g., intents and…
ESAinsTOD: A Unified End-to-End Schema-Aware Instruction-Tuning Framework for Task-Oriented Dialog Modeling
Dechuan Teng, Chunlin Lu, Libo Qin +1
Existing end-to-end modeling methods for modular task-oriented dialog systems are typically tailored to specific datasets, making it challenging to adapt to new dialog scenarios. I…
GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling
Libo Qin, Fuxuan Wei, Tianbao Xie +3
Multi-intent SLU can handle multiple intents in an utterance, which has attracted increasing attention. However, the state-of-the-art joint models heavily rely on autoregressive ap…
Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought
Qiguang Chen, Libo Qin, Jiaqi Wang +2
Chain-of-Thought (CoT) reasoning has emerged as a promising approach for enhancing the performance of large language models (LLMs) on complex reasoning tasks. Recently, a series of…
Pro-HAN: A Heterogeneous Graph Attention Network for Profile-Based Spoken Language Understanding
Dechuan Teng, Chunlin Lu, Xiao Xu +2
Recently, Profile-based Spoken Language Understanding (SLU) has gained increasing attention, which aims to incorporate various types of supplementary profile information (i.e., Kno…
MMLongCite: A Benchmark for Evaluating Fidelity of Long-Context Vision-Language Models
Keyan Zhou, Zecheng Tang, Lingfeng Ming +8
The rapid advancement of large vision language models (LVLMs) has led to a significant expansion of their context windows. However, an extended context window does not guarantee th…
A Two-Stage Framework with Self-Supervised Distillation For Cross-Domain Text Classification
Yunlong Feng, Bohan Li, Libo Qin +2
Cross-domain text classification aims to adapt models to a target domain that lacks labeled data. It leverages or reuses rich labeled data from the different but related source dom…
UniDU: Towards A Unified Generative Dialogue Understanding Framework
Zhi Chen, Lu Chen, Bei Chen +5
With the development of pre-trained language models, remarkable success has been witnessed in dialogue understanding (DU). However, current DU approaches usually employ independent…
Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information
Yongheng Zhang, Qiguang Chen, Jingxuan Zhou +5
Chain-of-Thought (CoT) has become a vital technique for enhancing the performance of Large Language Models (LLMs), attracting increasing attention from researchers. One stream of a…
CroPrompt: Cross-task Interactive Prompting for Zero-shot Spoken Language Understanding
Libo Qin, Fuxuan Wei, Qiguang Chen +5
Slot filling and intent detection are two highly correlated tasks in spoken language understanding (SLU). Recent SLU research attempts to explore zero-shot prompting techniques in…
AI4Research: A Survey of Artificial Intelligence for Scientific Research
Qiguang Chen, Mingda Yang, Libo Qin +13
Recent advancements in artificial intelligence (AI), particularly in large language models (LLMs) such as OpenAI-o1 and DeepSeek-R1, have demonstrated remarkable capabilities in co…
Large Language Models Meet NLP: A Survey
Libo Qin, Qiguang Chen, Xiachong Feng +6
While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this…
ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Models
Yongheng Zhang, Xu Liu, Ruihan Tao +4
Video understanding plays a vital role in bridging low-level visual signals with high-level cognitive reasoning, and is fundamental to applications such as autonomous driving, embo…
Noise-Robustness Through Noise: A Framework combining Asymmetric LoRA with Poisoning MoE
Zhaokun Wang, Jinyu Guo, Jingwen Pu +5
Current parameter-efficient fine-tuning methods for adapting pre-trained language models to downstream tasks are susceptible to interference from noisy data. Conventional noise-han…
What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration
Libo Qin, Qiguang Chen, Hao Fei +3
Recently, rapid advancements in Multi-Modal In-Context Learning (MM-ICL) have achieved notable success, which is capable of achieving superior performance across various tasks with…
Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages
Libo Qin, Qiguang Chen, Fuxuan Wei +2
Chain-of-thought (CoT) is capable of eliciting models to explicitly generate reasoning paths, thus promoting reasoning accuracy and attracting increasing attention. Specifically, z…
The Universal Landscape of Human Reasoning
Qiguang Chen, Jinhao Liu, Libo Qin +14
Understanding how information is dynamically accumulated and transformed in human reasoning has long challenged cognitive psychology, philosophy, and artificial intelligence. Exist…
MCoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought
Qiguang Chen, Libo Qin, Jin Zhang +3
Multi-modal Chain-of-Thought (MCoT) requires models to leverage knowledge from both textual and visual modalities for step-by-step reasoning, which gains increasing attention. Neve…
A Co-Interactive Transformer for Joint Slot Filling and Intent Detection
Libo Qin, Tailu Liu, Wanxiang Che +3
Intent detection and slot filling are two main tasks for building a spoken language understanding (SLU) system. The two tasks are closely related and the information of one task ca…
Dialogue State Induction Using Neural Latent Variable Models
Qingkai Min, Libo Qin, Zhiyang Teng +2
Dialogue state modules are a useful component in a task-oriented dialogue system. Traditional methods find dialogue states by manually labeling training corpora, upon which neural…
MMSD2.0: Towards a Reliable Multi-modal Sarcasm Detection System
Libo Qin, Shijue Huang, Qiguang Chen +5
Multi-modal sarcasm detection has attracted much recent attention. Nevertheless, the existing benchmark (MMSD) has some shortcomings that hinder the development of reliable multi-m…
A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding
Wenbo Pan, Qiguang Chen, Xiao Xu +2
Zero-shot dialogue understanding aims to enable dialogue to track the user's needs without any training data, which has gained increasing attention. In this work, we investigate th…
RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoning
Qiguang Chen, Libo Qin, Jinhao Liu +4
Chain-of-Thought (CoT) reasoning has proven effective in enhancing large language models (LLMs) on complex tasks, spurring research into its underlying mechanisms. However, two pri…
DTIAM: A unified framework for predicting drug-target interactions, binding affinities and activation/inhibition mechanisms
Zhangli Lu, Chuqi Lei, Kaili Wang +3
Accurate and robust prediction of drug-target interactions (DTIs) plays a vital role in drug discovery. Despite extensive efforts have been invested in predicting novel DTIs, exist…
CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP
Libo Qin, Minheng Ni, Yue Zhang +1
Multi-lingual contextualized embeddings, such as multilingual-BERT (mBERT), have shown success in a variety of zero-shot cross-lingual tasks. However, these models are limited by h…
Thinking in Video: Can Video Generators Really Reason About the Real World?
Yongheng Zhang, Guang Yang, Ruihan Hou +12
Recent advances in world models and video generation have given rise to an emerging reasoning paradigm that leverages video generative models to simulate, predict, and reason about…
HIT-SCIR at MMNLU-22: Consistency Regularization for Multilingual Spoken Language Understanding
Bo Zheng, Zhouyang Li, Fuxuan Wei +3
Multilingual spoken language understanding (SLU) consists of two sub-tasks, namely intent detection and slot filling. To improve the performance of these two sub-tasks, we propose…
AutoPR: Let's Automate Your Academic Promotion!
Qiguang Chen, Zheng Yan, Mingda Yang +10
As the volume of peer-reviewed research surges, scholars increasingly rely on social platforms for discovery, while authors invest considerable effort in promoting their work to en…
Aware First, Think Less: Dynamic Boundary Self-Awareness Drives Extreme Reasoning Efficiency in Large Language Models
Qiguang Chen, Dengyun Peng, Jinhao Liu +4
Recent advancements in large language models (LLMs) have greatly improved their capabilities on complex reasoning tasks through Long Chain-of-Thought (CoT). However, this approach…
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective
Linyi Yang, Shuibai Zhang, Libo Qin +6
Pre-trained language models (PLMs) are known to improve the generalization performance of natural language understanding models by leveraging large amounts of data during the pre-t…
DCR-Net: A Deep Co-Interactive Relation Network for Joint Dialog Act Recognition and Sentiment Classification
Libo Qin, Wanxiang Che, Yangming Li +2
In dialog system, dialog act recognition and sentiment classification are two correlative tasks to capture speakers intentions, where dialog act and sentiment can indicate the expl…
MPCC: A Novel Benchmark for Multimodal Planning with Complex Constraints in Multimodal Large Language Models
Yiyan Ji, Haoran Chen, Qiguang Chen +3
Multimodal planning capabilities refer to the ability to predict, reason, and design steps for task execution with multimodal context, which is essential for complex reasoning and…
Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification
Libo Qin, Zhouyang Li, Wanxiang Che +2
In a dialog system, dialog act recognition and sentiment classification are two correlative tasks to capture speakers intentions, where dialog act and sentiment can indicate the ex…
Self-Constructed Context Decompilation with Fined-grained Alignment Enhancement
Yunlong Feng, Dechuan Teng, Yang Xu +5
Decompilation transforms compiled code back into a high-level programming language for analysis when source code is unavailable. Previous work has primarily focused on enhancing de…
Less Languages, Less Tokens: An Efficient Unified Logic Cross-lingual Chain-of-Thought Reasoning Framework
Chenyuan Zhang, Qiguang Chen, Xie Chen +6
Cross-lingual chain-of-thought (XCoT) with self-consistency markedly enhances multilingual reasoning, yet existing methods remain costly due to extensive sampling of full trajector…
DyCon: Dynamic Reasoning Control via Evolving Difficulty Modeling
Tengyao Tu, Yulin Li, Hui-Ling Zhen +6
Recent advances in Large Reasoning Models (LRMs) demonstrate remarkable performance improvements by iteratively reflecting, exploring, and executing complex tasks, yet suffer from…
Chain of Strategy Optimization Makes Large Language Models Better Emotional Supporter
Weixiang Zhao, Xingyu Sui, Xinyang Han +9
The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they fac…
Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring
Honglin Mu, Han He, Yuxin Zhou +9
Large language model (LLM) safety is a critical issue, with numerous studies employing red team testing to enhance model security. Among these, jailbreak methods explore potential…
Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction
Yinghui Li, Shang Qin, Haojing Huang +6
Recently, Large Language Models (LLMs) have been widely studied by researchers for their roles in various downstream NLP tasks. As a fundamental task in the NLP field, Chinese Gram…
Can Large Language Models Resolve Semantic Discrepancy in Self-Destructive Subcultures? Evidence from Jirai Kei
Peng Wang, Xilin Tao, Siyi Yao +5
Self-destructive behaviors are linked to complex psychological states and can be challenging to diagnose. These behaviors may be even harder to identify within subcultural groups d…
HASH-RAG: Bridging Deep Hashing with Retriever for Efficient, Fine Retrieval and Augmented Generation
Jinyu Guo, Xunlei Chen, Qiyang Xia +5
Retrieval-Augmented Generation (RAG) encounters efficiency challenges when scaling to massive knowledge bases while preserving contextual relevance. We propose Hash-RAG, a framewor…
Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale
Wenzhen Zheng, Wenbo Pan, Xu Xu +3
In recent years, Large Language Models (LLMs) have made significant strides towards Artificial General Intelligence. However, training these models from scratch requires substantia…
The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoning
Qiguang Chen, Yantao Du, Ziniu Li +10
Large language models (LLMs) often fail to learn effective long chain-of-thought (Long CoT) reasoning from human or non-Long-CoT LLMs imitation. To understand this, we propose that…
Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System
Libo Qin, Tianbao Xie, Shijue Huang +3
Consistency Identification has obtained remarkable success on open-domain dialogue, which can be used for preventing inconsistent response generation. However, in contrast to the r…
Accelerating Adaptive Retrieval Augmented Generation via Instruction-Driven Representation Reduction of Retrieval Overlaps
Jie Ou, Jinyu Guo, Shuaihong Jiang +4
Retrieval-augmented generation (RAG) has emerged as a pivotal method for expanding the knowledge of large language models. To handle complex queries more effectively, researchers d…
Retrieval-Infused Reasoning Sandbox: A Benchmark for Decoupling Retrieval and Reasoning Capabilities
Shuangshuang Ying, Zheyu Wang, Yunjian Peng +16
Despite strong performance on existing benchmarks, it remains unclear whether large language models can reason over genuinely novel scientific information. Most evaluations score e…
AutoCAP: Towards Automatic Cross-lingual Alignment Planning for Zero-shot Chain-of-Thought
Yongheng Zhang, Qiguang Chen, Min Li +2
Cross-lingual chain-of-thought can effectively complete reasoning tasks across languages, which gains increasing attention. Recently, dominant approaches in the literature improve…
CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention
Zekai Ye, Qiming Li, Xiaocheng Feng +10
Large Vision-Language Models (LVLMs) have demonstrated impressive multimodal abilities but remain prone to multilingual object hallucination, with a higher likelihood of generating…
Multi-Domain Spoken Language Understanding Using Domain- and Task-Aware Parameterization
Libo Qin, Minheng Ni, Yue Zhang +3
Spoken language understanding has been addressed as a supervised learning problem, where a set of training data is available for each domain. However, annotating data for each doma…
Exploring Cross-lingual Latent Transplantation: Mutual Opportunities and Open Challenges
Yangfan Ye, Xiaocheng Feng, Xiachong Feng +11
Current large language models (LLMs) often exhibit imbalances in multilingual capabilities and cultural adaptability, largely attributed to their English-centric pre-training data.…
Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks
Jiannan Guan, Qiguang Chen, Libo Qin +5
Large Language Models (LLMs) excel in reasoning tasks requiring a single correct answer, but they perform poorly in multi-solution tasks that require generating comprehensive and d…
Text is no more Enough! A Benchmark for Profile-based Spoken Language Understanding
Xiao Xu, Libo Qin, Kaiji Chen +3
Current researches on spoken language understanding (SLU) heavily are limited to a simple setting: the plain text-based SLU that takes the user utterance as input and generates its…
SAVOIR: Learning Social Savoir-Faire via Shapley-based Reward Attribution
Xiachong Feng, Yi Jiang, Xiaocheng Feng +9
Social intelligence, the ability to navigate complex interpersonal interactions, presents a fundamental challenge for language agents. Training such agents via reinforcement learni…
CAST: Mitigating Object Hallucination in Large Vision-Language Models via Caption-Guided Visual Attention Steering
Qiming Li, Zekai Ye, Xiaocheng Feng +9
Although Large Vision-Language Models (LVLMs) have demonstrated remarkable performance on downstream tasks, they frequently produce contents that deviate from visual information, l…
An Efficient Task-Oriented Dialogue Policy: Evolutionary Reinforcement Learning Injected by Elite Individuals
Yangyang Zhao, Ben Niu, Libo Qin +1
Deep Reinforcement Learning (DRL) is widely used in task-oriented dialogue systems to optimize dialogue policy, but it struggles to balance exploration and exploitation due to the…
MLDebugging: Towards Benchmarking Code Debugging Across Multi-Library Scenarios
Jinyang Huang, Xiachong Feng, Qiguang Chen +6
Code debugging is a crucial task in software engineering, which attracts increasing attention. While remarkable success has been made in the era of large language models (LLMs), cu…
Injecting Word Information with Multi-Level Word Adapter for Chinese Spoken Language Understanding
Dechuan Teng, Libo Qin, Wanxiang Che +2
In this paper, we improve Chinese spoken language understanding (SLU) by injecting word information. Previous studies on Chinese SLU do not consider the word information, failing t…
TextAtlas5M: A Large-scale Dataset for Dense Text Image Generation
Alex Jinpeng Wang, Dongxing Mao, Jiawei Zhang +9
Text-conditioned image generation has gained significant attention in recent years and are processing increasingly longer and comprehensive text prompt. In everyday life, dense and…
Learning the Boundary of Solvability: Aligning LLMs to Detect Unsolvable Problems
Dengyun Peng, Qiguang Chen, Bofei Liu +6
Ensuring large language model (LLM) reliability requires distinguishing objective unsolvability (inherent contradictions) from subjective capability limitations (tasks exceeding mo…
End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions
Libo Qin, Wenbo Pan, Qiguang Chen +5
End-to-end task-oriented dialogue (EToD) can directly generate responses in an end-to-end fashion without modular training, which attracts escalating popularity. The advancement of…
From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence
Jian Yang, Xianglong Liu, Weifeng Lv +68
Large language models (LLMs) have fundamentally transformed automated software development by enabling direct translation of natural language descriptions into functional code, dri…
Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play
Xiachong Feng, Deyi Yin, Xiaocheng Feng +9
Games offer a compelling paradigm for developing general reasoning capabilities in language models, as they naturally demand strategic planning, probabilistic inference, and adapti…
CCHall: A Novel Benchmark for Joint Cross-Lingual and Cross-Modal Hallucinations Detection in Large Language Models
Yongheng Zhang, Xu Liu, Ruoxi Zhou +4
Investigating hallucination issues in large language models (LLMs) within cross-lingual and cross-modal scenarios can greatly advance the large-scale deployment in real-world appli…
CAI: Caption-Sensitive Attention Intervention for Mitigating Object Hallucination in Large Vision-Language Models
Qiming Li, Zekai Ye, Xiaocheng Feng +8
Although Large Vision-Language Models (LVLMs) have demonstrated powerful capabilities in interpreting visual information, they frequently produce content that deviates from visual…
Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Tasks
Wenbo Pan, Jie Xu, Qiguang Chen +5
Large Language Models (LLMs) should refuse to answer questions beyond their knowledge. This capability, which we term knowledge-aware refusal, is crucial for factual reliability, w…
GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization
Yangfan Ye, Xiachong Feng, Xiaocheng Feng +6
News summarization in today's global scene can be daunting with its flood of multilingual content and varied viewpoints from different sources. However, current studies often negle…
SDIF-DA: A Shallow-to-Deep Interaction Framework with Data Augmentation for Multi-modal Intent Detection
Shijue Huang, Libo Qin, Bingbing Wang +2
Multi-modal intent detection aims to utilize various modalities to understand the user's intentions, which is essential for the deployment of dialogue systems in real-world scenari…
LESA: Learnable LLM Layer Scaling-Up
Yifei Yang, Zouying Cao, Xinbei Ma +4
Training Large Language Models (LLMs) from scratch requires immense computational resources, making it prohibitively expensive. Model scaling-up offers a promising solution by leve…
A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding
Libo Qin, Wanxiang Che, Yangming Li +2
Intent detection and slot filling are two main tasks for building a spoken language understanding (SLU) system. The two tasks are closely tied and the slots often highly depend on…
Let's Think with Images Efficiently! An Interleaved-Modal Chain-of-Thought Reasoning Framework with Dynamic and Precise Visual Thoughts
Xu Liu, Yongheng Zhang, Qiguang Chen +3
Recently, Interleaved-modal Chain-of-Thought (ICoT) reasoning has achieved remarkable success by leveraging both multimodal inputs and outputs, attracting increasing attention. Whi…
What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practices
Zhi Chen, Qiguang Chen, Libo Qin +7
Recent advancements in large language models (LLMs) with extended context windows have significantly improved tasks such as information extraction, question answering, and complex…
GL-CLeF: A Global-Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding
Libo Qin, Qiguang Chen, Tianbao Xie +4
Due to high data demands of current methods, attention to zero-shot cross-lingual spoken language understanding (SLU) has grown, as such approaches greatly reduce human annotation…
CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models
Zihui Cheng, Qiguang Chen, Jin Zhang +5
Large Vision-Language Models (LVLMs) have recently demonstrated amazing success in multi-modal tasks, including advancements in Multi-modal Chain-of-Thought (MCoT) reasoning. Despi…
Exploring Multi-Grained Concept Annotations for Multimodal Large Language Models
Xiao Xu, Tianhao Niu, Yuxi Xie +3
Multimodal Large Language Models (MLLMs) excel in vision--language tasks by pre-training solely on coarse-grained concept annotations (e.g., image captions). We hypothesize that in…
DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding
Yanhua Jiao, Tianyi Wu, Xiaoxi Sun +6
While parallel decoding is central to the efficiency of Diffusion Large Language Models (dLLMs), current strategies are often hindered by overly conservative confidence thresholds.…
Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
Qiguang Chen, Libo Qin, Jinhao Liu +7
Recent advancements in reasoning with large language models (RLLMs), such as OpenAI-O1 and DeepSeek-R1, have demonstrated their impressive capabilities in complex domains like math…
MAC-SLU: Multi-Intent Automotive Cabin Spoken Language Understanding Benchmark
Yuezhang Peng, Chonghao Cai, Ziang Liu +10
Spoken Language Understanding (SLU), which aims to extract user semantics to execute downstream tasks, is a crucial component of task-oriented dialog systems. Existing SLU datasets…
Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought
Zihui Cheng, Qiguang Chen, Xiao Xu +8
Large Vision-Language Models (LVLMs) have achieved significant success in multimodal tasks, with multimodal chain-of-thought (MCoT) further enhancing performance and interpretabili…
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann +122
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained…
Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog
Libo Qin, Xiao Xu, Wanxiang Che +2
Recent studies have shown remarkable success in end-to-end task-oriented dialog system. However, most neural models rely on large training data, which are only available for a cert…
Manager: Aggregating Insights from Unimodal Experts in Two-Tower VLMs and MLLMs
Xiao Xu, Libo Qin, Wanxiang Che +1
Two-Tower Vision--Language Models (VLMs) have demonstrated strong performance across various downstream VL tasks. While BridgeTower further enhances performance by building bridges…
KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing
Yifei Yang, Zouying Cao, Qiguang Chen +4
The development of large language models (LLMs) has significantly expanded model sizes, resulting in substantial GPU memory requirements during inference. The key and value storage…