148 citations · 708 across the 141 of their papers we have counts for
32 papers · 1 filter
Self-Evolution Knowledge Distillation for LLM-based Machine Translation
Yuncheng Song, Liang Ding, Changtong Zan +1
Knowledge distillation (KD) has shown great promise in transferring knowledge from larger teacher models to smaller student models. However, existing KD strategies for large langua…
DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs
Xiabin Zhou, Wenbin Wang, Minyan Zeng +5
Efficient KV cache management in LLMs is crucial for long-context tasks like RAG and summarization. Existing KV cache compression methods enforce a fixed pattern, neglecting task-s…
Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL
Qihuang Zhong, Kunfeng Chen, Liang Ding +3
Large Language Models (LLMs) have shown promising performance in text-to-SQL, which involves translating natural language questions into SQL queries. However, current text-to-SQL L…
Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
Fei Wang, Li Shen, Liang Ding +3
Fine-tuning is powerful for adapting large language models to downstream tasks, but it often results in huge memory usages. A promising approach to mitigate this is using Zeroth-Or…
Self-Powered LLM Modality Expansion for Large Speech-Text Models
Tengfei Yu, Xuebo Liu, Zhiyi Hou +3
Large language models (LLMs) exhibit remarkable performance across diverse tasks, indicating their potential for expansion into large speech-text models (LSMs) by integrating speec…
CogSteer: Cognition-Inspired Selective Layer Intervention for Efficiently Steering Large Language Models
Xintong Wang, Jingheng Pan, Liang Ding +4
Large Language Models (LLMs) achieve remarkable performance through pretraining on extensive data. This enables efficient adaptation to diverse downstream tasks. However, the lack…
End-to-End Graph Flattening Method for Large Language Models
Bin Hong, Jinze Wu, Jiayu Liu +5
In recent years, the breakthrough of Large Language Models (LLMs) offers new ideas for achieving universal methods on graph data. The common practice of converting graphs into natu…
Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models
Jun Rao, Xuebo Liu, Zepeng Lin +4
Knowledge distillation (KD) is a technique that compresses large teacher models by training smaller student models to mimic them. The success of KD in auto-regressive language mode…
: Improving Code Generation of LLMs by Uncertainty-Aware Selective Contrastive Decoding
Shuai Wang, Liang Ding, Li Shen +4
Large language models (LLMs) have shown remarkable capabilities in code generation. However, the effects of hallucinations (e.g., output noise) make it particularly challenging for…
MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluators
Qingyu Lu, Liang Ding, Kanjian Zhang +2
Large Language Models (LLMs) have shown significant potential as judges for Machine Translation (MT) quality assessment, providing both scores and fine-grained feedback. Although a…
Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
Wenbin Wang, Liang Ding, Minyan Zeng +4
Multimodal large language models (MLLMs) have experienced significant advancements recently, but still struggle to recognize and interpret intricate details in high-resolution (HR)…
Aligning Large Language Models from Self-Reference AI Feedback with one General Principle
Rong Bao, Rui Zheng, Shihan Dou +6
In aligning large language models (LLMs), utilizing feedback from existing advanced AI rather than humans is an important method to scale supervisory signals. However, it is highly…
Uncertainty Aware Learning for Language Model Alignment
Yikun Wang, Rui Zheng, Liang Ding +3
As instruction-tuned large language models (LLMs) evolve, aligning pretrained foundation models presents increasing challenges. Existing alignment strategies, which typically lever…
Revisiting Catastrophic Forgetting in Large Language Model Tuning
Hongyu Li, Liang Ding, Meng Fang +1
Catastrophic Forgetting (CF) means models forgetting previously acquired knowledge when learning new data. It compromises the effectiveness of large language models (LLMs) during f…
Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning
Tianle Xia, Liang Ding, Guojia Wan +3
Answering complex queries over incomplete knowledge graphs (KGs) is a challenging job. Most previous works have focused on learning entity/relation embeddings and simulating first-…
3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset
Xinyu Ma, Xuebo Liu, Derek F. Wong +6
Multimodal machine translation (MMT) is a challenging task that seeks to improve translation quality by incorporating visual information. However, recent studies have indicated tha…
Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems
Qihuang Zhong, Kang Wang, Ziyang Xu +3
Chain-of-Thought (CoT) prompting has enhanced the performance of Large Language Models (LLMs) across various reasoning tasks. However, CoT still falls short in dealing with complex…
Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction
Ziyang Xu, Keqin Peng, Liang Ding +2
Recent research shows that pre-trained language models (PLMs) suffer from "prompt bias" in factual knowledge extraction, i.e., prompts tend to introduce biases toward specific labe…
Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning
Changtong Zan, Liang Ding, Li Shen +3
Translation-tailored Large language models (LLMs) exhibit remarkable translation capabilities, even competing with supervised-trained commercial translation systems. However, off-t…
Towards Training A Chinese Large Language Model for Anesthesiology
Zhonghai Wang, Jie Jiang, Yibing Zhan +8
Medical large language models (LLMs) have gained popularity recently due to their significant practical utility. However, most existing research focuses on general medicine, and th…
Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding
Xintong Wang, Jingheng Pan, Liang Ding +1
Large Vision-Language Models (LVLMs) are increasingly adept at generating contextually detailed and coherent responses from visual inputs. However, their application in multimodal…
OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models
Shuai Wang, Liang Ding, Li Shen +3
Advancing automated programming necessitates robust and comprehensive code generation benchmarks, yet current evaluation frameworks largely neglect object-oriented programming (OOP…
Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation
Zhiyao Ren, Yibing Zhan, Baosheng Yu +2
The copilot framework, which aims to enhance and tailor large language models (LLMs) for specific complex tasks without requiring fine-tuning, is gaining increasing attention from…
WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge
Wenbin Wang, Liang Ding, Li Shen +3
Sentiment analysis is rapidly advancing by utilizing various data modalities (e.g., text, image). However, most previous works relied on superficial information, neglecting the inc…