5 citations · 14 across the 17 of their papers we have counts for
17 papers
ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning
Ruizhong Qiu, Hanqing Zeng, Yinglong Xia +15
Low-rank adapters (LoRAs) are a parameter-efficient finetuning technique that injects trainable low-rank matrices into pretrained models to adapt them to new tasks. Mixture-of-LoRA…
TCPO: Thought-Centric Preference Optimization for Effective Embodied Decision-making
Kechen Jiao, Zhirui Fang, Jiahao Liu +9
Using effective generalization capabilities of vision language models (VLMs) in context-specific dynamic tasks for embodied artificial intelligence remains a significant challenge.…
Dynamic Fisher-weighted Model Merging via Bayesian Optimization
Sanwoo Lee, Jiahao Liu, Qifan Wang +3
The fine-tuning of pre-trained language models has resulted in the widespread availability of task-specific models. Model merging offers an efficient way to create multi-task model…
FIRP: Faster LLM inference via future intermediate representation prediction
Pengfei Wu, Jiahao Liu, Zhuocheng Gong +5
Recent advancements in Large Language Models (LLMs) have shown remarkable performance across a wide range of tasks. Despite this, the auto-regressive nature of LLM decoding, which…
Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism
Jiahao Liu, Qifan Wang, Jingang Wang +1
The recent advancements in large language models (LLMs) have been extraordinary, yet the escalating inference costs associated with them present challenges in real-world applicatio…
Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
Pengfei Wu, Jiahao Liu, Zhuocheng Gong +5
Large language models (LLMs) have recently shown remarkable performance across a wide range of tasks. However, the substantial number of parameters in LLMs contributes to significa…
C-ICL: Contrastive In-context Learning for Information Extraction
Ying Mo, Jiahao Liu, Jian Yang +4
There has been increasing interest in exploring the capabilities of advanced large language models (LLMs) in the field of information extraction (IE), specifically focusing on task…
Improving Input-label Mapping with Demonstration Replay for In-context Learning
Zhuocheng Gong, Jiahao Liu, Qifan Wang +4
In-context learning (ICL) is an emerging capability of large autoregressive language models where a few input-label demonstrations are appended to the input to enhance the model's…
Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression
Jiduan Liu, Jiahao Liu, Qifan Wang +5
Large-scale pre-trained language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks. However, the massive size of these mode…
mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning
Ying Mo, Jian Yang, Jiahao Liu +4
Cross-lingual named entity recognition (CrossNER) faces challenges stemming from uneven performance due to the scarcity of multilingual corpora, especially for non-English data. Wh…
PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models
Zhuocheng Gong, Jiahao Liu, Qifan Wang +6
While transformer-based pre-trained language models (PLMs) have dominated a number of NLP applications, these models are heavy to deploy and expensive to use. Therefore, effectivel…
RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank
Jiduan Liu, Jiahao Liu, Qifan Wang +6
Unsupervised sentence representation learning is one of the fundamental problems in natural language processing with various downstream applications. Recently, contrastive learning…
Multi-task Transformer with Relation-attention and Type-attention for Named Entity Recognition
Ying Mo, Hongyin Tang, Jiahao Liu +5
Named entity recognition (NER) is an important research problem in natural language processing. There are three types of NER tasks, including flat, nested and discontinuous entity…
XPrompt: Exploring the Extreme of Prompt Tuning
Fang Ma, Chen Zhang, Lei Ren +5
Prompt tuning learns soft prompts to condition frozen Pre-trained Language Models (PLMs) for performing downstream tasks in a parameter-efficient manner. While prompt tuning has gr…
CLOWER: A Pre-trained Language Model with Contrastive Learning over Word and Character Representations
Borun Chen, Hongyin Tang, Jiahao Bu +6
Pre-trained Language Models (PLMs) have achieved remarkable performance gains across numerous downstream tasks in natural language understanding. Various Chinese PLMs have been suc…
MiniDisc: Minimal Distillation Schedule for Language Model Compression
Chen Zhang, Yang Yang, Qifan Wang +4
Recent studies have uncovered that language model distillation is less effective when facing a large capacity gap between the teacher and the student, and introduced teacher assist…
VIRT: Improving Representation-based Models for Text Matching through Virtual Interaction
Dan Li, Yang Yang, Hongyin Tang +4
With the booming of pre-trained transformers, representation-based models based on Siamese transformer encoders have become mainstream techniques for efficient text matching. Howev…