2 citations · 9 across the 19 of their papers we have counts for
16 papers · 1 filter
Dynamic Fisher-weighted Model Merging via Bayesian Optimization
Sanwoo Lee, Jiahao Liu, Qifan Wang +3
The fine-tuning of pre-trained language models has resulted in the widespread availability of task-specific models. Model merging offers an efficient way to create multi-task model…
Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective
Ruichen Shao, Bei Li, Gangao Liu +5
Direct Preference Optimization (DPO) has gained attention as an efficient alternative to reinforcement learning from human feedback (RLHF) for aligning large language models (LLMs)…
Predictor-Corrector Enhanced Transformers with Exponential Moving Average Coefficient Learning
Bei Li, Tong Zheng, Rui Wang +8
Residual networks, as discrete approximations of Ordinary Differential Equations (ODEs), have inspired significant advancements in neural network design, including multistep method…
FIRP: Faster LLM inference via future intermediate representation prediction
Pengfei Wu, Jiahao Liu, Zhuocheng Gong +5
Recent advancements in Large Language Models (LLMs) have shown remarkable performance across a wide range of tasks. Despite this, the auto-regressive nature of LLM decoding, which…
Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-context Models
Xinyu Liu, Runsong Zhao, Pengcheng Huang +5
Numerous recent works target to extend effective context length for language models and various methods, tasks and benchmarks exist to measure model's effective memorization length…
ReMamba: Equip Mamba with Effective Long-Sequence Modeling
Danlong Yuan, Jiahao Liu, Bei Li +4
While the Mamba architecture demonstrates superior inference efficiency and competitive performance on short-context natural language processing (NLP) tasks, empirical evidence sug…
Graph-Structured Speculative Decoding
Zhuocheng Gong, Jiahao Liu, Ziyue Wang +5
Speculative decoding has emerged as a promising technique to accelerate the inference of Large Language Models (LLMs) by employing a small language model to draft a hypothesis sequ…
Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism
Jiahao Liu, Qifan Wang, Jingang Wang +1
The recent advancements in large language models (LLMs) have been extraordinary, yet the escalating inference costs associated with them present challenges in real-world applicatio…
Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
Pengfei Wu, Jiahao Liu, Zhuocheng Gong +5
Large language models (LLMs) have recently shown remarkable performance across a wide range of tasks. However, the substantial number of parameters in LLMs contributes to significa…
C-ICL: Contrastive In-context Learning for Information Extraction
Ying Mo, Jiahao Liu, Jian Yang +4
There has been increasing interest in exploring the capabilities of advanced large language models (LLMs) in the field of information extraction (IE), specifically focusing on task…
Improving Input-label Mapping with Demonstration Replay for In-context Learning
Zhuocheng Gong, Jiahao Liu, Qifan Wang +4
In-context learning (ICL) is an emerging capability of large autoregressive language models where a few input-label demonstrations are appended to the input to enhance the model's…
Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression
Jiduan Liu, Jiahao Liu, Qifan Wang +5
Large-scale pre-trained language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks. However, the massive size of these mode…
mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning
Ying Mo, Jian Yang, Jiahao Liu +4
Cross-lingual named entity recognition (CrossNER) faces challenges stemming from uneven performance due to the scarcity of multilingual corpora, especially for non-English data. Wh…
PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models
Zhuocheng Gong, Jiahao Liu, Qifan Wang +6
While transformer-based pre-trained language models (PLMs) have dominated a number of NLP applications, these models are heavy to deploy and expensive to use. Therefore, effectivel…
RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank
Jiduan Liu, Jiahao Liu, Qifan Wang +6
Unsupervised sentence representation learning is one of the fundamental problems in natural language processing with various downstream applications. Recently, contrastive learning…
Multi-task Transformer with Relation-attention and Type-attention for Named Entity Recognition
Ying Mo, Hongyin Tang, Jiahao Liu +5
Named entity recognition (NER) is an important research problem in natural language processing. There are three types of NER tasks, including flat, nested and discontinuous entity…