1 citations · 1 across the 17 of their papers we have counts for
19 papers
RoLaQ: Joint Rotation and Range Calibration for 2-Bit KV Caches in Multi-Head Latent Attention
Shiyu Ji, Peng Ling, Chen Lin +4
Multi-head latent attention (MLA) compresses the content of a key-value cache into a shared latent per token, making low-bit storage attractive for long-context inference. At two b…
Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction
Haresh Rengaraj Rajamohan, Xiang Gao, Weicheng Zhu +10
While large-scale pretraining has revolutionized language modeling, its potential remains underexplored in healthcare with structured electronic health records (EHRs). We present R…
EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction
Shiyu Ji, Yixuan Wang, Yijun Liu +2
The increasing memory demand of the Key-Value (KV) cache poses a significant bottleneck for Large Language Models (LLMs) in long-context applications. Existing low-rank KV compress…
Seer Self-Consistency: Advance Budget Estimation for Adaptive Test-Time Scaling
Shiyu Ji, Yixuan Wang, Yijun Liu +2
Test-time scaling improves the inference performance of Large Language Models (LLMs) but also incurs substantial computational costs. Although recent studies have reduced token con…
Automated Snippet-Alignment Data Augmentation for Code Translation
Zhiming Zhang, Qingfu Zhu, Xianzhen Luo +3
Code translation aims to translate the code from its source language to the target language and is used in various software development scenarios. Recent developments in Large Lang…
RE: Improving Chinese Grammatical Error Correction via Retrieving Appropriate Examples with Explanation
Baoxin Wang, Yumeng Luo, Yixuan Wang +3
The primary objective of Chinese grammatical error correction (CGEC) is to detect and correct errors in Chinese sentences. Recent research shows that large language models (LLMs) h…
ProxyAttn: Guided Sparse Attention via Representative Heads
Yixuan Wang, Huang He, Siqi Bao +4
The quadratic complexity of attention mechanisms limits the efficiency of Large Language Models (LLMs) on long-text tasks. Recently, methods that dynamically estimate block importa…
Judge Q: Trainable Queries for Optimized Information Retention in KV Cache Eviction
Yijun Liu, Yixuan Wang, Yuzhuang Xu +4
Large language models (LLMs) utilize key-value (KV) cache to store historical information during sequence processing. The size of KV cache grows linearly as the length of the seque…
CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing
Yixuan Wang, Haoyu Qiao, Lujun Li +2
Large Language Models (LLMs) confront significant memory challenges due to the escalating KV cache with increasing sequence length. As a crucial technique, existing cross-layer KV…
CAMERA: Multi-Matrix Joint Compression for MoE Models via Micro-Expert Redundancy Analysis
Yuzhuang Xu, Xu Han, Yuanchi Zhang +5
Large Language Models (LLMs) with Mixture-of-Experts (MoE) architectures are distinguished by their strong performance scaling with increasing parameters across a wide range of tas…
Tag-Evol: Achieving Efficient Instruction Evolving via Tag Injection
Yixuan Wang, Shiqi Zhou, Chuanzhe Guo +1
Evol-Instruct has made significant improvements as a data synthesis method in several areas. Existing methods typically rely on a fixed set of strategies to evolve, which require m…
Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decoding
Yixuan Wang, Yijun Liu, Shiyu ji +4
Large language models (LLMs) suffer from high inference latency due to the auto-regressive decoding process. Speculative decoding accelerates inference by generating multiple draft…
Success is in the Details: Evaluate and Enhance Details Sensitivity of Code LLMs through Counterfactuals
Xianzhen Luo, Qingfu Zhu, Zhiming Zhang +8
Code Sensitivity refers to the ability of Code LLMs to recognize and respond to details changes in problem descriptions. While current code benchmarks and instruction data focus on…
Lookahead Q-Cache: Achieving More Consistent KV Cache Eviction via Pseudo Query
Yixuan Wang, Shiyu Ji, Yijun Liu +4
Large language models (LLMs) rely on key-value cache (KV cache) to accelerate decoding by reducing redundant computations. However, the KV cache memory usage grows substantially wi…
Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling
Xianzhen Luo, Yixuan Wang, Qingfu Zhu +4
Massive parameters of LLMs have made inference latency a fundamental bottleneck. Speculative decoding represents a lossless approach to accelerate inference through a guess-and-ver…
Improving Grammatical Error Correction via Contextual Data Augmentation
Yixuan Wang, Baoxin Wang, Yijun Liu +3
Nowadays, data augmentation through synthetic data has been widely used in the field of Grammatical Error Correction (GEC) to alleviate the problem of data scarcity. However, these…
Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training
Yixuan Wang, Xianzhen Luo, Fuxuan Wei +6
Existing speculative decoding methods typically require additional model structure and training processes to assist the model for draft token generation. This makes the migration o…
LM-Combiner: A Contextual Rewriting Model for Chinese Grammatical Error Correction
Yixuan Wang, Baoxin Wang, Yijun Liu +2
Over-correction is a critical problem in Chinese grammatical error correction (CGEC) task. Recent work using model ensemble methods based on voting can effectively mitigate over-co…
CSED: A Chinese Semantic Error Diagnosis Corpus
Bo Sun, Baoxin Wang, Yixuan Wang +4
Recently, much Chinese text error correction work has focused on Chinese Spelling Check (CSC) and Chinese Grammatical Error Diagnosis (CGED). In contrast, little attention has been…