12 papers
Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction
Haresh Rengaraj Rajamohan, Xiang Gao, Weicheng Zhu +10
While large-scale pretraining has revolutionized language modeling, its potential remains underexplored in healthcare with structured electronic health records (EHRs). We present R…
Seer Self-Consistency: Advance Budget Estimation for Adaptive Test-Time Scaling
Shiyu Ji, Yixuan Wang, Yijun Liu +2
Test-time scaling improves the inference performance of Large Language Models (LLMs) but also incurs substantial computational costs. Although recent studies have reduced token con…
Automated Snippet-Alignment Data Augmentation for Code Translation
Zhiming Zhang, Qingfu Zhu, Xianzhen Luo +3
Code translation aims to translate the code from its source language to the target language and is used in various software development scenarios. Recent developments in Large Lang…
RE: Improving Chinese Grammatical Error Correction via Retrieving Appropriate Examples with Explanation
Baoxin Wang, Yumeng Luo, Yixuan Wang +3
The primary objective of Chinese grammatical error correction (CGEC) is to detect and correct errors in Chinese sentences. Recent research shows that large language models (LLMs) h…
ProxyAttn: Guided Sparse Attention via Representative Heads
Yixuan Wang, Huang He, Siqi Bao +4
The quadratic complexity of attention mechanisms limits the efficiency of Large Language Models (LLMs) on long-text tasks. Recently, methods that dynamically estimate block importa…
Judge Q: Trainable Queries for Optimized Information Retention in KV Cache Eviction
Yijun Liu, Yixuan Wang, Yuzhuang Xu +4
Large language models (LLMs) utilize key-value (KV) cache to store historical information during sequence processing. The size of KV cache grows linearly as the length of the seque…