8 papers
AttSVD:Prompt-Adaptive Low-Rank KV Cache Compression via Attention-Guided SVD
Sara Abdali, Jongwoo Ko, Pashmina Cameron
The key-value (KV) cache of autoregressive transformers grows linearly with context length and dominates memory at long context. Most training-free remedies evict low-importance to…
Scaling Reasoning Efficiently via Relaxed On-Policy Distillation
Jongwoo Ko, Sara Abdali, Young Jin Kim +2
On-policy distillation is pivotal for transferring reasoning capabilities to capacity-constrained models, yet remains prone to instability and negative transfer. We show that on-po…
StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
Tianyi Chen, Sihan Chen, Xiaoyi Qu +5
Quantization-aware training (QAT) is essential for deploying large models under strict memory and latency constraints, yet achieving stable and robust optimization at ultra-low bit…
CUA-Skill: Develop Skills for Computer Using Agent
Tianyi Chen, Yinheng Li, Michael Solodko +12
Computer-Using Agents (CUAs) aim to autonomously operate computer systems to complete real-world tasks. However, existing agentic systems remain difficult to scale and lag behind h…
AppSelectBench: Application-Level Tool Selection Benchmark
Tianyi Chen, Michael Solodko, Sen Wang +14
Computer Using Agents (CUAs) are increasingly equipped with external tools, enabling them to perform complex and realistic tasks. For CUAs to operate effectively, application selec…
WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
Sihan Chen, Dan Zhao, Jongwoo Ko +5
The growing computational demands of large language models (LLMs) make efficient inference and activation strategies increasingly critical. While recent approaches, such as Mixture…
DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
Jongwoo Ko, Tianyi Chen, Sungnyun Kim +4
Despite the success of distillation in large language models (LLMs), most prior work applies identical loss functions to both teacher- and student-generated data. These strategies…
HESSO: Towards Automatic Efficient and User Friendly Any Neural Network Training and Pruning
Tianyi Chen, Xiaoyi Qu, David Aponte +7
Structured pruning is one of the most popular approaches to effectively compress the heavy deep neural networks (DNNs) into compact sub-networks while retaining performance. The ex…