4 papers
InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context
Xin Teng, Canyu Zhang, Shaoyi Zheng +3
Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precomp…
Submodular Context Partitioning and Compression for In-Context Learning
Shaoyi Zheng, Canyu Zhang, Tianyi Zhou +1
In-context learning (ICL) enables efficient few-shot learning in large language models (LLMs) without training, but suffers from the quadratic input complexity of transformers, lim…
Tilted Sharpness-Aware Minimization
Tian Li, Tianyi Zhou, Jeffrey A. Bilmes
Sharpness-Aware Minimization (SAM) has been demonstrated to improve the generalization performance of overparameterized models by seeking flat minima on the loss landscape through…
Many-Objective Multi-Solution Transport
Ziyue Li, Tian Li, Virginia Smith +2
Optimizing the performance of many objectives (instantiated by tasks or clients) jointly with a few Pareto stationary solutions (models) is critical in machine learning. However, p…