7 papers
Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
Xinghong Fu, Yanhong Li, Georgios Papaioannou +1
Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has be…
Fast KV Compaction via Attention Matching
Adam Zweiger, Xinghong Fu, Han Guo +1
Scaling language models to long contexts is often bottlenecked by the size of the key-value (KV) cache. In deployed settings, long contexts are typically managed through compaction…
PRiSM: Benchmarking Phone Realization in Speech Models
Shikhar Bharadwaj, Chin-Jou Li, Yoonjae Kim +13
Phone recognition (PR) serves as the atomic interface for language-agnostic modeling for cross-lingual speech processing and phonetic analysis. Despite prolonged efforts in develop…
MIDI-LLM: Improving Text-to-MIDI Music Generation via Adapting Large Language Models
Shih-Lun Wu, Dave Carlton, Ryan Miyakawa +3
We present MIDI-LLM, a recipe that improves multitrack text-to-MIDI generation via adapting Large Language Models (LLMs). MIDI-LLM expands an LLM's text vocabulary to include MIDI…
On the Duality between Gradient Transformations and Adapters
Lucas Torroba-Hennigen, Hunter Lang, Han Guo +1
We study memory-efficient optimization of neural networks (in particular language models) with linear gradient transformations, where the gradients are linearly mapped to a lower d…
ARES: Auxiliary Range Expansion for Outlier Synthesis
Eui-Soo Jung, Hae-Hun Seo, Hyun-Woo Jung +2
Recent successes of artificial intelligence and deep learning often depend on the well-collected training dataset which is assumed to have an identical distribution with the test d…