28 papers
Evolutionary Feature Engineering for Structured Data
Ege Onur Taga, Yilin Zhuang, M. Emrullah Ildiz +4
Large language models are increasingly used as open-ended search operators in evolutionary optimization. We introduce Evolutionary Feature Engineering (EFE), a framework for using…
Stochastic Sparse Attention for Memory-Bound Inference
Kyle Lee, Corentin Delacour, Kevin Callahan-Coray +5
Autoregressive decoding becomes bandwidth-limited at long contexts, as generating each token requires reading all key and value vectors from KV cache. We present Stochastic A…
Learning to Bet for Horizon-Aware Anytime-Valid Testing
Ege Onur Taga, Samet Oymak, Shubhanshu Shekhar
We develop horizon-aware anytime-valid tests and confidence sequences for bounded means under a strict deadline . Using the betting/e-process framework, we cast horizon-aware be…
Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery
Halil Alperen Gozeten, Xuechen Zhang, Emrullah Ildiz +3
Recent LLM-guided evolutionary search methods have shown that iterative program mutation can discover strong algorithms, but they typically optimize each task independently, even w…
Latent Chain-of-Thought Improves Structured-Data Transformers
Carson Dudley, Samet Oymak
Chain-of-thought and more broadly test-time compute are known to augment the expressive capabilities of language models and have led to major innovations in reasoning. Motivated by…
VSPO: Vector-Steered Policy Optimization for Behavioral Control
Xuechen Zhang, Zijian Huang, Kai Yang +3
Modern language models often need to optimize a primary accuracy objective while also accommodating secondary behavioral preferences, such as verbosity, agreeableness, or the level…