13 citations · 33 across the 14 of their papers we have counts for
10 papers · 1 filter
Evolutionary Feature Engineering for Structured Data
Ege Onur Taga, Yilin Zhuang, M. Emrullah Ildiz +4
Large language models are increasingly used as open-ended search operators in evolutionary optimization. We introduce Evolutionary Feature Engineering (EFE), a framework for using…
Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery
Halil Alperen Gozeten, Xuechen Zhang, Emrullah Ildiz +3
Recent LLM-guided evolutionary search methods have shown that iterative program mutation can discover strong algorithms, but they typically optimize each task independently, even w…
Learning to Correct: Calibrated Reinforcement Learning for Multi-Attempt Chain-of-Thought
Muhammed Emrullah Ildiz, Halil Alperen Gozeten, Ege Onur Taga +1
State-of-the-art reasoning models utilize long chain-of-thought (CoT) to solve increasingly complex problems using more test-time computation. In this work, we explore a long CoT s…
Continuous Chain of Thought Enables Parallel Exploration and Reasoning
Halil Alperen Gozeten, M. Emrullah Ildiz, Xuechen Zhang +3
Modern language models generate chain-of-thought traces by autoregressively sampling tokens from a finite vocabulary. While this discrete sampling has achieved remarkable success,…
Test-Time Training Provably Improves Transformers as In-context Learners
Halil Alperen Gozeten, M. Emrullah Ildiz, Xuechen Zhang +3
Test-time training (TTT) methods explicitly update the weights of a model to adapt to the specific test instance, and they have found success in a variety of settings, including mo…
TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data
Ege Onur Taga, M. Emrullah Ildiz, Samet Oymak
The diversity of time series applications and scarcity of domain-specific data highlight the need for time-series models with strong few-shot learning capabilities. In this work, w…
Retrieval Augmented Time Series Forecasting
Kutay Tire, Ege Onur Taga, Muhammed Emrullah Ildiz +1
Retrieval-augmented generation (RAG) is a central component of modern LLM systems, particularly in scenarios where up-to-date information is crucial for accurately responding to us…
Mechanics of Next Token Prediction with Self-Attention
Yingcong Li, Yixiao Huang, M. Emrullah Ildiz +2
Transformer-based language models are trained on large datasets to predict the next token given an input sequence. Despite this simple training objective, they have led to revoluti…
From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers
M. Emrullah Ildiz, Yixiao Huang, Yingcong Li +2
Modern language models rely on the transformer architecture and attention mechanism to perform language understanding and text generation. In this work, we study learning a 1-layer…
Transformers as Algorithms: Generalization and Stability in In-context Learning
Yingcong Li, M. Emrullah Ildiz, Dimitris Papailiopoulos +1
In-context learning (ICL) is a type of prompting where a transformer model operates on a sequence of (input, output) examples and performs inference on-the-fly. In this work, we fo…