7 papers
Regression Language Models for Code
Yash Akhauri, Xingyou Song, Arissa Wongpanich +2
We study code-to-metric regression: predicting numeric outcomes of code executions, a challenging task due to the open-ended nature of programming languages. While prior methods ha…
Towards Robust Mathematical Reasoning
Thang Luong, Dawsen Hwang, Hoang H. Nguyen +17
Finding the right north-star metrics is highly critical for advancing the mathematical reasoning capabilities of foundation models, especially given that existing evaluations are e…
Language Model Embeddings Can Be Sufficient for Bayesian Optimization
Tung Nguyen, Qiuyi Zhang, Bangding Yang +6
Bayesian Optimization is ubiquitous in experimental design and black-box optimization for improving search efficiency. However, most existing approaches rely on regression models w…
Decoding-based Regression
Xingyou Song, Dara Bahri
Language models have recently been shown capable of performing regression wherein numeric predictions are represented as decoded strings. In this work, we provide theoretical groun…
Performance Prediction for Large Systems via Text-to-Text Regression
Yash Akhauri, Bryan Lewandowski, Cheng-Hsi Lin +7
In many industries, predicting metric outcomes of large systems is a fundamental problem, driven largely by traditional tabular regression. However, such methods struggle on comple…
Understanding LLM Embeddings for Regression
Eric Tang, Bangding Yang, Xingyou Song
With the rise of large language models (LLMs) for flexibly processing information as strings, a natural application is regression, specifically by preprocessing string representati…