14 citations · 41 across the 14 of their papers we have counts for
15 papers
At the Limits of Computability: Steps Towards Autonomous AI Discovery on the Busy Beaver Frontier
Marco Dos Santos, Vivek Veeriah, Alex Havrilla +4
Recent breakthroughs in AI for mathematics---ranging from Olympiad medal performances to resolving long-standing open conjectures---have rapidly expanded what automated systems can…
COrigami: An AI Pipeline for Co-Designing Flat-Foldable Visually Recognisable Origami
Tom Zahavy, Shaobo Hou, Thomas Tumiel +16
While generative AI has achieved remarkable success in solving problems with verifiable solutions, generating physical art that satisfies both strict geometric constraints and subj…
SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithms
Alex Havrilla, Edward Hughes, Mikayel Samvelyan +1
Large language model (LLM) driven synthetic data generation has emerged as a powerful method for improving model reasoning capabilities. However, most methods either distill large…
Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights
Zhaiming Shen, Alex Havrilla, Rongjie Lai +2
Transformers serve as the foundational architecture for large language and video generation models, such as GPT, BERT, SORA and their successors. Empirical studies have demonstrate…
IGDA: Interactive Graph Discovery through Large Language Model Agents
Alex Havrilla, David Alvarez-Melis, Nicolo Fusi
Large language models () have emerged as a powerful method for discovery. Instead of utilizing numerical data, LLMs utilize associated variable $\textit{semantic met…
Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models
Alex Havrilla, Andrew Dai, Laura O'Mahony +17
Synthetic data generation with Large Language Models is a promising paradigm for augmenting natural data over a nearly infinite range of tasks. Given this variety, direct compariso…
Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data
Alex Havrilla, Wenjing Liao
When training deep neural networks, a model's generalization error is often observed to follow a power scaling law dependent both on the model size and the data size. Perhaps the b…
Teaching Large Language Models to Reason with Reinforcement Learning
Alex Havrilla, Yuqing Du, Sharath Chandra Raparthy +6
Reinforcement Learning from Human Feedback (\textbf{RLHF}) has emerged as a dominant approach for aligning LLM outputs with human preferences. Inspired by the success of RLHF, we s…
Understanding the Effect of Noise in LLM Training Data with Algorithmic Chains of Thought
Alex Havrilla, Maia Iyer
During both pretraining and fine-tuning, Large Language Models (\textbf{LLMs}) are trained on trillions of tokens of text of widely varying quality. Both phases of training typical…
GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements
Alex Havrilla, Sharath Raparthy, Christoforus Nalmpantis +4
State-of-the-art language models can exhibit impressive reasoning refinement capabilities on math, science or coding tasks. However, recent work demonstrates that even the best mod…
ARB: Advanced Reasoning Benchmark for Large Language Models
Tomohiro Sawada, Daniel Paleka, Alexander Havrilla +6
Large Language Models (LLMs) have demonstrated remarkable performance on various quantitative reasoning and knowledge benchmarks. However, many of these benchmarks are losing utili…
Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustness
Hao Liu, Alex Havrilla, Rongjie Lai +1
Autoencoders have demonstrated remarkable success in learning low-dimensional latent features of high-dimensional data across various applications. Assuming that data are sampled n…
Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning
Louis Castricato, Alexander Havrilla, Shahbuland Matiana +5
Controlled automated story generation seeks to generate natural language stories satisfying constraints from natural language critiques or preferences. Existing methods to control…
Khinchin-type inequalities via Hadamard's factorisation
Alex Havrilla, Piotr Nayar, Tomasz Tkocz
We prove Khinchin-type inequalities with sharp constants for type L random variables and all even moments. Our main tool is Hadamard's factorisation theorem from complex analysis,…
Sharp Khinchin-type inequalities for symmetric discrete uniform random variables
Alex Havrilla, Tomasz Tkocz
We establish several optimal moment comparison inequalities (Khinchin-type inequalities) for weighted sums of independent identically distributed symmetric discrete random variable…