6 papers
ABC implies that Ramanujan's tau function misses almost all primes
David Kurniadi Angdinata, Evan Chen, Chris Cummins +21
Lehmer conjectured that Ramanujan's tau-function never vanishes. In a related direction, a folklore conjecture asserts that infinitely many primes arise as absolute values of Raman…
ProofOptimizer: Training Language Models to Simplify Proofs without Human Demonstrations
Alex Gu, Bartosz Piotrowski, Fabian Gloeckle +2
Neural theorem proving has advanced rapidly in the past year, reaching IMO gold-medalist capabilities and producing formal proofs that span thousands of lines. Although such proofs…
Continual Learning via Sparse Memory Finetuning
Jessy Lin, Luke Zettlemoyer, Gargi Ghosh +4
Modern language models are powerful, but typically static after deployment. A major obstacle to building models that continually learn over time is catastrophic forgetting, where u…
Detecting Prefix Bias in LLM-based Reward Models
Ashwin Kumar, Yuzi He, Aram H. Markosyan +2
Reinforcement Learning with Human Feedback (RLHF) has emerged as a key paradigm for task-specific fine-tuning of language models using human preference data. While numerous publicl…
What I cannot execute, I do not understand: Training and Evaluating LLMs on Program Execution Traces
Jordi Armengol-Estapé, Quentin Carbonneaux, Tianjun Zhang +8
Code generation and understanding are critical capabilities for large language models (LLMs). Thus, most LLMs are pretrained and fine-tuned on code data. However, these datasets ty…
With Great Backbones Comes Great Adversarial Transferability
Erik Arakelyan, Karen Hambardzumyan, Davit Papikyan +4
Advances in self-supervised learning (SSL) for machine vision have improved representation robustness and model performance, giving rise to pre-trained backbones like \emph{ResNet}…