1 citations · 2 across the 14 of their papers we have counts for
14 papers
Sakana Fugu Technical Report
Yujin Tang, Edoardo Cetin, Jinglue Xu +11
The capabilities of frontier Large Language Models (LLMs) continue to advance, with different providers increasingly specializing in distinct domains. This raises a natural next ob…
Sparser, Faster, Lighter Transformer Language Models
Edoardo Cetin, Stefano Peluchetti, Emilio Castillo +3
Scaling autoregressive large language models (LLMs) has driven unprecedented progress but comes with vast computational costs. In this work, we tackle these costs by leveraging uns…
Learning from Partial Chain-of-Thought via Truncated-Reasoning Self-Distillation
Gianluigi Silvestri, Edoardo Cetin
Reasoning-oriented language models achieve strong performance by generating long chain-of-thought traces at inference time. However, this capability comes with substantial and ofte…
Doc-to-LoRA: Learning to Instantly Internalize Contexts
Rujikorn Charakorn, Edoardo Cetin, Shinnosuke Uesaka +1
Long input sequences are central to in-context learning, document understanding, and multi-step reasoning of Large Language Models (LLMs). However, the quadratic attention cost of…
Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings
Yoav Gelberg, Koshi Eguchi, Takuya Akiba +1
So far, expensive finetuning beyond the pretraining sequence length has been a requirement for effectively extending the context of language models (LM). In this work, we break thi…
TRINITY: An Evolved LLM Coordinator
Jinglue Xu, Qi Sun, Peter Schwendeman +3
Combining diverse foundation models is promising, but weight-merging is limited by mismatched architectures and closed APIs. Trinity addresses this with a lightweight coordinator t…
Learning to Orchestrate Agents in Natural Language with the Conductor
Stefan Nielsen, Edoardo Cetin, Peter Schwendeman +3
Powerful large language models (LLMs) from different providers have been expensively trained and finetuned to specialize across varying domains. In this work, we introduce a new ki…
ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
Robert Tjarko Lange, Yuki Imajuku, Edoardo Cetin
We introduce ShinkaEvolve: a new open-source framework leveraging large language models (LLMs) to advance scientific discovery with state-of-the-art performance and unprecedented e…
Text-to-LoRA: Instant Transformer Adaption
Rujikorn Charakorn, Edoardo Cetin, Yujin Tang +1
While Foundation Models provide a general tool for rapid content creation, they regularly require task-specific adaptation. Traditionally, this exercise involves careful curation o…
Reinforcement Learning Teachers of Test Time Scaling
Edoardo Cetin, Tianyu Zhao, Yujin Tang
Training reasoning language models (LMs) with reinforcement learning (RL) for one-hot correctness inherently relies on the LM being able to explore and solve its task with some cha…
Sudoku-Bench: Evaluating creative reasoning with Sudoku variants
Jeffrey Seely, Yuki Imajuku, Tianyu Zhao +2
Existing reasoning benchmarks for large language models (LLMs) frequently fail to capture authentic creativity, often rewarding memorization of previously observed patterns. We add…
Transformer-Squared: Self-adaptive LLMs
Qi Sun, Edoardo Cetin, Yujin Tang
Self-adaptive large language models (LLMs) aim to solve the challenges posed by traditional fine-tuning methods, which are often computationally intensive and static in their abili…
Large Language Models to Diffusion Finetuning
Edoardo Cetin, Tianyu Zhao, Yujin Tang
We propose a new finetuning method to provide pre-trained large language models (LMs) the ability to scale test-time compute through the diffusion framework. By increasing the numb…
An Evolved Universal Transformer Memory
Edoardo Cetin, Qi Sun, Tianyu Zhao +1
Prior methods propose to offset the escalating costs of modern foundation models by dropping specific parts of their contexts with hand-designed rules, while attempting to preserve…