5 papers
Counsel: A Meta-Evaluation Dataset for Agentic Tasks
Sashank Pisupati, Henry Broomfield, Eujeong Choi +5
As agentic systems tackle increasingly complex multi-step tasks, evaluating their trajectories presents a major bottleneck - human annotation of a single trajectory on popular agen…
Learning is Forgetting: LLM Training As Lossy Compression
Henry C. Conklin, Tom Hosking, Tan Yi-Chern +5
Despite the increasing prevalence of large language models (LLMs), we still have a limited understanding of how their representational spaces are structured. This limits our abilit…
Reverse Engineering Human Preferences with Reinforcement Learning
Lisa Alazraki, Tan Yi-Chern, Jon Ander Campos +3
The capabilities of Large Language Models (LLMs) are routinely evaluated by other LLMs trained to predict human preferences. This framework--known as LLM-as-a-judge--is highly scal…
Command A: An Enterprise-Ready Large Language Model
Team Cohere, :, Aakanksha +227
In this report we describe the development of Command A, a powerful large language model purpose-built to excel at real-world enterprise use cases. Command A is an agent-optimised…
Atla Selene Mini: A General Purpose Evaluation Model
Andrei Alexandru, Antonia Calvi, Henry Broomfield +9
We introduce Atla Selene Mini, a state-of-the-art small language model-as-a-judge (SLMJ). Selene Mini is a general-purpose evaluator that outperforms the best SLMJs and GPT-4o-mini…