11 citations · 17 across the 10 of their papers we have counts for
4 papers · 1 filter
Beyond pass@k: Redundancy-Aware RLVR for Multi-Sample Code Generation
Le Bronnec Florian, Alexandre Verine, Rio Yokota +1
LLMs for code generation are commonly evaluated in repeated-sampling settings using Pass@k, where multiple candidate programs are executed against unit tests under a finite samplin…
Improving Diversity in Language Models: When Temperature Fails, Change the Loss
Alexandre Verine, Florian Le Bronnec, Kunhao Zheng +3
Increasing diversity in language models is a challenging yet essential objective. A common approach is to raise the decoding temperature. In this work, we investigate this approach…
What Makes Large Language Models Reason in (Multi-Turn) Code Generation?
Kunhao Zheng, Juliette Decugis, Jonas Gehring +3
Prompting techniques such as chain-of-thought have established themselves as a popular vehicle for improving the outputs of large language models (LLMs). For code generation, howev…
Exploring Precision and Recall to assess the quality and diversity of LLMs
Florian Le Bronnec, Alexandre Verine, Benjamin Negrevergne +2
We introduce a novel evaluation framework for Large Language Models (LLMs) such as \textsc{Llama-2} and \textsc{Mistral}, focusing on importing Precision and Recall metrics from im…