288 citations · 380 across the 12 of their papers we have counts for
12 papers
ReasoningWeekly: A General Knowledge and Verbal Reasoning Challenge for Large Language Models
Zixuan Wu, Francesca Lucchetti, Aleksander Boruch-Gruszecki +5
Existing benchmarks for frontier models often test specialized, "PhD-level" knowledge that is difficult for non-experts to grasp. In contrast, we present a benchmark with 613 probl…
SelfCodeAlign: Self-Alignment for Code Generation
Yuxiang Wei, Federico Cassano, Jiawei Liu +7
Instruction tuning is a supervised fine-tuning approach that significantly improves the ability of large language models (LLMs) to follow human instructions. We propose SelfCodeAli…
StarCoder 2 and The Stack v2: The Next Generation
Anton Lozhkov, Raymond Li, Loubna Ben Allal +63
The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder2. In partnership wi…
Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions
Federico Cassano, Luisa Li, Akul Sethi +8
A significant amount of research is focused on developing and evaluating large language models for a variety of code synthesis tasks. These include synthesizing code from natural l…
npm-follower: A Complete Dataset Tracking the NPM Ecosystem
Donald Pinckney, Federico Cassano, Arjun Guha +1
Software developers typically rely upon a large network of dependencies to build their applications. For instance, the NPM package repository contains over 3 million packages and s…
Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs
Federico Cassano, John Gouwar, Francesca Lucchetti +7
Over the past few years, Large Language Models of Code (Code LLMs) have started to have a significant impact on programming practice. Code LLMs are also emerging as building blocks…
Type Prediction With Program Decomposition and Fill-in-the-Type Training
Federico Cassano, Ming-Ho Yee, Noah Shinn +2
TypeScript and Python are two programming languages that support optional type annotations, which are useful but tedious to introduce and maintain. This has motivated automated typ…
SafeLLVM: LLVM Without The ROP Gadgets!
Federico Cassano, Charles Bershatsky, Jacob Ginesin +1
Memory safety is a cornerstone of secure and robust software systems, as it prevents a wide range of vulnerabilities and exploitation techniques. Among these, we focus on Return-Or…
A Large Scale Analysis of Semantic Versioning in NPM
Donald Pinckney, Federico Cassano, Arjun Guha +1
The NPM package repository contains over two million packages and serves tens of billions of downloads per-week. Nearly every single JavaScript application uses the NPM package man…
Reflexion: Language Agents with Verbal Reinforcement Learning
Noah Shinn, Federico Cassano, Edward Berman +3
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challengin…
MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation
Federico Cassano, John Gouwar, Daniel Nguyen +10
Large language models have demonstrated the ability to generate both natural language and programming language text. Such models open up the possibility of multi-language code gene…
Flexible and Optimal Dependency Management via Max-SMT
Donald Pinckney, Federico Cassano, Arjun Guha +3
Package managers such as NPM have become essential for software development. The NPM repository hosts over 2 million packages and serves over 43 billion downloads every week. Unfor…