23 citations · 31 across the 10 of their papers we have counts for
11 papers
LLMs versus the Halting Problem: Characterizing Program Termination Reasoning
Oren Sultan, Jordi Armengol-Estape, Pascal Kesseli +4
Determining whether a program terminates is a central problem in computer science. Turing's Halting Problem established termination as undecidable, showing that no algorithm can un…
CWM: An Open-Weights LLM for Research on Code Generation with World Models
FAIR CodeGen team, Jade Copet, Quentin Carbonneaux +48
We release Code World Model (CWM), a 32-billion-parameter open-weights LLM, to advance research on code generation with world models. To improve code understanding beyond what can…
What I cannot execute, I do not understand: Training and Evaluating LLMs on Program Execution Traces
Jordi Armengol-Estapé, Quentin Carbonneaux, Tianjun Zhang +8
Code generation and understanding are critical capabilities for large language models (LLMs). Thus, most LLMs are pretrained and fine-tuned on code data. However, these datasets ty…
Don't Transform the Code, Code the Transforms: Towards Precise Code Rewriting using LLMs
Chris Cummins, Volker Seeker, Jordi Armengol-Estapé +3
Tools for rewriting, refactoring and optimizing code should be fast and correct. Large language models (LLMs), by their nature, possess neither of these qualities. Yet, there remai…
On the Limits of Multi-modal Meta-Learning with Auxiliary Task Modulation Using Conditional Batch Normalization
Jordi Armengol-Estapé, Vincent Michalski, Ramnath Kumar +3
Few-shot learning aims to learn representations that can tackle novel tasks given a small number of examples. Recent studies show that cross-modal learning can improve representati…
Forklift: An Extensible Neural Lifter
Jordi Armengol-Estapé, Rodrigo C. O. Rocha, Jackson Woodruff +2
The escalating demand to migrate legacy software across different Instruction Set Architectures (ISAs) has driven the development of assembly-to-assembly translators to map between…
Sequence-to-Sequence Resources for Catalan
Ona de Gibert, Ksenia Kharitonova, Blanca Calvo Figueras +2
In this work, we introduce sequence-to-sequence language resources for Catalan, a moderately under-resourced language, towards two tasks, namely: Summarization and Machine Translat…
Spanish Legalese Language Model and Corpora
Asier Gutiérrez-Fandiño, Jordi Armengol-Estapé, Aitor Gonzalez-Agirre +1
There are many Language Models for the English language according to its worldwide relevance. However, for the Spanish language, even if it is a widely spoken language, there are v…
Biomedical and Clinical Language Models for Spanish: On the Benefits of Domain-Specific Pretraining in a Mid-Resource Scenario
Casimiro Pio Carrino, Jordi Armengol-Estapé, Asier Gutiérrez-Fandiño +4
This work presents biomedical and clinical language models for Spanish by experimenting with different pretraining choices, such as masking at word and subword level, varying the v…
Spanish Biomedical Crawled Corpus: A Large, Diverse Dataset for Spanish Biomedical Language Models
Casimiro Pio Carrino, Jordi Armengol-Estapé, Ona de Gibert Bonet +4
We introduce CoWeSe (the Corpus Web Salud Español), the largest Spanish biomedical corpus to date, consisting of 4.5GB (about 750M tokens) of clean plain text. CoWeSe is the result…
On the Multilingual Capabilities of Very Large-Scale English Language Models
Jordi Armengol-Estapé, Ona de Gibert Bonet, Maite Melero
Generative Pre-trained Transformers (GPTs) have recently been scaled to unprecedented sizes in the history of machine learning. These models, solely trained on the language modelin…