2 citations · 2 across the 5 of their papers we have counts for
4 papers · 1 filter
On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1D
Fabio J. Fehr, Philip Torr
General-purpose models can adapt to many tasks from context, while specialised models can execute individual functions with less capacity. Yet obtaining such specialists requires t…
CoRet: Improved Retriever for Code Editing
Fabio Fehr, Prabhu Teja Sivaprasad, Luca Franceschi +1
In this paper, we introduce CoRet, a dense retrieval model designed for code-editing tasks that integrates code semantics, repository structure, and call graph dependencies. The mo…
Nonparametric Variational Regularisation of Pretrained Transformers
Fabio Fehr, James Henderson
The current paradigm of large-scale pre-training and fine-tuning Transformer large language models has lead to significant improvements across the board in natural language process…
A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck
James Henderson, Fabio Fehr
We propose a VAE for Transformers by developing a variational information bottleneck regulariser for Transformer embeddings. We formalise the embedding space of Transformer encoder…