2 citations · 2 across the 5 of their papers we have counts for
8 papers
On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1D
Fabio J. Fehr, Philip Torr
General-purpose models can adapt to many tasks from context, while specialised models can execute individual functions with less capacity. Yet obtaining such specialists requires t…
Building Legal Reward Models for Grounding and Abstention
Rilton Franzone, Valentin Noël, Puyu Wang +2
Large language models are increasingly used in high-stakes domains such as law, where systems must ground their reasoning in retrieved evidence and abstain when that evidence is in…
When Rubrics Fail: Hallucinations Reveal Blind Spots in Medical AI Evaluation
Griffin Farrow, Lily Sijia Li, Jack Johnson +4
Hallucinations can undermine clinician trust in LLMs, making it important that evaluation methods capture clinically relevant errors. Rubric-based evaluation has become the leading…
CoRet: Improved Retriever for Code Editing
Fabio Fehr, Prabhu Teja Sivaprasad, Luca Franceschi +1
In this paper, we introduce CoRet, a dense retrieval model designed for code-editing tasks that integrates code semantics, repository structure, and call graph dependencies. The mo…
Nonparametric Variational Regularisation of Pretrained Transformers
Fabio Fehr, James Henderson
The current paradigm of large-scale pre-training and fine-tuning Transformer large language models has lead to significant improvements across the board in natural language process…
Learning to Abstract with Nonparametric Variational Information Bottleneck
Melika Behjati, Fabio Fehr, James Henderson
Learned representations at the level of characters, sub-words, words and sentences, have each contributed to advances in understanding different NLP tasks and linguistic phenomena.…
A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck
James Henderson, Fabio Fehr
We propose a VAE for Transformers by developing a variational information bottleneck regulariser for Transformer embeddings. We formalise the embedding space of Transformer encoder…
HyperMixer: An MLP-based Low Cost Alternative to Transformers
Florian Mai, Arnaud Pannatier, Fabio Fehr +4
Transformer-based architectures are the model of choice for natural language understanding, but they come at a significant cost, as they have quadratic complexity in the input leng…