8 papers
Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
Gabriele Prato, Shagun Sodhani, Alessandro Sordoni +1
The standard practice for training large language models involves packing multiple documents together to optimize computational efficiency. However, the impact of this process on t…
Small Encoders Can Rival Large Decoders in Detecting Groundedness
Istabrak Abbes, Gabriele Prato, Quentin Fournier +4
Augmenting large language models (LLMs) with external context significantly improves their performance in natural language processing (NLP) tasks. However, LLMs struggle to answer…
Do Large Language Models Know How Much They Know?
Gabriele Prato, Jerry Huang, Prasanna Parthasarathi +2
Large Language Models (LLMs) have emerged as highly capable systems and are increasingly being integrated into various uses. However, the rapid pace of their deployment has outpace…
EpiK-Eval: Evaluation for Language Models as Epistemic Models
Gabriele Prato, Jerry Huang, Prasannna Parthasarathi +2
In the age of artificial intelligence, the role of large language models (LLMs) is becoming increasingly central. Despite their growing prevalence, their capacity to consolidate kn…
PatchBlender: A Motion Prior for Video Transformers
Gabriele Prato, Yale Song, Janarthanan Rajendran +3
Transformers have become one of the dominant architectures in the field of computer vision. However, there are yet several challenges when applying such architectures to video data…
Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers
Gabriele Prato, Simon Guiroy, Ethan Caballero +2
Empirical science of neural scaling laws is a rapidly growing area of significant importance to the future of machine learning, particularly in the light of recent breakthroughs ac…
Fully Quantized Transformer for Machine Translation
Gabriele Prato, Ella Charlaix, Mehdi Rezagholizadeh
State-of-the-art neural machine translation methods employ massive amounts of parameters. Drastically reducing computational costs of such methods without affecting performance has…
Towards Lossless Encoding of Sentences
Gabriele Prato, Mathieu Duchesneau, Sarath Chandar +1
A lot of work has been done in the field of image compression via machine learning, but not much attention has been given to the compression of natural language. Compressing text i…