10 citations · 38 across the 45 of their papers we have counts for
27 papers · 1 filter
When Can Proxies Improve the Sample Complexity of Preference Learning?
Yuchen Zhu, Daniel Augusto de Souza, Zhengyan Shi +4
We address the problem of reward hacking, where maximising a proxy reward does not necessarily increase the true reward. This is a key concern for Large Language Models (LLMs), as…
Mixtures of In-Context Learners
Giwon Hong, Emile van Krieken, Edoardo Ponti +2
In-context learning (ICL) adapts LLMs by providing demonstrations without fine-tuning the model parameters; however, it does not differentiate between demonstrations and quadratica…
Aligning Generalisation Between Humans and Machines
Filip Ilievski, Barbara Hammer, Frank van Harmelen +22
Recent advances in AI -- including generative approaches -- have resulted in technology that can support humans in scientific discovery and forming decisions, but may also disrupt…
DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations
Aryo Pradipta Gema, Chen Jin, Ahmed Abdulaal +5
Large Language Models (LLMs) often hallucinate, producing unfaithful or factually incorrect outputs by misrepresenting the provided context or incorrectly recalling internal knowle…
Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs
Xin Zhou, Ping Nie, Yiwen Guo +7
Retrieval-Augmented Generation (RAG) significantly improved the ability of Large Language Models (LLMs) to solve knowledge-intensive tasks. While existing research seeks to enhance…
An Auditing Test To Detect Behavioral Shift in Language Models
Leo Richter, Xuanli He, Pasquale Minervini +1
As language models (LMs) approach human-level performance, a comprehensive understanding of their behavior becomes crucial. This includes evaluating capabilities, biases, task perf…
Analysing the Residual Stream of Language Models Under Knowledge Conflicts
Yu Zhao, Xiaotang Du, Giwon Hong +6
Large language models (LLMs) can store a significant amount of factual knowledge in their parameters. However, their parametric knowledge may conflict with the information provided…
Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering
Yu Zhao, Alessio Devoto, Giwon Hong +6
Large language models (LLMs) can store a significant amount of factual knowledge in their parameters. However, their parametric knowledge may conflict with the information provided…
Is Complex Query Answering Really Complex?
Cosimo Gregucci, Bo Xiong, Daniel Hernandez +4
Complex query answering (CQA) on knowledge graphs (KGs) is gaining momentum as a challenging reasoning task. In this paper, we show that the current benchmarks for CQA might not be…
FLARE: Faithful Logic-Aided Reasoning and Exploration
Erik Arakelyan, Pasquale Minervini, Pat Verga +2
Modern Question Answering (QA) and Reasoning approaches based on Large Language Models (LLMs) commonly use prompting techniques, such as Chain-of-Thought (CoT), assuming the result…
Dot Product is All You Need: Bridging the Gap Between Item Recommendation and Link Prediction
Daniele Malitesta, Alberto Carlo Maria Mancino, Pasquale Minervini +1
Item recommendation (the task of predicting if a user may interact with new items from the catalogue in a recommendation system) and link prediction (the task of identifying missin…
Adaptive Layer Selection for Efficient Vision Transformer Fine-Tuning
Alessio Devoto, Federico Alvetreti, Jary Pomponi +3
Recently, foundation models based on Vision Transformers (ViTs) have become widely available. However, their fine-tuning process is highly resource-intensive, and it hinders their…
Attention Is All You Need But You Don't Need All Of It For Inference of Large Language Models
Georgy Tyukin, Gbetondji J-S Dovonon, Jean Kaddour +1
The inference demand for LLMs has skyrocketed in recent months, and serving models with low latencies remains challenging due to the quadratic input length complexity of the attent…
Probing the Emergence of Cross-lingual Alignment during LLM Training
Hetong Wang, Pasquale Minervini, Edoardo M. Ponti
Multilingual Large Language Models (LLMs) achieve remarkable levels of zero-shot cross-lingual transfer performance. We speculate that this is predicated on their ability to align…
SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages
Gayane Ghazaryan, Erik Arakelyan, Pasquale Minervini +1
Question Answering (QA) datasets have been instrumental in developing and evaluating Large Language Model (LLM) capabilities. However, such datasets are scarce for languages other…
A Simple and Effective Norm-Based Strategy for KV Cache Compression
Alessio Devoto, Yu Zhao, Simone Scardapane +1
The deployment of large language models (LLMs) is often hindered by the extensive memory requirements of the Key-Value (KV) cache, especially as context lengths increase. Existing…
Are We Done with MMLU?
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong +13
Maybe not. We identify and analyse errors in the popular Massive Multitask Language Understanding (MMLU) benchmark. Even though MMLU is widely adopted, our analysis demonstrates nu…
Edinburgh Clinical NLP at MEDIQA-CORR 2024: Guiding Large Language Models with Hints
Aryo Pradipta Gema, Chaeeun Lee, Pasquale Minervini +3
The MEDIQA-CORR 2024 shared task aims to assess the ability of Large Language Models (LLMs) to identify and correct medical errors in clinical notes. In this study, we evaluate the…
Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learning
Simon Yu, Jie He, Pasquale Minervini +1
With the emergence of large language models, such as LLaMA and OpenAI GPT-3, In-Context Learning (ICL) gained significant attention due to its effectiveness and efficiency. However…
The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models
Giwon Hong, Aryo Pradipta Gema, Rohit Saxena +8
Large Language Models (LLMs) have transformed the Natural Language Processing (NLP) landscape with their remarkable ability to understand and generate human-like text. However, the…
TuBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuning
Xuanli He, Jun Wang, Qiongkai Xu +4
The implications of backdoor attacks on English-centric large language models (LLMs) have been widely examined - such attacks can be achieved by embedding malicious behaviors durin…
Forklift: An Extensible Neural Lifter
Jordi Armengol-Estapé, Rodrigo C. O. Rocha, Jackson Woodruff +2
The escalating demand to migrate legacy software across different Instruction Set Architectures (ISAs) has driven the development of assembly-to-assembly translators to map between…
On the Independence Assumption in Neurosymbolic Learning
Emile van Krieken, Pasquale Minervini, Edoardo M. Ponti +1
State-of-the-art neurosymbolic learning systems use probabilistic reasoning to guide neural networks towards predictions that conform to logical constraints over symbols. Many such…
Edinburgh Clinical NLP at SemEval-2024 Task 2: Fine-tune your model unless you have access to GPT-4
Aryo Pradipta Gema, Giwon Hong, Pasquale Minervini +2
The NLI4CT task assesses Natural Language Inference systems in predicting whether hypotheses entail or contradict evidence from Clinical Trial Reports. In this study, we evaluate v…