6 papers
Emergence of Context Characteristics Sensitivity in Large Language Models
Nadya Yuki Wangsajaya, Haeun Yu, Isabelle Augenstein
During instruction fine-tuning (IFT), large language models (LLMs) learn to follow instructions by using the provided context to answer a query. While prior work has studied how co…
CUB: Benchmarking Context Utilisation Techniques for Language Models
Lovisa Hagström, Youna Kim, Haeun Yu +4
Incorporating external knowledge is crucial for knowledge-intensive tasks, such as question answering and fact checking. However, language models (LMs) may ignore relevant informat…
BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Elicitation
Sekh Mainul Islam, Nadav Borenstein, Siddhesh Milind Pawar +3
Understanding biases and stereotypes encoded in the weights of Large Language Models (LLMs) is crucial for developing effective mitigation strategies. However, biased behaviour is…
CulTrace: Tracing Internal Cultural Reasoning in Large Language Models
Haeun Yu, Arnav Arora Seogyeong Jeong, Seogyeong Jeong +8
The growing deployment of large language models (LLMs) across diverse cultural contexts necessitates a deeper understanding of models' hidden representations of different cultures.…
A Reality Check on Context Utilisation for Retrieval-Augmented Generation
Lovisa Hagström, Sara Vera MarjanoviÄ, Haeun Yu +5
Retrieval-augmented generation (RAG) helps address the limitations of parametric knowledge embedded within a language model (LM). In real world settings, retrieved information can…
DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models
Sara Vera MarjanoviÄ, Haeun Yu, Pepa Atanasova +3
Knowledge-intensive language understanding tasks require Language Models (LMs) to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated k…