6 papers
Memory vs. Context? Influential Factors of Factual Recall in Language Models
Guilhem Fouilhé, Nicholas Asher, Philippe Muller
We reproduce and stress-test the work of Yu et al. (2023), who characterize how language models (LMs) arbitrate between memorized knowledge and contradictory in-context statements.…
Object-Level Explanations for Image Geolocation Models: a GeoGuessr use-case
Emilie Durrieu, Christophe Hurter, Philippe Muller +1
When humans play geolocation games such as GeoGuessr, they rely on concrete visual cues, such as road markings, vegetation, or architectural details, to infer where an image was ca…
Minimal Clips, Maximum Salience: Long Video Summarization via Key Moment Extraction
Galann Pennec, Zhengyuan Liu, Nicholas Asher +2
Vision-Language Models (VLMs) are able to process increasingly longer videos. Yet, important visual information is easily lost throughout the entire context and missed by VLMs. Als…
Integrating Video and Text: A Balanced Approach to Multimodal Summary Generation and Evaluation
Galann Pennec, Zhengyuan Liu, Nicholas Asher +2
Vision-Language Models (VLMs) often struggle to balance visual and textual information when summarizing complex multimodal inputs, such as entire TV show episodes. In this paper, w…
In2Core: Leveraging Influence Functions for Coreset Selection in Instruction Finetuning of Large Language Models
Ayrton San Joaquin, Bin Wang, Zhengyuan Liu +4
Despite advancements, fine-tuning Large Language Models (LLMs) remains costly due to the extensive parameter count and substantial data requirements for model generalization. Acces…
MO-DEHB: Evolutionary-based Hyperband for Multi-Objective Optimization
Noor Awad, Ayushi Sharma, Philipp Muller +2
Hyperparameter optimization (HPO) is a powerful technique for automating the tuning of machine learning (ML) models. However, in many real-world applications, accuracy is only one…