13 citations · 17 across the 11 of their papers we have counts for
11 papers
Debias-SparseGPT: Bias-Aware Pruning for Large Language Models
Irina Proskurina, Guillaume Metzler, Antoine Gourru +1
Model compression techniques such as pruning and quantization facilitate the efficient deployment and acceleration of Large Language Models (LLMs). However, recent studies show tha…
Are You Sure You're Sure? On the Impact of Instruction Tuning on Confidence and Lexical Diversity
Irina Proskurina, Mayank Kumar, Oyindolapo O. Komolafe
Instruction-tuned language models achieve strong performance across a range of generation tasks but have recently been shown to exhibit verbalized overconfidence, which may manifes…
The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data
Irina Proskurina, Antoine Gourru, Julien Velcin
Generative models trained on artificially generated data have been shown to exhibit model collapse, resulting in significant performance degradation. As synthetic content increasin…
RILEC: Detection and Generation of L1 Russian Interference Errors in English Learner Texts
Darya Kharlamova, Irina Proskurina
Many errors in student essays can be explained by influence from the native language (L1). L1 interference refers to errors influenced by a speaker's first language, such as using…
HatePrototypes: Interpretable and Transferable Representations for Implicit and Explicit Hate Speech Detection
Irina Proskurina, Marc-Antoine Carpentier, Julien Velcin
Optimization of offensive content moderation models for different types of hateful messages is typically achieved through continued pre-training or fine-tuning on new hate speech b…
Fair-GPTQ: Bias-Aware Quantization for Large Language Models
Irina Proskurina, Guillaume Metzler, Julien Velcin
The high memory demands of generative language models have drawn attention to quantization, which reduces memory usage by mapping model weights to lower-precision integers. However…
Histoires Morales: A French Dataset for Assessing Moral Alignment
Thibaud Leteno, Irina Proskurina, Antoine Gourru +4
Aligning language models with human values is crucial, especially as they become more integrated into everyday life. While models are often adapted to user preferences, it is equal…
When Quantization Affects Confidence of Large Language Models?
Irina Proskurina, Luc Brun, Guillaume Metzler +1
Recent studies introduced effective compression techniques for Large Language Models (LLMs) via post-training quantization or low-bit weight representation. Although quantized weig…
Mini Minds: Exploring Bebeshka and Zlata Baby Models
Irina Proskurina, Guillaume Metzler, Julien Velcin
In this paper, we describe the University of Lyon 2 submission to the Strict-Small track of the BabyLM competition. The shared task is created with an emphasis on small-scale langu…
Can BERT eat RuCoLA? Topological Data Analysis to Explain
Irina Proskurina, Irina Piontkovskaya, Ekaterina Artemova
This paper investigates how Transformer language models (LMs) fine-tuned for acceptability classification capture linguistic features. Our approach uses the best practices of topol…
Acceptability Judgements via Examining the Topology of Attention Maps
Daniil Cherniavskii, Eduard Tulchinskii, Vladislav Mikhailov +7
The role of the attention mechanism in encoding linguistic knowledge has received special interest in NLP. However, the ability of the attention heads to judge the grammatical acce…