6 citations · 7 across the 13 of their papers we have counts for
10 papers · 1 filter
Language Models Learn Universal Representations of Numbers and Here's Why You Should Care
Michal Štefánik, Timothee Mickus, Marek Kadlčík +7
Prior work has shown that large language models (LLMs) often converge to accurate input embedding for numbers, based on sinusoidal representations. In this work, we quantify that t…
Can Out-of-Distribution Evaluations Uncover Reliance on Shortcuts? A Case Study in Question Answering
Michal Štefánik, Timothee Mickus, Marek Kadlčík +2
A majority of recent work in AI assesses models' generalization capabilities through the lens of performance on out-of-distribution (OOD) datasets. Despite their practicality, such…
Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers
Marek Kadlčík, Michal Štefánik, Timothee Mickus +2
Pretrained language models (LMs) are prone to arithmetic errors. Existing work showed limited success in probing numeric values from models' representations, indicating that these…
Negation: A Pink Elephant in the Large Language Models' Room?
Tereza Vrabcová, Marek Kadlčík, Petr Sojka +2
Negations are key to determining sentence meaning, making them essential for logical reasoning. Despite their importance, negations pose a substantial challenge for large language…
Self-training Language Models for Arithmetic Reasoning
Marek Kadlčík, Michal Štefánik
Recent language models achieve impressive results in tasks involving complex multistep reasoning, but scaling these capabilities further traditionally requires expensive collection…
Concept-aware Data Construction Improves In-context Learning of Language Models
Michal Štefánik, Marek Kadlčík, Petr Sojka
Many recent language models (LMs) are capable of in-context learning (ICL), manifested in the LMs' ability to perform a new task solely from natural-language instruction. Previous…
Concept-aware Training Improves In-context Learning Ability of Language Models
Michal Štefánik, Marek Kadlčík
Many recent language models (LMs) of Transformers family exhibit so-called in-context learning (ICL) ability, manifested in the LMs' ability to modulate their function by a task de…
Resources and Few-shot Learners for In-context Learning in Slavic Languages
Michal Štefánik, Marek Kadlčík, Piotr Gramacki +1
Despite the rapid recent progress in creating accurate and compact in-context learners, most recent work focuses on in-context learning (ICL) for tasks in English. However, the abi…
Can In-context Learners Learn a Reasoning Concept from Demonstrations?
Michal Štefánik, Marek Kadlčík
Language models exhibit an emergent ability to learn a new task from a small number of input-output demonstrations. However, recent work shows that in-context learners largely rely…
Soft Alignment Objectives for Robust Adaptation of Language Generation
Michal Štefánik, Marek Kadlčík, Petr Sojka
Domain adaptation allows generative language models to address specific flaws caused by the domain shift of their application. However, the traditional adaptation by further traini…