4 papers
Gumbel Machine: Counterfactual Student Writing Generation via Gumbel Noise Steering
Hunter McNichols, Alexander Scarlatos, Mihai Dascalu +2
An effective method of teaching across disciplines is to provide examples of high-quality work. However, an example may be significantly different from a student's current work, ma…
Value-Aware Numerical Representations for Transformer Language Models
Andreea Dutulescu, Stefan Ruseti, Mihai Dascalu
Transformer-based language models often achieve strong results on mathematical reasoning benchmarks while remaining fragile on basic numerical understanding and arithmetic operatio…
Training Language Models with homotokens Leads to Delayed Overfitting
Adrian Cosma, Stefan Ruseti, Emilian Radoi +1
Subword tokenization introduces a computational layer in language models where many distinct token sequences decode to the same surface form and preserve meaning, yet induce differ…
The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models
Adrian Cosma, Stefan Ruseti, Emilian Radoi +1
Despite their remarkable progress across diverse domains, Large Language Models (LLMs) consistently fail at simple character-level tasks, such as counting letters in words, due to…