3 citations · 8 across the 11 of their papers we have counts for
12 papers
Stochastic Estimation of Transduced Language Models
Vésteinn Snæbjarnarson, Samuel Kiegeland, Manuel de Prada Corral +2
Transduced language models (TLMs) compose a pretrained \emph{source} language model with a functional finite-state transducer to induce a language model over \emph{target} strings.…
Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling
Jannek Ulm, Kevin Du, Vésteinn Snæbjarnarson
Large language models (LLMs) are trained on huge amounts of textual data, and concerns have been raised that the limits of such data may soon be reached. A potential solution is to…
Activation Scaling for Steering and Interpreting Language Models
Niklas Stoehr, Kevin Du, Vésteinn Snæbjarnarson +3
Given the prompt "Rome is in", can we steer a language model to flip its prediction of an incorrect token "France" to a correct token "Italy" by only multiplying a few relevant act…
LoQT: Low-Rank Adapters for Quantized Pretraining
Sebastian Loeschcke, Mads Toftrup, Michael J. Kastoryano +2
Despite advances using low-rank adapters and quantization, pretraining of large models on consumer hardware has not been possible without model sharding, offloading during training…
Context versus Prior Knowledge in Language Models
Kevin Du, Vésteinn Snæbjarnarson, Niklas Stoehr +3
To answer a question, language models often need to integrate prior knowledge learned during pretraining and new information presented in context. We hypothesize that models perfor…
Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora
Svanhvít Lilja Ingólfsdóttir, Pétur Orri Ragnarsson, Haukur Páll Jónsson +3
Grammatical error correction (GEC) is the task of correcting typos, spelling, punctuation and grammatical issues in text. Approaching the problem as a sequence-to-sequence task, we…
Discriminative Class Tokens for Text-to-Image Diffusion Models
Idan Schwartz, Vésteinn Snæbjarnarson, Hila Chefer +4
Recent advances in text-to-image diffusion models have enabled the generation of diverse and high-quality images. While impressive, the images often fall short of depicting subtle…
Assessing Neural Network Robustness via Adversarial Pivotal Tuning
Peter Ebert Christensen, Vésteinn Snæbjarnarson, Andrea Dittadi +2
The robustness of image classifiers is essential to their deployment in the real world. The ability to assess this resilience to manipulations or deviations from the training data…
Cross-Lingual QA as a Stepping Stone for Monolingual Open QA in Icelandic
Vésteinn Snæbjarnarson, Hafsteinn Einarsson
It can be challenging to build effective open question answering (open QA) systems for languages other than English, mainly due to a lack of labeled data for training. We present a…
A Warm Start and a Clean Crawled Corpus -- A Recipe for Good Language Models
Vésteinn Snæbjarnarson, Haukur Barri Símonarson, Pétur Orri Ragnarsson +4
We train several language models for Icelandic, including IceBERT, that achieve state-of-the-art performance in a variety of downstream tasks, including part-of-speech tagging, nam…
Miðeind's WMT 2021 submission
Haukur Barri Símonarson, Vésteinn Snæbjarnarson, Pétur Orri Ragnarsson +2
We present Miðeind's submission for the EnglishIcelandic and IcelandicEnglish subsets of the 2021 WMT news translation task. Transformer-base models are trained for trans…
Icelandic Parallel Abstracts Corpus
Haukur Barri Símonarson, Vésteinn Snæbjarnarson
We present a new Icelandic-English parallel corpus, the Icelandic Parallel Abstracts Corpus (IPAC), composed of abstracts from student theses and dissertations. The texts were coll…