3 citations · 8 across the 11 of their papers we have counts for
9 papers · 1 filter
Stochastic Estimation of Transduced Language Models
Vésteinn Snæbjarnarson, Samuel Kiegeland, Manuel de Prada Corral +2
Transduced language models (TLMs) compose a pretrained \emph{source} language model with a functional finite-state transducer to induce a language model over \emph{target} strings.…
Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling
Jannek Ulm, Kevin Du, Vésteinn Snæbjarnarson
Large language models (LLMs) are trained on huge amounts of textual data, and concerns have been raised that the limits of such data may soon be reached. A potential solution is to…
Activation Scaling for Steering and Interpreting Language Models
Niklas Stoehr, Kevin Du, Vésteinn Snæbjarnarson +3
Given the prompt "Rome is in", can we steer a language model to flip its prediction of an incorrect token "France" to a correct token "Italy" by only multiplying a few relevant act…
Context versus Prior Knowledge in Language Models
Kevin Du, Vésteinn Snæbjarnarson, Niklas Stoehr +3
To answer a question, language models often need to integrate prior knowledge learned during pretraining and new information presented in context. We hypothesize that models perfor…
Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora
Svanhvít Lilja Ingólfsdóttir, Pétur Orri Ragnarsson, Haukur Páll Jónsson +3
Grammatical error correction (GEC) is the task of correcting typos, spelling, punctuation and grammatical issues in text. Approaching the problem as a sequence-to-sequence task, we…
Cross-Lingual QA as a Stepping Stone for Monolingual Open QA in Icelandic
Vésteinn Snæbjarnarson, Hafsteinn Einarsson
It can be challenging to build effective open question answering (open QA) systems for languages other than English, mainly due to a lack of labeled data for training. We present a…
A Warm Start and a Clean Crawled Corpus -- A Recipe for Good Language Models
Vésteinn Snæbjarnarson, Haukur Barri Símonarson, Pétur Orri Ragnarsson +4
We train several language models for Icelandic, including IceBERT, that achieve state-of-the-art performance in a variety of downstream tasks, including part-of-speech tagging, nam…
Miðeind's WMT 2021 submission
Haukur Barri Símonarson, Vésteinn Snæbjarnarson, Pétur Orri Ragnarsson +2
We present Miðeind's submission for the EnglishIcelandic and IcelandicEnglish subsets of the 2021 WMT news translation task. Transformer-base models are trained for trans…
Icelandic Parallel Abstracts Corpus
Haukur Barri Símonarson, Vésteinn Snæbjarnarson
We present a new Icelandic-English parallel corpus, the Icelandic Parallel Abstracts Corpus (IPAC), composed of abstracts from student theses and dissertations. The texts were coll…