34 citations · 67 across the 12 of their papers we have counts for
12 papers
Entity Linking using LLMs for Automated Product Carbon Footprint Estimation
Steffen Castle, Julian Moreno Schneider, Leonhard Hennig +1
Growing concerns about climate change and sustainability are driving manufacturers to take significant steps toward reducing their carbon footprints. For these manufacturers, a fir…
Reward Modeling with Weak Supervision for Language Models
Ben Hauptvogel, Malte Ostendorff, Georg Rehm +1
Recent advancements in large language models (LLMs) have led to their increased application across various tasks, with reinforcement learning from human feedback (RLHF) being a cru…
Data Processing for the OpenGPT-X Model Family
Nicolo' Brandizzi, Hammam Abdelwahab, Anirban Bhowmick +19
This paper presents a comprehensive overview of the data preparation pipeline developed for the OpenGPT-X project, a large-scale initiative aimed at creating open and high-performa…
Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs
Mehdi Ali, Michael Fromm, Klaudia Thellmann +38
We present two multilingual LLMs, Teuken 7B-base and Teuken 7B-instruct, designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European U…
Symmetric Dot-Product Attention for Efficient Training of BERT Language Models
Martin Courtois, Malte Ostendorff, Leonhard Hennig +1
Initially introduced as a machine translation model, the Transformer architecture has now become the foundation for modern deep learning architecture, with applications in a wide r…
Investigating Gender Bias in Turkish Language Models
Orhun Mersin Caglidil, Malte Ostendorff, Georg Rehm
Language models are trained mostly on Web data, which often contains social stereotypes and biases that the models can inherit. This has potentially negative consequences, as model…
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning
Malte Ostendorff, Georg Rehm
Most Transformer language models are primarily pretrained on English text, limiting their use for other languages. As the model sizes grow, the performance gap between English and…
User Experience Design for Automatic Credibility Assessment of News Content About COVID-19
Konstantin Schulz, Jens Rauenbusch, Jan Fillies +3
The increasingly rapid spread of information about COVID-19 on the web calls for automatic measures of quality assurance. In that context, we check the credibility of news content…
Specialized Document Embeddings for Aspect-based Similarity of Research Papers
Malte Ostendorff, Till Blume, Terry Ruas +2
Document embeddings and similarity measures underpin content-based recommender systems, whereby a document is commonly represented as a single generic embedding. However, similarit…
HiStruct+: Improving Extractive Text Summarization with Hierarchical Structure Information
Qian Ruan, Malte Ostendorff, Georg Rehm
Transformer-based language models usually treat texts as linear sequences. However, most texts also have an inherent hierarchical structure, i.e., parts of a text can be identified…
Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings
Malte Ostendorff, Nils Rethmeier, Isabelle Augenstein +2
Learning scientific document representations can be substantially improved through contrastive learning objectives, where the challenge lies in creating positive and negative train…
Evaluating Document Representations for Content-based Legal Literature Recommendations
Malte Ostendorff, Elliott Ash, Terry Ruas +3
Recommender systems assist legal professionals in finding relevant literature for supporting their case. Despite its importance for the profession, legal applications do not reflec…