6 papers
From Data to Device: ELMOD An Efficient German-First 2.7B Language Model for Mobile Inference
Darina Gold, Alexander Schwirjow, Viktor Haag +4
We present ELMOD - Efficient Language Model for On-Device Deployment - a compact (2.7B) German language model designed for efficient inference on resource-constrained hardware. ELM…
CEDAR: Context Engineering for Agentic Data Science
Rishiraj Saha Roy, Chris Hinze, Luzian Hahn +1
We demonstrate CEDAR, an application for automating data science (DS) tasks with an agentic setup. Solving DS problems with LLMs is an underexplored area that has immense market va…
Data Processing for the OpenGPT-X Model Family
Nicolo' Brandizzi, Hammam Abdelwahab, Anirban Bhowmick +19
This paper presents a comprehensive overview of the data preparation pipeline developed for the OpenGPT-X project, a large-scale initiative aimed at creating open and high-performa…
Pre-Training LLMs on a budget: A comparison of three optimizers
Joel Schlotthauer, Christian Kroos, Chris Hinze +3
Optimizers play a decisive role in reducing pre-training times for LLMs and achieving better-performing models. In this study, we compare three major variants: the de-facto standar…
RAGONITE: Iterative Retrieval on Induced Databases and Verbalized RDF for Conversational QA over KGs with RAG
Rishiraj Saha Roy, Chris Hinze, Joel Schlotthauer +5
Conversational question answering (ConvQA) is a convenient means of searching over RDF knowledge graphs (KGs), where a prevalent approach is to translate natural language questions…
Evidence Contextualization and Counterfactual Attribution for Conversational QA over Heterogeneous Data with RAG Systems
Rishiraj Saha Roy, Joel Schlotthauer, Chris Hinze +3
Retrieval Augmented Generation (RAG) works as a backbone for interacting with an enterprise's own data via Conversational Question Answering (ConvQA). In a RAG system, a retriever…