2 citations · 2 across the 9 of their papers we have counts for
6 papers · 1 filter
Vocabulary Customization for Efficient Domain-Specific LLM Deployment
Christian Herold, Michael Kozielski, Nicholas Santavas +2
When using an LLM to process text outside the training domain(s), an often overlooked factor is vocabulary mismatch, where the general-domain tokenizer fails to capture frequent do…
Lost at the Beginning of Reasoning
Baohao Liao, Xinyi Chen, Sara Rajaee +5
Recent advancements in large language models (LLMs) have significantly advanced complex reasoning capabilities, particularly through extended chain-of-thought (CoT) reasoning that…
Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation
Stefan Vasilev, Christian Herold, Baohao Liao +3
This paper introduces Unilogit, a novel self-distillation method for machine unlearning in Large Language Models. Unilogit addresses the challenge of selectively forgetting specifi…
CONGRAD:Conflicting Gradient Filtering for Multilingual Preference Alignment
Jiangnan Li, Thuy-Trang Vu, Christian Herold +3
Naive joint training of large language models (LLMs) for multilingual preference alignment can suffer from negative interference. This is a known issue in multilingual training, wh…
ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning
Baohao Liao, Christian Herold, Seyyed Hadi Hashemi +3
As large language models (LLMs) scale, model compression is crucial for edge deployment and accessibility. Weight-only quantization reduces model size but suffers from performance…
Domain Adaptation of Foundation LLMs for e-Commerce
Christian Herold, Michael Kozielski, Tala Bazazo +6
We present the e-Llama models: 8 billion and 70 billion parameter large language models that are adapted towards the e-commerce domain. These models are meant as foundation models…