7 papers · 1 filter
Measuring and Guiding Monosemanticity
Ruben Härle, Felix Friedrich, Manuel Brack +4
There is growing interest in leveraging mechanistic interpretability and controllability to better understand and influence the internal dynamics of large language models (LLMs). H…
CHRONOBERG: Capturing Language Evolution and Temporal Awareness in Foundation Models
Niharika Hegde, Subarnaduti Paul, Lars Joel-Frey +4
Large language models (LLMs) excel at operating at scale by leveraging social media and various data crawled from the web. Whereas existing corpora are diverse, their frequent lack…
Beyond Overcorrection: Evaluating Diversity in T2I Models with DivBench
Felix Friedrich, Thiemo Ganesha Welsch, Manuel Brack +2
Current diversification strategies for text-to-image (T2I) models often ignore contextual appropriateness, leading to over-diversification where demographic attributes are modified…
LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Inconsistencies
Felix Friedrich, Simone Tedeschi, Patrick Schramowski +5
Building safe Large Language Models (LLMs) across multiple languages is essential in ensuring both safe access and linguistic diversity. To this end, we conduct a large-scale, comp…
Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
Mehdi Ali, Manuel Brack, Max Lübbering +15
High-quality multilingual training data is essential for effectively pretraining large language models (LLMs). Yet, the availability of suitable open-source multilingual datasets r…
T-FREE: Subword Tokenizer-Free Generative LLMs via Sparse Representations for Memory-Efficient Embeddings
Björn Deiseroth, Manuel Brack, Patrick Schramowski +2
Tokenizers are crucial for encoding information in Large Language Models, but their development has recently stagnated, and they contain inherent weaknesses. Major limitations incl…