7 citations · 12 across the 9 of their papers we have counts for
7 papers · 1 filter
Byte BPE Tokenization as an Inverse string Homomorphism
Saibo Geng, Sankalp Gambhir, Chris Wendler +1
Tokenization is an important preprocessing step in the training and inference of large language models (LLMs). While there has been extensive research on the expressive power of th…
Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers
Clément Dumas, Chris Wendler, Veniamin Veselovsky +2
A central question in multilingual language modeling is whether large language models (LLMs) develop a universal concept representation, disentangled from specific languages. In th…
Agentic AI: The Era of Semantic Decoding
Maxime Peyrard, Martin Josifoski, Robert West
Recent work demonstrated great promise in the idea of orchestrating collaborations between LLMs, human input, and various tools to address the inherent limitations of LLMs. We prop…
Symbolic Autoencoding for Self-Supervised Sequence Learning
Mohammad Hossein Amani, Nicolas Mario Baldwin, Amin Mansouri +3
Traditional language models, adept at next-token prediction in text sequences, often struggle with transduction tasks between distinct symbolic systems, particularly when parallel…
Do Llamas Work in English? On the Latent Language of Multilingual Transformers
Chris Wendler, Veniamin Veselovsky, Giovanni Monea +1
We ask whether multilingual language models trained on unbalanced, English-dominated corpora use English as an internal pivot language -- a question of key importance for understan…
Sketch-Guided Constrained Decoding for Boosting Blackbox Large Language Models without Logit Access
Saibo Geng, Berkay Döner, Chris Wendler +2
Constrained decoding, a technique for enforcing constraints on language model outputs, offers a way to control text generation without retraining or architectural modifications. It…
Evaluating Language Model Agency through Negotiations
Tim R. Davidson, Veniamin Veselovsky, Martin Josifoski +4
We introduce an approach to evaluate language model (LM) agency using negotiation games. This approach better reflects real-world use cases and addresses some of the shortcomings o…