6 papers
Defragmenting Language Models: An Interpretability-based Approach for Vocabulary Expansion
Maitrey Mehta, Nishant Subramani, Zhichao Xu +2
All languages are equal; when it comes to tokenization, some are more equal than others. Tokens are the hidden currency that dictate the cost and latency of access to contemporary…
Distillation versus Contrastive Learning: How to Train Your Rerankers
Zhichao Xu, Zhiqi Huang, Shengyao Zhuang +1
Training effective text rerankers is crucial for information retrieval. Two strategies are widely used: contrastive learning (optimizing directly on ground-truth labels) and knowle…
Unequal Voices: How LLMs Construct Constrained Queer Narratives
Atreya Ghosal, Ashim Gupta, Vivek Srikumar
One way social groups are marginalized in discourse is that the narratives told about them often default to a narrow, stereotyped range of topics. In contrast, default groups are a…
Found in Translation: Measuring Multilingual LLM Consistency as Simple as Translate then Evaluate
Ashim Gupta, Maitrey Mehta, Zhichao Xu +1
Large language models (LLMs) provide detailed and impressive responses to queries in English. However, are they really consistent at responding to the same query in other languages…
Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation
Ashim Gupta, Vivek Srikumar
Inference-time scaling via repeated sampling has shown promise in reasoning tasks, but its effectiveness in multilingual generation remains underexplored. We evaluate this approach…
State Space Models are Strong Text Rerankers
Zhichao Xu, Jinghua Yan, Ashim Gupta +1
Transformers dominate NLP and IR; but their inference inefficiencies and challenges in extrapolating to longer contexts have sparked interest in alternative model architectures. Am…