1 citations · 1 across the 9 of their papers we have counts for
9 papers · 1 filter
Defragmenting Language Models: An Interpretability-based Approach for Vocabulary Expansion
Maitrey Mehta, Nishant Subramani, Zhichao Xu +2
All languages are equal; when it comes to tokenization, some are more equal than others. Tokens are the hidden currency that dictate the cost and latency of access to contemporary…
Distillation versus Contrastive Learning: How to Train Your Rerankers
Zhichao Xu, Zhiqi Huang, Shengyao Zhuang +1
Training effective text rerankers is crucial for information retrieval. Two strategies are widely used: contrastive learning (optimizing directly on ground-truth labels) and knowle…
Found in Translation: Measuring Multilingual LLM Consistency as Simple as Translate then Evaluate
Ashim Gupta, Maitrey Mehta, Zhichao Xu +1
Large language models (LLMs) provide detailed and impressive responses to queries in English. However, are they really consistent at responding to the same query in other languages…
Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation
Ashim Gupta, Vivek Srikumar
Inference-time scaling via repeated sampling has shown promise in reasoning tasks, but its effectiveness in multilingual generation remains underexplored. We evaluate this approach…
State Space Models are Strong Text Rerankers
Zhichao Xu, Jinghua Yan, Ashim Gupta +1
Transformers dominate NLP and IR; but their inference inefficiencies and challenges in extrapolating to longer contexts have sparked interest in alternative model architectures. Am…
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression
Zhichao Xu, Ashim Gupta, Tao Li +2
Increasingly, model compression techniques enable large language models (LLMs) to be deployed in real-world applications. As a result of this momentum towards local deployment, com…
An Empirical Investigation of Matrix Factorization Methods for Pre-trained Transformers
Ashim Gupta, Sina Mahdipour Saravani, P. Sadayappan +1
The increasing size of transformer-based models in NLP makes the question of compressing them important. In this work, we present a comprehensive analysis of factorization based mo…
Enhancing Question Answering on Charts Through Effective Pre-training Tasks
Ashim Gupta, Vivek Gupta, Shuo Zhang +3
To completely understand a document, the use of textual information is not enough. Understanding visual cues, such as layouts and charts, is also required. While the current state-…
Whispers of Doubt Amidst Echoes of Triumph in NLP Robustness
Ashim Gupta, Rishanth Rajendhran, Nathan Stringham +2
Do larger and more performant models resolve NLP's longstanding robustness issues? We investigate this question using over 20 models of different sizes spanning different architect…