1 citations · 1 across the 8 of their papers we have counts for
8 papers
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
A Systematic Survey of Agentic Skills: Architecture, Lifecycle, and Security
Sanket Badhe, Deep Shah, Priyanka Tiwari +1
Autonomous large language model (LLM) agents increasingly face reliability, context consumption, and execution stability bottlenecks when deployed on complex, long-horizon tasks. W…
MVEB: Massive Video Embedding Benchmark
Adnan El Assadi, Roman Solomatin, Isaac Chung +13
We introduce the Massive Video Embedding Benchmark (MVEB), a 23-task benchmark for video embeddings spanning classification, zero-shot classification, clustering, pair classificati…
The Silent Vote: Improving Zero-Shot LLM Reliability by Aggregating Semantic Neighborhoods
Sanket Badhe, Priyanka Tiwari, Deep Shah
Large Language Models are increasingly used as zero-shot classifiers in complex reasoning tasks. However, standard constrained decoding suffers from a phenomenon we define as Renor…
CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization
Deep Shah, Sanket Badhe, Nehal Kathrotia +1
Large Language Models utilizing reasoning techniques improve task performance but incur significant latency and token costs due to verbose generation. Existing automatic prompt opt…
Long-Tail Knowledge in Large Language Models: Taxonomy, Mechanisms, Interventions and Implications
Sanket Badhe, Deep Shah, Nehal Kathrotia
Large language models (LLMs) are trained on web-scale corpora that exhibit steep power-law distributions, in which the distribution of knowledge is highly long-tailed, with most ap…
Taxonomy of the Retrieval System Framework: Pitfalls and Paradigms
Deep Shah, Sanket Badhe, Nehal Kathrotia
Designing an embedding retrieval system requires navigating a complex design space of conflicting trade-offs between efficiency and effectiveness. This work structures these decisi…
Collective Communication for 100k+ GPUs
Min Si, Pavan Balaji, Yongzhou Chen +36
The increasing scale of large language models (LLMs) necessitates highly efficient collective communication frameworks, particularly as training workloads extend to hundreds of tho…