91 citations · 101 across the 10 of their papers we have counts for
10 papers
Does quantization affect models' performance on long-context tasks?
Anmol Mekala, Anirudh Atmakuru, Yixiao Song +2
Large language models (LLMs) now support context windows exceeding 128K tokens, but this comes with significant memory requirements and high inference latency. Quantization can mit…
BEARCUBS: A benchmark for computer-using web agents
Yixiao Song, Katherine Thai, Chau Minh Pham +3
Modern web agents possess computer use abilities that allow them to interact with webpages by sending commands to a virtual keyboard and mouse. While such agents have considerable…
Localizing and Mitigating Errors in Long-form Question Answering
Rachneet Sachdeva, Yixiao Song, Mohit Iyyer +1
Long-form question answering (LFQA) aims to provide thorough and in-depth answers to complex questions, enhancing comprehension. However, such detailed responses are prone to hallu…
VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation
Yixiao Song, Yekyung Kim, Mohit Iyyer
Existing metrics for evaluating the factuality of long-form text, such as FACTSCORE (Min et al., 2023) and SAFE (Wei et al., 2024), decompose an input text into "atomic claims" and…
GEE! Grammar Error Explanation with Large Language Models
Yixiao Song, Kalpesh Krishna, Rajesh Bhatt +2
Grammatical error correction tools are effective at correcting grammatical errors in users' input sentences but do not provide users with \textit{natural language} explanations abo…
A Critical Evaluation of Evaluations for Long-form Question Answering
Fangyuan Xu, Yixiao Song, Mohit Iyyer +1
Long-form question answering (LFQA) enables answering a wide range of questions, but its flexibility poses enormous challenges for evaluation. We perform the first targeted study o…
KNN-LM Does Not Improve Open-ended Text Generation
Shufan Wang, Yixiao Song, Andrew Drozdov +3
In this paper, we study the generation quality of interpolation-based retrieval-augmented language models (LMs). These methods, best exemplified by the KNN-LM, interpolate the LM's…
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Kalpesh Krishna, Yixiao Song, Marzena Karpinska +2
The rise in malicious usage of large language models, such as fake content creation and academic plagiarism, has motivated the development of approaches that identify AI-generated…
DEMETR: Diagnosing Evaluation Metrics for Translation
Marzena Karpinska, Nishant Raj, Katherine Thai +3
While machine translation evaluation metrics based on string overlap (e.g., BLEU) have their limitations, their computations are transparent: the BLEU score assigned to a particula…
SLING: Sino Linguistic Evaluation of Large Language Models
Yixiao Song, Kalpesh Krishna, Rajesh Bhatt +1
To understand what kinds of linguistic knowledge are encoded by pretrained Chinese language models (LMs), we introduce the benchmark of Sino LINGuistics (SLING), which consists of…