263 citations · 694 across the 35 of their papers we have counts for
36 papers
How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench
Qinyuan Ye, Harvey Yiyun Fu, Xiang Ren +1
We investigate the predictability of large language model (LLM) capabilities: given records of past experiments using different model families, numbers of parameters, tasks, and nu…
Chain-of-Questions Training with Latent Answers for Robust Multistep Question Answering
Wang Zhu, Jesse Thomason, Robin Jia
We train a language model (LM) to robustly answer multistep questions by generating and answering sub-questions. We propose Chain-of-Questions, a framework that trains a model to g…
Estimating Large Language Model Capabilities without Labeled Test Data
Harvey Yiyun Fu, Qinyuan Ye, Albert Xu +2
Large Language Models (LLMs) have the impressive ability to perform in-context learning (ICL) from only a few examples, but the success of ICL varies widely from task to task. Thus…
Operationalizing content moderation "accuracy" in the Digital Services Act
Johnny Tian-Zheng Wei, Frederike Zufall, Robin Jia
The Digital Services Act, recently adopted by the EU, requires social media platforms to report the "accuracy" of their automated content moderation systems. The colloquial term is…
SCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examples
Deqing Fu, Ameya Godbole, Robin Jia
Detecting negatives (such as non-entailment relationships, unanswerable questions, and false claims) is an important and challenging aspect of many natural language understanding t…
Data Curation Alone Can Stabilize In-context Learning
Ting-Yun Chang, Robin Jia
In-context learning (ICL) enables large language models (LLMs) to perform new tasks by prompting them with a sequence of training examples. However, it is known that ICL is very se…