63 citations · 74 across the 7 of their papers we have counts for
7 papers
What do Language Models Learn and When? The Implicit Curriculum Hypothesis
Emmy Liu, Kaiser Sun, Millicent Li +4
Large language models (LLMs) can perform remarkably complex tasks, yet the fine-grained details of how these capabilities emerge during pretraining remain poorly understood. Scalin…
Do Activation Verbalization Methods Convey Privileged Information?
Millicent Li, Alberto Mario Ceballos Arroyo, Giordano Rogers +2
Recent interpretability methods have proposed to translate LLM internal representations into natural language descriptions using a second verbalizer LLM. This is intended to illumi…
Multi-Field Adaptive Retrieval
Millicent Li, Tongfei Chen, Benjamin Van Durme +1
Document retrieval for tasks such as search and retrieval-augmented generation typically involves datasets that are unstructured: free-form text without explicit internal structure…
The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysis
Aaron Mueller, Jannik Brinkmann, Millicent Li +10
Interpretability provides a toolset for understanding how and why neural networks behave in certain ways. However, there is little unity in the field: most studies employ ad-hoc ev…
Function Vectors in Large Language Models
Eric Todd, Millicent L. Li, Arnab Sen Sharma +3
We report the presence of a simple neural mechanism that represents an input-output function as a vector within autoregressive transformer language models (LMs). Using causal media…
Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)
Chantal Shaib, Millicent L. Li, Sebastian Joseph +3
Large language models, particularly GPT-3, are able to produce high quality summaries of general domain news articles in few- and zero-shot settings. However, it is unclear if such…
A Review on Language Models as Knowledge Bases
Badr AlKhamissi, Millicent Li, Asli Celikyilmaz +2
Recently, there has been a surge of interest in the NLP community on the use of pretrained Language Models (LMs) as Knowledge Bases (KBs). Researchers have shown that LMs trained o…