Publications (16)
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions
Siru Ouyang, Shuohang Wang, Yang Liu +7
Recent progress in Large Language Models (LLMs) has produced models that exhibit remarkable performance across a variety of NLP tasks. However, it remains unclear whether the exist…
In-Context Demonstration Selection with Cross Entropy Difference
Dan Iter, Reid Pryzant, Ruochen Xu +4
Large language models (LLMs) can use in-context demonstrations to improve performance on zero-shot tasks. However, selecting the best in-context examples is challenging because mod…
The Trade-offs of Domain Adaptation for Neural Language Models
David Grangier, Dan Iter
This work connects language model adaptation with concepts of machine learning theory. We consider a training setup with a large out-of-domain set and a small in-domain set. We der…
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models
Dan Iter, Kelvin Guu, Larry Lansing +1
Recent models for unsupervised representation learning of text have employed a number of techniques to improve contextual word representations but have put little focus on discours…
Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference
William Held, Dan Iter, Dan Jurafsky
Performing event and entity coreference resolution across documents vastly increases the number of candidate mentions, making it intractable to do the full pairwise compariso…
Generate rather than Retrieve: Large Language Models are Strong Context Generators
Wenhao Yu, Dan Iter, Shuohang Wang +6
Knowledge-intensive tasks, such as open-domain question answering (QA), require access to a large amount of world or domain knowledge. A common approach for knowledge-intensive tas…
LMGQS: A Large-scale Dataset for Query-focused Summarization
Ruochen Xu, Song Wang, Yang Liu +5
Query-focused summarization (QFS) aims to extract or generate a summary of an input document that directly answers or is relevant to a given query. The lack of large-scale datasets…
Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models
Zhihan Zhang, Shuohang Wang, Wenhao Yu +6
Large language models (LLMs) can perform a wide range of tasks by following natural language instructions, without the necessity of task-specific fine-tuning. Unfortunately, the pe…
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
Yang Liu, Dan Iter, Yichong Xu +3
The quality of texts generated by natural language generation (NLG) systems is hard to measure automatically. Conventional reference-based metrics, such as BLEU and ROUGE, have bee…
How Does In-Context Learning Help Prompt Tuning?
Simeng Sun, Yang Liu, Dan Iter +2
Fine-tuning large language models is becoming ever more impractical due to their rapidly-growing scale. This motivates the use of parameter-efficient adaptation methods such as pro…
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Marah Abdin, Jyoti Aneja, Hany Awadalla +126
We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal test…
On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation
Dan Iter, David Grangier
Domain adaptation of neural networks commonly relies on three training phases: pretraining, selected data training and then fine tuning. Data selection improves target domain gener…
Socratic Learning: Augmenting Generative Models to Incorporate Latent Subsets in Training Data
Paroma Varma, Bryan He, Dan Iter +4
A challenge in training discriminative models like neural networks is obtaining enough labeled training data. Recent approaches use generative models to combine weak supervision so…
Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs
Stefan Hadjis, Ce Zhang, Ioannis Mitliagkas +2
We study the factors affecting training time in multi-device deep learning systems. Given a specification of a convolutional neural network, our goal is to minimize the time to tra…
Automatic Prompt Optimization with "Gradient Descent" and Beam Search
Reid Pryzant, Dan Iter, Jerry Li +3
Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous…
InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
Yichong Xu, Ruochen Xu, Dan Iter +4
While large models such as GPT-3 demonstrate exceptional performance in zeroshot and fewshot summarization tasks, their extensive serving and fine-tuning costs hinder their utiliza…