3k citations · 3.8k across the 30 of their papers we have counts for
26 papers · 1 filter
PolyAlign: Conditional Human-Distribution Alignment
L. D. M. S. Sai Teja, Ufaq Khan, Sathira Silva +2
Post-training methods such as supervised fine-tuning (SFT) and preference optimization typically align language models toward a single global assistant behavior. While effective fo…
RTTC: Reward-Guided Collaborative Test-Time Compute
J. Pablo Muñoz, Jinjie Yuan
Test-Time Compute (TTC) has emerged as a powerful paradigm for enhancing the performance of Large Language Models (LLMs) at inference, leveraging strategies such as Test-Time Train…
SEA-LION: Southeast Asian Languages in One Network
Raymond Ng, Thanh Ngan Nguyen, Yuli Huang +28
Recently, Large Language Models (LLMs) have dominated much of the artificial intelligence scene with their ability to process and generate natural languages. However, the majority…
Considering Length Diversity in Retrieval-Augmented Summarization
Juseon-Do, Jaesung Hwang, Jingun Kwon +2
This study investigates retrieval-augmented summarization by specifically examining the impact of exemplar summary lengths under length constraints, not covered by previous work. W…
Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions
Angana Borah, Rada Mihalcea
As Large Language Models (LLMs) continue to evolve, they are increasingly being employed in numerous studies to simulate societies and execute diverse social tasks. However, LLMs a…
FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data
Haoran Sun, Renren Jin, Shaoyang Xu +10
Large language models (LLMs) have demonstrated prowess in a wide range of tasks. However, many LLMs exhibit significant performance discrepancies between high- and low-resource lan…
Naive Bayes-based Context Extension for Large Language Models
Jianlin Su, Murtadha Ahmed, Wenbo +3
Large Language Models (LLMs) have shown promising in-context learning abilities. However, conventional In-Context Learning (ICL) approaches are often impeded by length limitations…
Assessing biomedical knowledge robustness in large language models by query-efficient sampling attacks
R. Patrick Xian, Alex J. Lee, Satvik Lolla +4
The increasing depth of parametric domain knowledge in large language models (LLMs) is fueling their rapid deployment in real-world applications. Understanding model vulnerabilitie…
BLESS: Benchmarking Large Language Models on Sentence Simplification
Tannon Kew, Alison Chi, Laura Vásquez-Rodríguez +4
We present BLESS, a comprehensive performance benchmark of the most recent state-of-the-art large language models (LLMs) on the task of text simplification (TS). We examine how wel…
Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
Xiuying Wei, Yunchen Zhang, Yuhang Li +4
Post-training quantization~(PTQ) of transformer language models faces significant challenges due to the existence of detrimental outliers in activations. We observe that these outl…
GPT-4 Technical Report
OpenAI, Josh Achiam, Steven Adler +278
We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-wor…
Text and Code Embeddings by Contrastive Pre-Training
Arvind Neelakantan, Tao Xu, Raul Puri +22
Text embeddings are useful features in many applications such as semantic search and computing text similarity. Previous work typically trains models customized for different use c…
Unsupervised Neural Machine Translation with Generative Language Models Only
Jesse Michael Han, Igor Babuschkin, Harrison Edwards +8
We show how to derive state-of-the-art unsupervised neural machine translation systems from generatively pre-trained language models. Our method consists of three steps: few-shot a…
On Task-Level Dialogue Composition of Generative Transformer Model
Prasanna Parthasarathi, Arvind Neelakantan, Sharan Narang
Task-oriented dialogue systems help users accomplish tasks such as booking a movie ticket and ordering food via conversation. Generative models parameterized by a deep neural netwo…
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typicall…
Trading Off Diversity and Quality in Natural Language Generation
Hugh Zhang, Daniel Duckworth, Daphne Ippolito +1
For open-ended language generation tasks such as storytelling and dialogue, choosing the right decoding algorithm is critical to controlling the tradeoff between generation quality…
Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset
Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar +7
A significant barrier to progress in data-driven approaches to building dialog systems is the lack of high quality, goal-oriented conversational data. To help satisfy this elementa…
Parallel Scheduled Sampling
Daniel Duckworth, Arvind Neelakantan, Ben Goodrich +2
Auto-regressive models are widely used in sequence generation problems. The output sequence is typically generated in a predetermined order, one discrete unit (pixel or word or cha…
RelNet: End-to-End Modeling of Entities & Relations
Trapit Bansal, Arvind Neelakantan, Andrew McCallum
We introduce RelNet: a new model for relational reasoning. RelNet is a memory augmented neural network which models entities as abstract memory slots and is equipped with an additi…
Learning a Natural Language Interface with Neural Programmer
Arvind Neelakantan, Quoc V. Le, Martin Abadi +2
Learning a natural language interface for database tables is a challenging task that involves deep language understanding and multi-step reasoning. The task is often approached by…
Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks
Rajarshi Das, Arvind Neelakantan, David Belanger +1
Our goal is to combine the rich multistep inference of symbolic logical reasoning with the generalization capabilities of neural networks. We are particularly interested in complex…
Generalizing to Unseen Entities and Entity Pairs with Row-less Universal Schema
Patrick Verga, Arvind Neelakantan, Andrew McCallum
Universal schema predicts the types of entities and relations in a knowledge base (KB) by jointly embedding the union of all available schema types---not only types from multiple s…
Compositional Vector Space Models for Knowledge Base Completion
Arvind Neelakantan, Benjamin Roth, Andrew McCallum
Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by inferring with high likelihood nationality(X,Y) from bornIn(X,Y). Mos…
Inferring Missing Entity Type Instances for Knowledge Base Completion: New Dataset and Methods
Arvind Neelakantan, Ming-Wei Chang
Most of previous work in knowledge base (KB) completion has focused on the problem of relation extraction. In this work, we focus on the task of inferring missing entity type insta…