789 citations · 3.9k across the 31 of their papers we have counts for
34 papers
OpenAI GPT-5 System Card
Aaditya Singh, Adam Fry, Adam Perelman +483
This is the system card published alongside the OpenAI GPT-5 launch, August 2025. GPT-5 is a unified system with a smart and fast model that answers most questions, a deeper reason…
Balancing Knowledge Delivery and Emotional Comfort in Healthcare Conversational Systems
Shang-Chi Tsai, Yun-Nung Chen
With the advancement of large language models, many dialogue systems are now capable of providing reasonable and informative responses to patients' medical conditions. However, whe…
GPT-4o System Card
OpenAI, :, Aaron Hurst +416
GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's…
GPT-4 Technical Report
OpenAI, Josh Achiam, Steven Adler +278
We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-wor…
Generating Negative Samples for Sequential Recommendation
Yongjun Chen, Jia Li, Zhiwei Liu +4
To make Sequential Recommendation (SR) successful, recent works focus on designing effective sequential encoders, fusing side information, and mining extra positive self-supervisio…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Modeling Multi-hop Question Answering as Single Sequence Prediction
Semih Yavuz, Kazuma Hashimoto, Yingbo Zhou +2
Fusion-in-decoder (Fid) (Izacard and Grave, 2020) is a generative question answering (QA) model that leverages passage retrieval with a pre-trained transformer and pushed the state…
Combining Data-driven Supervision with Human-in-the-loop Feedback for Entity Resolution
Wenpeng Yin, Shelby Heinecke, Jia Li +7
The distribution gap between training datasets and data encountered in production is well acknowledged. Training datasets are often constructed over a fixed period of time and by c…
GeDi: Generative Discriminator Guided Sequence Generation
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann +4
While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of…
Unsupervised Paraphrasing with Pretrained Language Models
Tong Niu, Semih Yavuz, Yingbo Zhou +3
Paraphrase generation has benefited extensively from recent progress in the designing of training objectives and model architectures. However, previous explorations have largely fo…
Char2Subword: Extending the Subword Embedding Space Using Robust Character Compositionality
Gustavo Aguilar, Bryan McCann, Tong Niu +3
Byte-pair encoding (BPE) is a ubiquitous algorithm in the subword tokenization process of language models as it provides multiple benefits. However, this process is solely based on…
Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity
Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar +1
Neural text decoding is important for generating high-quality texts using language models. To generate high-quality text, popular decoding algorithms like top-k, top-p (nucleus), a…
Improving out-of-distribution generalization via multi-task self-supervised pretraining
Isabela Albuquerque, Nikhil Naik, Junnan Li +2
Self-supervised feature representations have been shown to be useful for supervised classification, few-shot learning, and adversarial robustness. We show that features obtained us…
ProGen: Language Modeling for Protein Generation
Ali Madani, Bryan McCann, Nikhil Naik +5
Generative modeling for protein engineering is key to solving fundamental problems in synthetic biology, medicine, and material science. We pose protein engineering as an unsupervi…
Global Capacity Measures for Deep ReLU Networks via Path Sampling
Ryan Theisen, Jason M. Klusowski, Huan Wang +3
Classical results on the statistical complexity of linear models have commonly identified the norm of the weights as a fundamental capacity measure. Generalizations of this…
CTRL: A Conditional Transformer Language Model for Controllable Generation
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney +2
Large-scale language models show promising text generation capabilities, but users cannot easily control particular aspects of the generated text. We release CTRL, a 1.63 billion-p…
Pretrained AI Models: Performativity, Mobility, and Change
Lav R. Varshney, Nitish Shirish Keskar, Richard Socher
The paradigm of pretrained deep learning models has recently emerged in artificial intelligence practice, allowing deployment in numerous societal settings with limited computation…
Neural Text Summarization: A Critical Evaluation
Wojciech Kryściński, Nitish Shirish Keskar, Bryan McCann +2
Text summarization aims at compressing long documents into a shorter form that conveys the most important parts of the original document. Despite increased interest in the communit…
XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering
Jasdeep Singh, Bryan McCann, Nitish Shirish Keskar +2
While natural language processing systems often focus on a single language, multilingual transfer learning has the potential to improve performance, especially for low-resource lan…
Unifying Question Answering, Text Classification, and Regression via Span Extraction
Nitish Shirish Keskar, Bryan McCann, Caiming Xiong +1
Even as pre-trained language encoders such as BERT are shared across many tasks, the output layers of question answering, text classification, and regression models are significant…
Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering
Victor Zhong, Caiming Xiong, Nitish Shirish Keskar +1
End-to-end neural models have made significant progress in question answering, however recent studies show that these models implicitly assume that the answer and evidence appear c…
A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation
Akhilesh Gotmare, Nitish Shirish Keskar, Caiming Xiong +1
The convergence rate and final performance of common deep learning models have significantly benefited from heuristics such as learning rate schedules, knowledge distillation, skip…
Identifying Generalization Properties in Neural Networks
Huan Wang, Nitish Shirish Keskar, Caiming Xiong +1
While it has not yet been proven, empirical evidence suggests that model generalization is related to local properties of the optima which can be described via the Hessian. We conn…
The Natural Language Decathlon: Multitask Learning as Question Answering
Bryan McCann, Nitish Shirish Keskar, Caiming Xiong +1
Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the…