Publications (16)
From Feedback to Checklists: Grounded Evaluation of AI-Generated Clinical Notes
Karen Zhou, John Giorgi, Pranav Mani +3
AI-generated clinical notes are increasingly used in healthcare, but evaluating their quality remains a challenge due to high subjectivity and limited scalability of expert review.…
Co-training and Co-distillation for Quality Improvement and Compression of Language Models
Hayeon Lee, Rui Hou, Jongpil Kim +4
Knowledge Distillation (KD) compresses computationally expensive pre-trained language models (PLMs) by transferring their knowledge to smaller models, allowing their use in resourc…
TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding
Zhiheng Huang, Peng Xu, Davis Liang +2
Bidirectional Encoder Representations from Transformers (BERT) has recently achieved state-of-the-art performance on a broad range of NLP tasks including sentence classification, m…
Embedding-based Zero-shot Retrieval through Query Generation
Davis Liang, Peng Xu, Siamak Shakeri +4
Passage retrieval addresses the problem of locating relevant passages, usually from a large corpus, given a query. In practice, lexical term-matching algorithms like BM25 are popul…
Improve Transformer Models with Better Relative Position Embeddings
Zhiheng Huang, Davis Liang, Peng Xu +1
Transformer architectures rely on explicit position encodings in order to preserve a notion of word order. In this paper, we argue that existing work does not fully utilize positio…
The Curious Language Model: Strategic Test-Time Information Acquisition
Michael Cooper, Rohan Wadhawan, John Michael Giorgi +2
Decision-makers often possess insufficient information to render a confident decision. In these cases, the decision-maker can often undertake actions to acquire the necessary infor…
Multiplicative Position-aware Transformer Models for Language Understanding
Zhiheng Huang, Davis Liang, Peng Xu +1
Transformer models, which leverage architectural improvements like self-attention, perform remarkably well on Natural Language Processing (NLP) tasks. The self-attention mechanism…
Masked Language Model Scoring
Julian Salazar, Davis Liang, Toan Q. Nguyen +1
Pretrained masked language models (MLMs) require finetuning for most NLP tasks. Instead, we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are co…
The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants
Lucas Bandarkar, Davis Liang, Benjamin Muller +7
We present Belebele, a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. Significantly expanding the language coverage of natural language…
Decoding and Diversity in Machine Translation
Nicholas Roberts, Davis Liang, Graham Neubig +1
Neural Machine Translation (NMT) systems are typically evaluated using automated metrics that assess the agreement between generated translations and ground truth candidates. To im…
A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models
Hayeon Lee, Rui Hou, Jongpil Kim +3
Distillation from Weak Teacher (DWT) is a method of transferring knowledge from a smaller, weaker teacher model to a larger student model to improve its performance. Previous studi…
Deep Automated Multi-task Learning
Davis Liang, Yan Shu
Multi-task learning (MTL) has recently contributed to learning better representations in service of various NLP tasks. MTL aims at improving the performance of a primary task, by j…
Learning Noise-Invariant Representations for Robust Speech Recognition
Davis Liang, Zhiheng Huang, Zachary C. Lipton
Despite rapid advances in speech recognition, current models remain brittle to superficial perturbations to their inputs. Small amounts of noise can destroy the performance of an o…
RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Training
Jaehyung Kim, Yuning Mao, Rui Hou +7
Fine-tuning pre-trained language models (LMs) has become the de facto standard in many NLP tasks. Nevertheless, fine-tuned LMs are still prone to robustness issues, such as adversa…
XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models
Davis Liang, Hila Gonen, Yuning Mao +5
Large multilingual language models typically rely on a single vocabulary shared across 100+ languages. As these models have increased in parameter count and depth, vocabulary size…
Attention-guided Generative Models for Extractive Question Answering
Peng Xu, Davis Liang, Zhiheng Huang +1
We propose a novel method for applying Transformer models to extractive question answering (QA) tasks. Recently, pretrained generative sequence-to-sequence (seq2seq) models have ac…