129 citations · 275 across the 21 of their papers we have counts for
21 papers
Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM
Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal +2
Contrastive decoding (CD) (Li et al., 2023) improves the next-token distribution of a large expert language model (LM) using a small amateur LM. Although CD is applied to various L…
CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints
Anirudh Atmakuru, Jatin Nainani, Rohith Siddhartha Reddy Bheemreddy +4
Evaluating the creativity of large language models (LLMs) in story writing is difficult because LLM-generated stories could seemingly look creative but be very similar to some exis…
LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints
Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin +7
Instruction following is a key capability for LLMs. However, recent studies have shown that LLMs often struggle with instructions containing multiple constraints (e.g. a request to…
REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy
Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal +2
Decoding methods for large language models (LLMs) usually struggle with the tradeoff between ensuring factuality and maintaining diversity. For example, a higher p threshold in the…
To Copy, or not to Copy; That is a Critical Issue of the Output Softmax Layer in Neural Sequential Recommenders
Haw-Shiuan Chang, Nikhil Agarwal, Andrew McCallum
Recent studies suggest that the existing neural models have difficulty handling repeated items in sequential recommendation tasks. However, our understanding of this difficulty is…
Encoding Multi-Domain Scientific Papers by Ensembling Multiple CLS Tokens
Ronald Seoh, Haw-Shiuan Chang, Andrew McCallum
Many useful tasks on scientific documents, such as topic classification and citation prediction, involve corpora that span multiple scientific domains. Typically, such tasks are ac…
Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyond
Haw-Shiuan Chang, Zonghai Yao, Alolika Gon +2
Is the output softmax layer, which is adopted by most language models (LMs), always the best way to compute the next word probability? Given so many attention layers in a modern tr…
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling
Haw-Shiuan Chang, Ruei-Yao Sun, Kathryn Ricci +1
Ensembling BERT models often significantly improves accuracy, but at the cost of significantly more computation and memory footprint. In this work, we propose Multi-CLS BERT, a nov…
Augmenting Scientific Creativity with Retrieval across Knowledge Domains
Hyeonsu B. Kang, Sheshera Mysore, Kevin Huang +5
Exposure to ideas in domains outside a scientist's own may benefit her in reformulating existing research problems in novel ways and discovering new application domains for existin…
Open Aspect Target Sentiment Classification with Natural Language Prompts
Ronald Seoh, Ian Birle, Mrinal Tak +3
For many business applications, we often seek to analyze sentiments associated with any arbitrary aspects of commercial products, despite having a very limited amount of labels or…
Multi-facet Universal Schema
Rohan Paul, Haw-Shiuan Chang, Andrew McCallum
Universal schema (USchema) assumes that two sentence patterns that share the same entity pairs are similar to each other. This assumption is widely adopted for solving various type…
Changing the Mind of Transformers for Topically-Controllable Language Generation
Haw-Shiuan Chang, Jiaming Yuan, Mohit Iyyer +1
Large Transformer-based language models can aid human authors by suggesting plausible continuations of text written so far. However, current interactive writing assistants do not a…
Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications
Haw-Shiuan Chang, Amol Agrawal, Andrew McCallum
Most unsupervised NLP models represent each word with a single point or single region in semantic space, while the existing multi-sense word embeddings cannot represent longer word…
AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Types
Xin Luna Dong, Xiang He, Andrey Kan +19
Can one build a knowledge graph (KG) for all products in the world? Knowledge graphs have firmly established themselves as valuable sources of information for search and question a…
Using Error Decay Prediction to Overcome Practical Issues of Deep Active Learning for Named Entity Recognition
Haw-Shiuan Chang, Shankar Vembu, Sunil Mohan +2
Existing deep active learning algorithms achieve impressive sampling efficiency on natural language processing tasks. However, they exhibit several weaknesses in practice, includin…
The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structures
Sheshera Mysore, Zach Jensen, Edward Kim +6
Materials science literature contains millions of materials synthesis procedures described in unstructured natural language text. Large-scale analysis of these synthesis procedures…
Inorganic Materials Synthesis Planning with Literature-Trained Neural Networks
Edward Kim, Zach Jensen, Alexander van Grootel +8
Leveraging new data sources is a key step in accelerating the pace of materials design and discovery. To complement the strides in synthesis planning driven by historical, experime…
Efficient Graph-based Word Sense Induction by Distributional Inclusion Vector Embeddings
Haw-Shiuan Chang, Amol Agrawal, Ananya Ganesh +4
Word sense induction (WSI), which addresses polysemy by unsupervised discovery of multiple word senses, resolves ambiguities for downstream NLP tasks and also makes word representa…
Automatically Extracting Action Graphs from Materials Science Synthesis Procedures
Sheshera Mysore, Edward Kim, Emma Strubell +6
Computational synthesis planning approaches have achieved recent success in organic chemistry, where tabulated synthesis procedures are readily available for supervised learning. T…
Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection
Haw-Shiuan Chang, ZiYun Wang, Luke Vilnis +1
Modeling hypernymy, such as poodle is-a dog, is an important generalization aid to many NLP tasks, such as entailment, coreference, relation extraction, and question answering. Sup…
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples
Haw-Shiuan Chang, Erik Learned-Miller, Andrew McCallum
Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates…