35 citations · 104 across the 27 of their papers we have counts for
27 papers
Towards Improving Interpretability of Language Model Generation through a Structured Knowledge Discovery Approach
Shuqi Liu, Han Wu, Guanzhi Deng +3
Knowledge-enhanced text generation aims to enhance the quality of generated text by utilizing internal or external knowledge sources. While language models have demonstrated impres…
Soundness-Aware Level: A Microscopic Signature that Predicts LLM Reasoning Potential
Xuansheng Wu, Xiaoman Pan, Wenlin Yao +1
Reinforcement learning with verifiable rewards (RLVR) can elicit strong reasoning in large language models (LLMs), while their performance after RLVR varies dramatically across dif…
Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models
Xinran Zhao, Hongming Zhang, Xiaoman Pan +4
For a LLM to be trustworthy, its confidence level should be well-calibrated with its actual performance. While it is now common sense that LLM performances are greatly impacted by…
Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
Rui Yang, Xiaoman Pan, Feng Luo +4
We consider the problem of multi-objective alignment of foundation models with human preferences, which is a critical step towards helpful and harmless AI systems. However, it is g…
MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning
Fuxiao Liu, Xiaoyang Wang, Wenlin Yao +5
With the rapid development of large language models (LLMs) and their integration into large multimodal models (LMMs), there has been impressive progress in zero-shot completion of…
From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning
Xuansheng Wu, Wenlin Yao, Jianshu Chen +4
Large Language Models (LLMs) have achieved remarkable success, where instruction tuning is the critical step in aligning LLMs with user intentions. In this work, we investigate how…
A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation
Neeraj Varshney, Wenlin Yao, Hongming Zhang +2
Recently developed large language models have achieved remarkable success in generating fluent and coherent text. However, these models often tend to 'hallucinate' which critically…
Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models
Jiaao Chen, Xiaoman Pan, Dian Yu +4
We investigate how to elicit compositional generalization capabilities in large language models (LLMs). Compositional generalization empowers LLMs to solve complex problems by comb…
Thrust: Adaptively Propels Large Language Models with External Knowledge
Xinran Zhao, Hongming Zhang, Xiaoman Pan +3
Although large-scale pre-trained language models (PTLMs) are shown to encode rich knowledge in their model parameters, the inherent knowledge in PTLMs can be opaque or static, maki…
PIVOINE: Instruction Tuning for Open-world Information Extraction
Keming Lu, Xiaoman Pan, Kaiqiang Song +3
We consider the problem of Open-world Information Extraction (Open-world IE), which extracts comprehensive entity profiles from unstructured texts. Different from the conventional…
CEO: Corpus-based Open-Domain Event Ontology Induction
Nan Xu, Hongming Zhang, Jianshu Chen
Existing event-centric NLP models often only apply to the pre-defined ontology, which significantly restricts their generalization capabilities. This paper presents CEO, a novel Co…
Learning Language Representations with Logical Inductive Bias
Jianshu Chen
Transformer architectures have achieved great success in solving natural language tasks, which learn strong language representations from large-scale unlabeled texts. In this paper…
ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completion
Pei Chen, Wenlin Yao, Hongming Zhang +4
Knowledge base completion (KBC) aims to predict the missing links in knowledge graphs. Previous KBC tasks and approaches mainly focus on the setting where all test entities and rel…
Join-Chain Network: A Logical Reasoning View of the Multi-head Attention in Transformer
Jianyi Zhang, Yiran Chen, Jianshu Chen
Developing neural architectures that are capable of logical reasoning has become increasingly important for a wide range of applications (e.g., natural language processing). Toward…
Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination
Yue Yang, Wenlin Yao, Hongming Zhang +3
Large-scale pretrained language models have made significant advances in solving downstream language understanding tasks. However, they generally suffer from reporting bias, the ph…
Explanations from Large Language Models Make Small Reasoners Better
Shiyang Li, Jianshu Chen, Yelong Shen +9
Integrating free-text explanations to in-context learning of large language models (LLM) is shown to elicit strong reasoning capabilities along with reasonable explanations. In thi…
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models
Xiaoman Pan, Wenlin Yao, Hongming Zhang +3
Fully-parametric language models generally require a huge number of model parameters to store the necessary knowledge for solving multiple natural language tasks in zero/few-shot s…
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks
Zhenhailong Wang, Xiaoman Pan, Dian Yu +3
Although large language models have achieved impressive zero-shot ability, the huge model size generally incurs high cost. Recently, semi-parametric language models, which augment…
Blessing of Class Diversity in Pre-training
Yulai Zhao, Jianshu Chen, Simon S. Du
This paper presents a new statistical analysis aiming to explain the recent superior achievements of the pre-training techniques in natural language processing (NLP). We prove that…
C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References
Xiang Yue, Xiaoman Pan, Wenlin Yao +3
We consider the problem of pretraining a two-stage open-domain question answering (QA) system (retriever + reader) with strong transfer capabilities. The key challenge is how to co…
Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension
Chao Zhao, Wenlin Yao, Dian Yu +3
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances, which are either scattered around or implicitly implied in different turns…
PRIMA: Planner-Reasoner Inside a Multi-task Reasoning Agent
Daoming Lyu, Bo Liu, Jianshu Chen
We consider the problem of multi-task reasoning (MTR), where an agent can solve multiple tasks via (first-order) logic reasoning. This capability is essential for human-like intell…
Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories
Wenlin Yao, Xiaoman Pan, Lifeng Jin +3
Word Sense Disambiguation (WSD) aims to automatically identify the exact meaning of one word according to its context. Existing supervised models struggle to make correct predictio…
Comprehensive Image Captioning via Scene Graph Decomposition
Yiwu Zhong, Liwei Wang, Jianshu Chen +2
We address the challenging problem of image captioning by revisiting the representation of image scene graph. At the core of our method lies the decomposition of a scene graph into…