116 citations · 538 across the 44 of their papers we have counts for
46 papers
Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models
Alfonso Amayuelas, Kyle Wong, Liangming Pan +2
This paper investigates the capabilities of Large Language Models (LLMs) in the context of understanding their knowledge and uncertainty over questions. Specifically, we focus on a…
On the Risk of Misinformation Pollution with Large Language Models
Yikang Pan, Liangming Pan, Wenhu Chen +3
In this paper, we comprehensively investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation and its subsequent impact…
EDIS: Entity-Driven Image Search over Multimodal Web Content
Siqi Liu, Weixi Feng, Tsu-jui Fu +2
Making image retrieval methods practical for real-world search applications requires significant progress in dataset scales, entity comprehension, and multimodal information fusion…
HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data
Kai Nakamura, Sharon Levy, Yi-Lin Tuan +2
A pressing challenge in current dialogue systems is to successfully converse with users on topics with information distributed across different modalities. Previous work in multitu…
Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains
Alon Albalak, Sharon Levy, William Yang Wang
Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains. However, low-resource settings such as new and emerging do…
Relational Graph Learning for Grounded Video Description Generation
Wenqiao Zhang, Xin Eric Wang, Siliang Tang +5
Grounded video description (GVD) encourages captioning models to attend to appropriate video regions (e.g., objects) dynamically and generate a description. Such a setting can help…
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling
Tsu-Jui Fu, Linjie Li, Zhe Gan +4
A great challenge in video-language (VidL) modeling lies in the disconnection between fixed video representations extracted from image/video understanding models and downstream Vid…
Open-Domain Question-Answering for COVID-19 and Other Emergent Domains
Sharon Levy, Kevin Mo, Wenhan Xiong +1
Since late 2019, COVID-19 has quickly emerged as the newest biomedical domain, resulting in a surge of new information. As with other emergent domains, the discussion surrounding t…
Off-policy Reinforcement Learning with Optimistic Exploration and Distribution Correction
Jiachen Li, Shuo Cheng, Zhenyu Liao +3
Improving the sample efficiency of reinforcement learning algorithms requires effective exploration. Following the principle of (OFU)…
Attacking Open-domain Question Answering by Injecting Misinformation
Liangming Pan, Wenhu Chen, Min-Yen Kan +1
With a rise in false, inaccurate, and misleading information in propaganda, news, and social media, real-world Question Answering (QA) systems face the challenges of synthesizing a…
Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer
Wenda Xu, Michael Saxon, Misha Sra +1
Expert-layman text style transfer technologies have the potential to improve communication between members of scientific communities and the general public. High-quality informatio…
A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space
Alex Jones, William Yang Wang, Kyle Mahowald
In cross-lingual language models, representations for many different languages live in the same space. Here, we investigate the linguistic and non-linguistic factors affecting sent…
D-REX: Dialogue Relation Extraction with Explanations
Alon Albalak, Varun Embar, Yi-Lin Tuan +2
Existing research studies on cross-sentence relation extraction in long-form multi-party conversations aim to improve relation extraction without considering the explainability of…
FinQA: A Dataset of Numerical Reasoning over Financial Data
Zhiyu Chen, Wenhu Chen, Charese Smiley +8
The sheer volume of financial statements makes it difficult for humans to access and analyze a business's financials. Robust numerical reasoning likewise faces unique challenges in…
A Dataset for Answering Time-Sensitive Questions
Wenhu Chen, Xinyi Wang, William Yang Wang
Time is an important dimension in our physical world. Lots of facts can evolve with respect to time. For example, the U.S. President might change every four years. Therefore, it is…
Local Explanation of Dialogue Response Generation
Yi-Lin Tuan, Connor Pryor, Wenhu Chen +2
In comparison to the interpretation of classification models, the explanation of sequence generation models is also an important problem, however it has seen little attention. In t…
ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation
Wanrong Zhu, Xin Eric Wang, An Yan +2
Automatic evaluations for natural language generation (NLG) conventionally rely on token-level or embedding-level comparisons with text references. This differs from human language…
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
Linjie Li, Jie Lei, Zhe Gan +12
Most existing video-and-language (VidL) research focuses on a single dataset, or multiple datasets of a single task. In reality, a truly useful VidL system is expected to be easily…
Counterfactual Maximum Likelihood Estimation for Training Deep Networks
Xinyi Wang, Wenhu Chen, Michael Saxon +1
Although deep learning models have driven state-of-the-art performance on a wide array of tasks, they are prone to spurious correlations that should not be learned as predictive cl…
Zero-shot Fact Verification by Claim Generation
Liangming Pan, Wenhu Chen, Wenhan Xiong +2
Neural models for automated fact verification have achieved promising results thanks to the availability of large, human-annotated datasets. However, for each new domain that requi…
Robotic Surgery With Lean Reinforcement Learning
Yotam Barnoy, Molly O'Brien, Will Wang +1
As surgical robots become more common, automating away some of the burden of complex direct human operation becomes ever more feasible. Model-free reinforcement learning (RL) is a…
FoveaTer: Foveated Transformer for Image Classification
Aditya Jonnalagadda, William Yang Wang, B. S. Manjunath +1
Many animals and humans process the visual field with a varying spatial resolution (foveated vision) and use peripheral processing to make eye movements and point the fovea to acqu…
Comparing Visual Reasoning in Humans and AI
Shravan Murlidaran, William Yang Wang, Miguel P. Eckstein
Recent advances in natural language processing and computer vision have led to AI models that interpret simple scenes at human levels. Yet, we do not have a complete understanding…
Gaze Perception in Humans and CNN-Based Model
Nicole X. Han, William Yang Wang, Miguel P. Eckstein
Making accurate inferences about other individuals' locus of attention is essential for human social interactions and will be important for AI to effectively interact with humans.…