116 citations · 538 across the 44 of their papers we have counts for
25 papers · 1 filter
Relational Graph Learning for Grounded Video Description Generation
Wenqiao Zhang, Xin Eric Wang, Siliang Tang +5
Grounded video description (GVD) encourages captioning models to attend to appropriate video regions (e.g., objects) dynamically and generate a description. Such a setting can help…
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling
Tsu-Jui Fu, Linjie Li, Zhe Gan +4
A great challenge in video-language (VidL) modeling lies in the disconnection between fixed video representations extracted from image/video understanding models and downstream Vid…
Open-Domain Question-Answering for COVID-19 and Other Emergent Domains
Sharon Levy, Kevin Mo, Wenhan Xiong +1
Since late 2019, COVID-19 has quickly emerged as the newest biomedical domain, resulting in a surge of new information. As with other emergent domains, the discussion surrounding t…
Off-policy Reinforcement Learning with Optimistic Exploration and Distribution Correction
Jiachen Li, Shuo Cheng, Zhenyu Liao +3
Improving the sample efficiency of reinforcement learning algorithms requires effective exploration. Following the principle of (OFU)…
Attacking Open-domain Question Answering by Injecting Misinformation
Liangming Pan, Wenhu Chen, Min-Yen Kan +1
With a rise in false, inaccurate, and misleading information in propaganda, news, and social media, real-world Question Answering (QA) systems face the challenges of synthesizing a…
Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer
Wenda Xu, Michael Saxon, Misha Sra +1
Expert-layman text style transfer technologies have the potential to improve communication between members of scientific communities and the general public. High-quality informatio…
A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space
Alex Jones, William Yang Wang, Kyle Mahowald
In cross-lingual language models, representations for many different languages live in the same space. Here, we investigate the linguistic and non-linguistic factors affecting sent…
D-REX: Dialogue Relation Extraction with Explanations
Alon Albalak, Varun Embar, Yi-Lin Tuan +2
Existing research studies on cross-sentence relation extraction in long-form multi-party conversations aim to improve relation extraction without considering the explainability of…
FinQA: A Dataset of Numerical Reasoning over Financial Data
Zhiyu Chen, Wenhu Chen, Charese Smiley +8
The sheer volume of financial statements makes it difficult for humans to access and analyze a business's financials. Robust numerical reasoning likewise faces unique challenges in…
A Dataset for Answering Time-Sensitive Questions
Wenhu Chen, Xinyi Wang, William Yang Wang
Time is an important dimension in our physical world. Lots of facts can evolve with respect to time. For example, the U.S. President might change every four years. Therefore, it is…
Local Explanation of Dialogue Response Generation
Yi-Lin Tuan, Connor Pryor, Wenhu Chen +2
In comparison to the interpretation of classification models, the explanation of sequence generation models is also an important problem, however it has seen little attention. In t…
ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation
Wanrong Zhu, Xin Eric Wang, An Yan +2
Automatic evaluations for natural language generation (NLG) conventionally rely on token-level or embedding-level comparisons with text references. This differs from human language…
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
Linjie Li, Jie Lei, Zhe Gan +12
Most existing video-and-language (VidL) research focuses on a single dataset, or multiple datasets of a single task. In reality, a truly useful VidL system is expected to be easily…
Counterfactual Maximum Likelihood Estimation for Training Deep Networks
Xinyi Wang, Wenhu Chen, Michael Saxon +1
Although deep learning models have driven state-of-the-art performance on a wide array of tasks, they are prone to spurious correlations that should not be learned as predictive cl…
Zero-shot Fact Verification by Claim Generation
Liangming Pan, Wenhu Chen, Wenhan Xiong +2
Neural models for automated fact verification have achieved promising results thanks to the availability of large, human-annotated datasets. However, for each new domain that requi…
Robotic Surgery With Lean Reinforcement Learning
Yotam Barnoy, Molly O'Brien, Will Wang +1
As surgical robots become more common, automating away some of the burden of complex direct human operation becomes ever more feasible. Model-free reinforcement learning (RL) is a…
FoveaTer: Foveated Transformer for Image Classification
Aditya Jonnalagadda, William Yang Wang, B. S. Manjunath +1
Many animals and humans process the visual field with a varying spatial resolution (foveated vision) and use peripheral processing to make eye movements and point the fovea to acqu…
Comparing Visual Reasoning in Humans and AI
Shravan Murlidaran, William Yang Wang, Miguel P. Eckstein
Recent advances in natural language processing and computer vision have led to AI models that interpret simple scenes at human levels. Yet, we do not have a complete understanding…
Gaze Perception in Humans and CNN-Based Model
Nicole X. Han, William Yang Wang, Miguel P. Eckstein
Making accurate inferences about other individuals' locus of attention is essential for human social interactions and will be important for AI to effectively interact with humans.…
M3L: Language-based Video Editing via Multi-Modal Multi-Level Transformers
Tsu-Jui Fu, Xin Eric Wang, Scott T. Grafton +2
Video editing tools are widely used nowadays for digital design. Although the demand for these tools is high, the prior knowledge required makes it difficult for novices to get sta…
On Hallucination and Predictive Uncertainty in Conditional Language Generation
Yijun Xiao, William Yang Wang
Despite improvements in performances on different natural language generation tasks, deep neural models are prone to hallucinating facts that are incorrect or nonexistent. Differen…
Diagnosing Vision-and-Language Navigation: What Really Matters
Wanrong Zhu, Yuankai Qi, Pradyumna Narayana +6
Vision-and-language navigation (VLN) is a multimodal task where an agent follows natural language instructions and navigates in visual environments. Multiple setups have been propo…
DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents
Tsu-Jui Fu, William Yang Wang, Daniel McDuff +1
Creating presentation materials requires complex multimodal reasoning skills to summarize key concepts and arrange them in a logical and visually pleasing manner. Can machines lear…
Modeling Disclosive Transparency in NLP Application Descriptions
Michael Saxon, Sharon Levy, Xinyi Wang +2
Broader disclosive transparencytruth and clarity in communication regarding the function of AI systemsis widely considered desirable. Unfortunately, it is a nebulous concept,…