150 citations · 326 across the 15 of their papers we have counts for
15 papers
ArK: Augmented Reality with Knowledge Interactive Emergent Ability
Qiuyuan Huang, Jae Sung Park, Abhinav Gupta +8
Despite the growing adoption of mixed reality and interactive AI agents, it remains challenging for these systems to generate high quality 2D/3D scenes in unseen environments. The…
Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
Baolin Peng, Michel Galley, Pengcheng He +8
Large language models (LLMs), such as ChatGPT, are able to generate human-like, fluent responses for many downstream tasks, e.g., task-oriented dialog and question answering. Howev…
Training Vision-Language Transformers from Captions
Liangke Gui, Yingshan Chang, Qiuyuan Huang +4
Vision-Language Transformers can be learned without low-level human labels (e.g. class labels, bounding boxes, etc). Existing work, whether explicitly utilizing bounding boxes or p…
KAT: A Knowledge Augmented Transformer for Vision-and-Language
Liangke Gui, Borui Wang, Qiuyuan Huang +3
The primary focus of recent work with largescale transformers has been on optimizing the amount of information packed into the model's parameters. In this work, we ask a different…
Mapping Natural-language Problems to Formal-language Solutions Using Structured Neural Representations
Kezhen Chen, Qiuyuan Huang, Hamid Palangi +3
Generating formal-language programs represented by relational tuples, such as Lisp programs or mathematical operations, to solve problems stated in natural language is a challengin…
REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning
Ming Jiang, Junjie Hu, Qiuyuan Huang +3
Popular metrics used for evaluating image captioning systems, such as BLEU and CIDEr, provide a single score to gauge the system's overall effectiveness. This score is often not in…
TIGEr: Text-to-Image Grounding for Image Caption Evaluation
Ming Jiang, Qiuyuan Huang, Lei Zhang +5
This paper presents a new metric called TIGEr for the automatic evaluation of image captioning systems. Popular metrics, such as BLEU and CIDEr, are based solely on text matching b…
Object-driven Text-to-Image Synthesis via Adversarial Training
Wenbo Li, Pengchuan Zhang, Lei Zhang +4
In this paper, we propose Object-driven Attentive Generative Adversarial Newtorks (Obj-GANs) that allow object-centered text-to-image synthesis for complex scenes. Following the tw…
Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
Xin Wang, Qiuyuan Huang, Asli Celikyilmaz +5
Vision-language navigation (VLN) is the task of navigating an embodied agent to carry out natural language instructions inside real 3D environments. In this paper, we study how to…
Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz +3
We propose a hierarchically structured reinforcement learning approach to address the challenges of planning for generating coherent multi-sentence stories for the visual storytell…
Generating Diverse and Accurate Visual Captions by Comparative Adversarial Learning
Dianqi Li, Qiuyuan Huang, Xiaodong He +2
We study how to generate captions that are not only accurate in describing an image but also discriminative across different images. The problem is both fundamental and interesting…
Attentive Tensor Product Learning
Qiuyuan Huang, Li Deng, Dapeng Wu +2
This paper proposes a new architecture - Attentive Tensor Product Learning (ATPL) - to represent grammatical structures in deep learning models. ATPL is a new architecture to bridg…
Structured Memory based Deep Model to Detect as well as Characterize Novel Inputs
Pratik Prabhanjan Brahma, Qiuyuan Huang, Dapeng Wu
While deep learning has pushed the boundaries in various machine learning tasks, the current models are still far away from replicating many functions that a normal human brain can…
A Neural-Symbolic Approach to Design of CAPTCHA
Qiuyuan Huang, Paul Smolensky, Xiaodong He +2
CAPTCHAs based on reading text are susceptible to machine-learning-based attacks due to recent significant advances in deep learning (DL). To address this, this paper promotes imag…
Tensor Product Generation Networks for Deep NLP Modeling
Qiuyuan Huang, Paul Smolensky, Xiaodong He +2
We present a new approach to the design of deep networks for natural language processing (NLP), based on the general technique of Tensor Product Representations (TPRs) for encoding…