14.5k citations · 48.8k across the 185 of their papers we have counts for
21 papers · 1 filter
Directional Analysis of Stochastic Gradient Descent via von Mises-Fisher Distributions in Deep learning
Cheolhyoung Lee, Kyunghyun Cho, Wanmo Kang
Although stochastic gradient descent (SGD) is a driving force behind the recent success of deep learning, our understanding of its dynamics in a high-dimensional parameter space is…
Importance of Search and Evaluation Strategies in Neural Dialogue Modeling
Ilia Kulikov, Alexander H. Miller, Kyunghyun Cho +1
We investigate the impact of search strategies in neural dialogue modeling. We first compare two standard search algorithms, greedy and beam search, as well as our newly proposed i…
Dialogue Natural Language Inference
Sean Welleck, Jason Weston, Arthur Szlam +1
Consistency is a long standing issue faced by dialogue models. In this paper, we frame the consistency of dialogue agents as natural language inference (NLI) and create a new natur…
Pommerman: A Multi-Agent Playground
Cinjon Resnick, Wes Eldridge, David Ha +5
We present Pommerman, a multi-agent environment based on the classic console game Bomberman. Pommerman consists of a set of scenarios, each having at least four players and contain…
Jump to better conclusions: SCAN both left and right
Jasmijn Bastings, Marco Baroni, Jason Weston +2
Lake and Baroni (2018) recently introduced the SCAN data set, which consists of simple commands paired with action sequences and is intended to test the strong generalization abili…
Grammar Induction with Neural Language Models: An Unusual Replication
Phu Mon Htut, Kyunghyun Cho, Samuel R. Bowman
A substantial thread of recent work on latent tree learning has attempted to develop neural network models with parse-valued latent variables and train them on non-parsing tasks, i…
Meta-Learning for Low-Resource Neural Machine Translation
Jiatao Gu, Yong Wang, Yun Chen +2
In this paper, we propose to extend the recently introduced model-agnostic meta-learning algorithm (MAML) for low-resource neural machine translation (NMT). We frame low-resource t…
Backplay: "Man muss immer umkehren"
Cinjon Resnick, Roberta Raileanu, Sanyam Kapoor +3
Model-free reinforcement learning (RL) requires a large number of trials to learn a good policy, especially in environments with sparse rewards. We explore a method to improve the…
Learning Distributed Representations from Reviews for Collaborative Filtering
Amjad Almahairi, Kyle Kastner, Kyunghyun Cho +1
Recent work has shown that collaborative filter-based recommender systems can be improved by incorporating side information, such as natural language reviews, as a way of regulariz…
DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder
Xiaodong Gu, Kyunghyun Cho, Jung-Woo Ha +1
Variational autoencoders~(VAEs) have shown a promise in data-driven conversation modeling. However, most VAE conversation models match the approximate posterior distribution over t…
Classifier-agnostic saliency map extraction
Konrad Zolna, Krzysztof J. Geras, Kyunghyun Cho
Currently available methods for extracting saliency maps identify parts of the input which are the most important to a specific fixed classifier. We show that this strong dependenc…
Conditional molecular design with deep generative models
Seokho Kang, Kyunghyun Cho
Although machine learning has been successfully used to propose novel molecules that satisfy desired properties, it is still challenging to explore a large chemical space efficient…
Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding
Lifu Huang, Kyunghyun Cho, Boliang Zhang +2
We construct a multilingual common semantic space based on distributional semantics, where words from multiple languages are projected into a shared space to enable knowledge and r…
Training a Ranking Function for Open-Domain Question Answering
Phu Mon Htut, Samuel R. Bowman, Kyunghyun Cho
In recent years, there have been amazing advances in deep learning methods for machine reading. In machine reading, the machine reader has to extract the answer from the given grou…
Fine-Grained Attention Mechanism for Neural Machine Translation
Heeyoul Choi, Kyunghyun Cho, Yoshua Bengio
Neural machine translation (NMT) has been a new paradigm in machine translation, and the attention mechanism has become the dominant approach with the state-of-the-art records in m…
Dynamic Meta-Embeddings for Improved Sentence Representations
Douwe Kiela, Changhan Wang, Kyunghyun Cho
While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by…
A Stable and Effective Learning Strategy for Trainable Greedy Decoding
Yun Chen, Victor O. K. Li, Kyunghyun Cho +1
Beam search is a widely used approximate search strategy for neural network decoders, and it generally outperforms simple greedy decoding on tasks like machine translation. However…
Vehicle Communication Strategies for Simulated Highway Driving
Cinjon Resnick, Ilya Kulikov, Kyunghyun Cho +1
Interest in emergent communication has recently surged in Machine Learning. The focus of this interest has largely been either on investigating the properties of the learned protoc…
Controlling Decoding for More Abstractive Summaries with Copy-Based Networks
Noah Weber, Leena Shekhar, Niranjan Balasubramanian +1
Attention-based neural abstractive summarization systems equipped with copy mechanisms have shown promising results. Despite this success, it has been noticed that such a system ge…
Retrieval-Augmented Convolutional Neural Networks for Improved Robustness against Adversarial Examples
Jake Zhao, Kyunghyun Cho
We propose a retrieval-augmented convolutional network and propose to train it with local mixup, a novel variant of the recently proposed mixup algorithm. The proposed hybrid archi…
Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement
Jason Lee, Elman Mansimov, Kyunghyun Cho
We propose a conditional non-autoregressive neural sequence model based on iterative refinement. The proposed model is designed based on the principles of latent variable models an…