activity
20152026
most citedHow NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation

603 citations · 1.2k across the 21 of their papers we have counts for

collaborators
Showing cs.CLShow all

22 papers · 1 filter

cs.CL2024★ 1 cited

How Teachers Can Use Large Language Models and Bloom's Taxonomy to Create Educational Quizzes

Sabina Elkins, Ekaterina Kochmar, Jackie C. K. Cheung +1

Question generation (QG) is a natural language processing task with an abundance of potential benefits and use cases in the educational domain. In order for this potential to be re…

cs.CL2023★ 3 cited

How Useful are Educational Questions Generated by Large Language Models?

Sabina Elkins, Ekaterina Kochmar, Jackie C. K. Cheung +1

Controllable text generation (CTG) by large language models has a huge potential to transform education for teachers and students alike. Specifically, high quality and diverse ques…

cs.CL2022★ 1 cited

Raising Student Completion Rates with Adaptive Curriculum and Contextual Bandits

Robert Belfer, Ekaterina Kochmar, Iulian Vlad Serban

We present an adaptive learning Intelligent Tutoring System, which uses model-based reinforcement learning in the form of contextual bandits to assign learning activities to studen…

cs.CL2022★ 1 cited

Few-shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems

Devang Kulshreshtha, Muhammad Shayan, Robert Belfer +3

Existing work on generating hints in Intelligent Tutoring Systems (ITS) focuses mostly on manual and non-personalized feedback. In this work, we explore automatically generated que…

cs.CL2022

Question Personalization in an Intelligent Tutoring System

Sabina Elkins, Robert Belfer, Ekaterina Kochmar +2

This paper investigates personalization in the field of intelligent tutoring systems (ITS). We hypothesize that personalization in the way questions are asked improves student lear…

cs.CL2021

Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval

Devang Kulshreshtha, Robert Belfer, Iulian Vlad Serban +1

In this work, we introduce back-training, an alternative to self-training for unsupervised domain adaptation (UDA) from source to target domain. While self-training generates synth…

cs.CL2021★ 1 cited

Deep Discourse Analysis for Generating Personalized Feedback in Intelligent Tutor Systems

Matt Grenander, Robert Belfer, Ekaterina Kochmar +3

We explore creating automated, personalized feedback in an intelligent tutoring system (ITS). Our goal is to pinpoint correct and incorrect concepts in student answers in order to…

cs.CL2020★ 2 cited

Automated Personalized Feedback Improves Learning Gains in an Intelligent Tutoring System

Ekaterina Kochmar, Dung Do Vu, Robert Belfer +3

We investigate how automated, data-driven, personalized feedback in a large-scale intelligent tutoring system (ITS) improves student learning outcomes. We propose a machine learnin…

cs.CL2018

The RLLChatbot: a solution to the ConvAI challenge

Nicolas Gontier, Koustuv Sinha, Peter Henderson +4

Current conversational systems can follow simple commands and answer basic questions, but they have difficulty maintaining coherent and open-ended conversations about specific topi…

cs.CL2018★ 14 cited

A Deep Reinforcement Learning Chatbot (Short Version)

Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain +15

We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capa…

cs.CL2017★ 200 cited

A Deep Reinforcement Learning Chatbot

Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain +15

We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capa…

cs.CL2017★ 1 cited

Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses

Ryan Lowe, Michael Noseworthy, Iulian V. Serban +3

Automatically evaluating the quality of dialogue responses for unstructured domains is a challenging problem. Unfortunately, existing automatic evaluation metrics are biased and co…

cs.CL2016

Piecewise Latent Variables for Neural Variational Text Processing

Iulian V. Serban, Alexander G. Ororbia, Joelle Pineau +1

Advances in neural variational inference have facilitated the learning of powerful directed graphical models with continuous latent variables, such as variational autoencoders. The…

cs.CL2016★ 67 cited

Generative Deep Neural Networks for Dialogue: A Short Review

Iulian Vlad Serban, Ryan Lowe, Laurent Charlin +1

Researchers have recently started investigating deep neural networks for dialogue applications. In particular, generative sequence-to-sequence (Seq2Seq) models have shown promising…

cs.CL2016★ 70 cited

Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation

Iulian Vlad Serban, Tim Klinger, Gerald Tesauro +4

We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic…

cs.CL2016

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe +4

Sequential data often possesses a hierarchical structure with complex dependencies between subsequences, such as found between the utterances in a dialogue. In an effort to model t…

cs.CL2016

On the Evaluation of Dialogue Systems with Next Utterance Classification

Ryan Lowe, Iulian V. Serban, Mike Noseworthy +2

An open challenge in constructing dialogue systems is developing methods for automatically learning dialogue strategies from large amounts of unlabelled data. Recent work has propo…

cs.CL2016★ 603 cited

How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation

Chia-Wei Liu, Ryan Lowe, Iulian V. Serban +3

We investigate evaluation metrics for dialogue response generation systems where supervised labels, such as task completion, are not available. Recent works in response generation…

cs.CL2016

Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus

Iulian Vlad Serban, Alberto García-Durán, Caglar Gulcehre +4

Over the past decade, large-scale supervised learning corpora have enabled machine learning researchers to make substantial advances. However, to this date, there are no large-scal…

cs.CL2015★ 181 cited

A Survey of Available Corpora for Building Data-Driven Dialogue Systems

Iulian Vlad Serban, Ryan Lowe, Peter Henderson +2

During the past decade, several areas of speech and language understanding have witnessed substantial breakthroughs from the use of data-driven models. In the area of dialogue syst…

cs.CL2015

Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio +2

We investigate the task of building open domain, conversational dialogue systems based on large dialogue corpora using generative models. Generative models produce system responses…

cs.CL2015

The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems

Ryan Lowe, Nissan Pow, Iulian Serban +1

This paper introduces the Ubuntu Dialogue Corpus, a dataset containing almost 1 million multi-turn dialogues, with a total of over 7 million utterances and 100 million words. This…