310 citations · 660 across the 22 of their papers we have counts for
23 papers
Vcc: Scaling Transformers to 128K Tokens or More by Prioritizing Important Tokens
Zhanpeng Zeng, Cole Hawkins, Mingyi Hong +4
Transformers are central in modern natural language processing and computer vision applications. Despite recent works devoted to reducing the quadratic cost of such models (as a fu…
Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing
Shufan Wang, Sebastien Jean, Sailik Sengupta +3
In executable task-oriented semantic parsing, the system aims to translate users' utterances in natural language to machine-interpretable programs (API calls) that can be executed…
Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification
Mujeen Sung, James Gung, Elman Mansimov +5
Intent classification (IC) plays an important role in task-oriented dialogue systems. However, IC models often generalize poorly when training without sufficient annotated examples…
Conversation Style Transfer using Few-Shot Learning
Shamik Roy, Raphael Shu, Nikolaos Pappas +4
Conventional text style transfer approaches focus on sentence-level style transfer without considering contextual information, and the style is described with attributes (e.g., for…
Backward Compatibility During Data Updates by Weight Interpolation
Raphael Schumann, Elman Mansimov, Yi-An Lai +3
Backward compatibility of model predictions is a desired property when updating a machine learning driven application. It allows to seamlessly improve the underlying model without…
Dialog2API: Task-Oriented Dialogue with API Description and Example Programs
Raphael Shu, Elman Mansimov, Tamer Alkhouli +6
Functionality and dialogue experience are two important factors of task-oriented dialogue systems. Conventional approaches with closed schema (e.g., conversational semantic parsing…
Modeling Context With Linear Attention for Scalable Document-Level Translation
Zhaofeng Wu, Hao Peng, Nikolaos Pappas +1
Document-level machine translation leverages inter-sentence dependencies to produce more coherent and consistent translations. However, these models, predominantly based on transfo…
ABC: Attention with Bounded-memory Control
Hao Peng, Jungo Kasai, Nikolaos Pappas +5
Transformer architectures have achieved state-of-the-art results on a variety of sequence modeling tasks. However, their attention mechanism comes with a quadratic complexity in se…
Sentence Bottleneck Autoencoders from Transformer Language Models
Ivan Montero, Nikolaos Pappas, Noah A. Smith
Representation learning for text via pretraining a language model on a large corpus has become a standard starting point for building NLP systems. This approach stands in contrast…
Random Feature Attention
Hao Peng, Nikolaos Pappas, Dani Yogatama +3
Transformers are state-of-the-art models for a variety of sequence modeling tasks. At their core is an attention function which models pairwise interactions between the inputs at e…
Finetuning Pretrained Transformers into RNNs
Jungo Kasai, Hao Peng, Yizhe Zhang +6
Transformers have outperformed recurrent neural networks (RNNs) in natural language generation. But this comes with a significant computational cost, as the attention mechanism's c…
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas +1
Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long se…
Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation
Jungo Kasai, Nikolaos Pappas, Hao Peng +2
Much recent effort has been invested in non-autoregressive neural machine translation, which appears to be an efficient alternative to state-of-the-art autoregressive machine trans…
Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains
Navid Rekabsaz, Nikolaos Pappas, James Henderson +2
Neural language modeling (LM) has led to significant improvements in several applications, including Automatic Speech Recognition. However, they typically require large amounts of…
Deep Residual Output Layers for Neural Language Generation
Nikolaos Pappas, James Henderson
Many tasks, including language generation, benefit from learning the structure of the output space, particularly when the space of output labels is large and the data is sparse. St…
Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation
Xiao Pu, Nikolaos Pappas, James Henderson +1
This paper demonstrates that word sense disambiguation (WSD) can improve neural machine translation (NMT) by widening the source context considered when modeling the senses of pote…
Document-Level Neural Machine Translation with Hierarchical Attention Networks
Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas +1
Neural Machine Translation (NMT) can be improved by including document-level contextual information. For this purpose, we propose a hierarchical attention model to capture the cont…
Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation
Nikolaos Pappas, Lesly Miculicich Werlen, James Henderson
Tying the weights of the target word embeddings with the target word classifiers of neural machine translation models leads to faster training and often to better translation quali…
GILE: A Generalized Input-Label Embedding for Text Classification
Nikolaos Pappas, James Henderson
Neural text classification models typically treat output labels as categorical variables which lack description and semantics. This forces their parametrization to be dependent on…
Self-Attentive Residual Decoder for Neural Machine Translation
Lesly Miculicich Werlen, Nikolaos Pappas, Dhananjay Ram +1
Neural sequence-to-sequence networks with attention have achieved remarkable performance for machine translation. One of the reasons for their effectiveness is their ability to cap…
Multilingual Hierarchical Attention Networks for Document Classification
Nikolaos Pappas, Andrei Popescu-Belis
Hierarchical attention networks have recently achieved remarkable performance for document classification in a given language. However, when multilingual document collections are c…
Multilingual Visual Sentiment Concept Matching
Nikolaos Pappas, Miriam Redi, Mercan Topkara +4
The impact of culture in visual emotion perception has recently captured the attention of multimedia research. In this study, we pro- vide powerful computational linguistics tools…
Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology
Brendan Jou, Tao Chen, Nikolaos Pappas +3
Every culture and language is unique. Our work expressly focuses on the uniqueness of culture and language in relation to human affect, specifically sentiment and emotion semantics…