363 citations · 559 across the 15 of their papers we have counts for
15 papers
Controlling the Extraction of Memorized Data from Large Language Models via Prompt-Tuning
Mustafa Safa Ozdayi, Charith Peris, Jack FitzGerald +5
Large Language Models (LLMs) are known to memorize significant portions of their training data. Parts of this memorized content have been shown to be extractable by simply querying…
Low-Resource Compositional Semantic Parsing with Concept Pretraining
Subendhu Rongali, Mukund Sridhar, Haidar Khan +3
Semantic parsing plays a key role in digital voice assistants such as Alexa, Siri, and Google Assistant by mapping natural language to structured meaning representations. When we w…
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model
Saleh Soltan, Shankar Ananthakrishnan, Jack FitzGerald +13
In this work, we demonstrate that multilingual large-scale sequence-to-sequence (seq2seq) models, pre-trained on a mixture of denoising and Causal Language Modeling (CLM) tasks, ar…
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
Jack FitzGerald, Shankar Ananthakrishnan, Konstantine Arkoudas +38
We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B, their subsequent distillation into smaller m…
Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models
Weiqi Sun, Haidar Khan, Nicolas Guenon des Mesnards +2
Semantic parsing is a key NLP task that maps natural language to structured meaning representations. As in many other NLP tasks, SOTA performance in semantic parsing is now attaine…
RescoreBERT: Discriminative Speech Recognition Rescoring with BERT
Liyan Xu, Yile Gu, Jari Kolehmainen +5
Second-pass rescoring is an important component in automatic speech recognition (ASR) systems that is used to improve the outputs from a first-pass decoder by implementing a lattic…
Output Randomization: A Novel Defense for both White-box and Black-box Adversarial Models
Daniel Park, Haidar Khan, Azer Khan +2
Adversarial examples pose a threat to deep neural network models in a variety of scenarios, from settings where the adversary has complete knowledge of the model in a "white box" s…
Using multiple ASR hypotheses to boost i18n NLU performance
Charith Peris, Gokmen Oz, Khadige Abboud +3
Current voice assistants typically use the best hypothesis yielded by their Automatic Speech Recognition (ASR) module as input to their Natural Language Understanding (NLU) module,…
Compressing Transformer-Based Semantic Parsing Models using Compositional Code Embeddings
Prafull Prakash, Saurabh Kumar Shashidhar, Wenlong Zhao +3
The current state-of-the-art task-oriented semantic parsing models use BERT or RoBERTa as pretrained encoders; these models have huge memory footprints. This poses a challenge to t…
Don't Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding
Qile Zhu, Haidar Khan, Saleh Soltan +2
Semantic parsing is one of the key components of natural language understanding systems. A successful parse transforms an input utterance to an action that is easily understood by…
Optimal Mini-Batch Size Selection for Fast Gradient Descent
Michael P. Perrone, Haidar Khan, Changhoan Kim +3
This paper presents a methodology for selecting the mini-batch size that minimizes Stochastic Gradient Descent (SGD) learning time for single and multiple learner problems. By deco…
Deep density ratio estimation for change point detection
Haidar Khan, Lara Marcuse, Bülent Yener
In this work, we propose new objective functions to train deep neural network based density ratio estimators and apply it to a change point detection problem. Existing methods use…
Thwarting finite difference adversarial attacks with output randomization
Haidar Khan, Daniel Park, Azer Khan +1
Adversarial examples pose a threat to deep neural network models in a variety of scenarios, from settings where the adversary has complete knowledge of the model and to the opposit…
Generation & Evaluation of Adversarial Examples for Malware Obfuscation
Daniel Park, Haidar Khan, Bülent Yener
There has been an increased interest in the application of convolutional neural networks for image based malware classification, but the susceptibility of neural networks to advers…
Focal onset seizure prediction using convolutional networks
Haidar Khan, Lara Marcuse, Madeline Fields +2
Objective: This work investigates the hypothesis that focal seizures can be predicted using scalp electroencephalogram (EEG) data. Our first aim is to learn features that distingui…