40 citations · 87 across the 27 of their papers we have counts for
19 papers · 1 filter
CoDi: Conversational Distillation for Grounded Question Answering
Patrick Huber, Arash Einolghozati, Rylan Conway +6
Distilling conversational skills into Small Language Models (SLMs) with approximately 1 billion parameters presents significant challenges. Firstly, SLMs have limited capacity in t…
PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken Language Understanding
Trang Le, Daniel Lazar, Suyoun Kim +6
Spoken Language Understanding (SLU) is a critical component of voice assistants; it consists of converting speech to semantic parses for task execution. Previous works have explore…
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich +10
We present LayerSkip, an end-to-end solution to speed-up inference of large language models (LLMs). First, during training we apply layer dropout, with low dropout rates for earlie…
Small But Funny: A Feedback-Driven Approach to Humor Distillation
Sahithya Ravi, Patrick Huber, Akshat Shrivastava +4
The emergence of Large Language Models (LLMs) has brought to light promising language generation capabilities, particularly in performing tasks like complex reasoning and creative…
Augmenting text for spoken language understanding with Large Language Models
Roshan Sharma, Suyoun Kim, Daniel Lazar +7
Spoken semantic parsing (SSP) involves generating machine-comprehensible parses from input speech. Training robust models for existing application domains represented in training d…
Modality Confidence Aware Training for Robust End-to-End Spoken Language Understanding
Suyoun Kim, Akshat Shrivastava, Duc Le +3
End-to-end (E2E) spoken language understanding (SLU) systems that generate a semantic parse from speech have become more promising recently. This approach uses a single model that…
TreePiece: Faster Semantic Parsing via Tree Tokenization
Sid Wang, Akshat Shrivastava, Sasha Livshits
Autoregressive (AR) encoder-decoder neural networks have proved successful in many NLP problems, including Semantic Parsing -- a task that translates natural language to machine-re…
Introducing Semantics into Speech Encoders
Derek Xu, Shuyan Dong, Changhan Wang +10
Recent studies find existing self-supervised speech encoders contain primarily acoustic rather than semantic information. As a result, pipelined supervised automatic speech recogni…
Data-Efficiency with a Single GPU: An Exploration of Transfer Methods for Small Language Models
Alon Albalak, Akshat Shrivastava, Chinnadhurai Sankar +2
Multi-task learning (MTL), instruction tuning, and prompting have recently been shown to improve the generalizability of large language models to new tasks. However, the benefits o…
STOP: A dataset for Spoken Task Oriented Semantic Parsing
Paden Tomasello, Akshat Shrivastava, Daniel Lazar +12
End-to-end spoken language understanding (SLU) predicts intent directly from audio using a single model. It promises to improve the performance of assistant systems by leveraging a…
Deliberation Model for On-Device Spoken Language Understanding
Duc Le, Akshat Shrivastava, Paden Tomasello +4
We propose a novel deliberation-based approach to end-to-end (E2E) spoken language understanding (SLU), where a streaming automatic speech recognition (ASR) model produces the firs…
Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing
Akshat Shrivastava, Shrey Desai, Anchit Gupta +4
Task-oriented semantic parsing models have achieved strong results in recent years, but unfortunately do not strike an appealing balance between model size, runtime latency, and cr…
RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing
Vivek Gupta, Akshat Shrivastava, Adithya Sagar +2
While large pre-trained language models accumulate a lot of knowledge in their parameters, it has been demonstrated that augmenting it with non-parametric retrieval-based memory ha…
Assessing Data Efficiency in Task-Oriented Semantic Parsing
Shrey Desai, Akshat Shrivastava, Justin Rill +4
Data efficiency, despite being an attractive characteristic, is often challenging to measure and optimize for in task-oriented semantic parsing; unlike exact match, it can require…
Low-Resource Task-Oriented Semantic Parsing via Intrinsic Modeling
Shrey Desai, Akshat Shrivastava, Alexander Zotov +1
Task-oriented semantic parsing models typically have high resource requirements: to support new ontologies (i.e., intents and slots), practitioners crowdsource thousands of samples…
Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog
Arun Babu, Akshat Shrivastava, Armen Aghajanyan +3
Semantic parsing using sequence-to-sequence models allows parsing of deeper representations compared to traditional word tagging based models. In spite of these advantages, widespr…
Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing
Akshat Shrivastava, Pierce Chuang, Arun Babu +4
An effective recipe for building seq2seq, non-autoregressive, task-oriented parsers to map utterances to semantic frames proceeds in three steps: encoding an utterance , predict…
Muppet: Massive Multi-task Representations with Pre-Finetuning
Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava +3
We propose pre-finetuning, an additional large-scale learning stage between language model pre-training and fine-tuning. Pre-finetuning is massively multi-task learning (around 50…
Conversational Semantic Parsing
Armen Aghajanyan, Jean Maillard, Akshat Shrivastava +8
The structured representation for semantic parsing in task-oriented assistant systems is geared towards simple understanding of one-turn queries. Due to the limitations of the repr…