64 citations · 81 across the 11 of their papers we have counts for
13 papers
Stress-Testing the Reasoning Competence of LLMs With Proofs Under Minimal Formalism
Konstantine Arkoudas, Serafim Batzoglou
We introduce ProofGrid, a benchmark suite for evaluating LLM reasoning through machine-checkable proofs rather than final answers alone. ProofGrid contains 15 tasks spanning proof…
GPT-4 Can't Reason
Konstantine Arkoudas
GPT-4 was released in March 2023 to wide acclaim, marking a very substantial improvement across the board over GPT-3.5 (OpenAI's previously best model, which had powered the initia…
Low-Resource Compositional Semantic Parsing with Concept Pretraining
Subendhu Rongali, Mukund Sridhar, Haidar Khan +3
Semantic parsing plays a key role in digital voice assistants such as Alexa, Siri, and Google Assistant by mapping natural language to structured meaning representations. When we w…
PIZZA: A new benchmark for complex end-to-end task-oriented parsing
Konstantine Arkoudas, Nicolas Guenon des Mesnards, Melanie Rubino +4
Much recent work in task-oriented parsing has focused on finding a middle ground between flat slots and intents, which are inexpressive but easy to annotate, and powerful represent…
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
Jack FitzGerald, Shankar Ananthakrishnan, Konstantine Arkoudas +38
We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B, their subsequent distillation into smaller m…
Cross-TOP: Zero-Shot Cross-Schema Task-Oriented Parsing
Melanie Rubino, Nicolas Guenon des Mesnards, Uday Shah +3
Deep learning methods have enabled task-oriented semantic parsing of increasingly complex utterances. However, a single model is still typically trained and deployed for each task…
Compositional Task-Oriented Parsing as Abstractive Question Answering
Wenting Zhao, Konstantine Arkoudas, Weiqi Sun +1
Task-oriented parsing (TOP) aims to convert natural language into machine-readable representations of specific tasks, such as setting an alarm. A popular approach to TOP is to appl…
Training Naturalized Semantic Parsers with Very Little Data
Subendhu Rongali, Konstantine Arkoudas, Melanie Rubino +1
Semantic parsing is an important NLP problem, particularly for voice assistants such as Alexa and Google Assistant. State-of-the-art (SOTA) semantic parsers are seq2seq architectur…
Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models
Weiqi Sun, Haidar Khan, Nicolas Guenon des Mesnards +2
Semantic parsing is a key NLP task that maps natural language to structured meaning representations. As in many other NLP tasks, SOTA performance in semantic parsing is now attaine…
Exploring Transfer Learning For End-to-End Spoken Language Understanding
Subendhu Rongali, Beiye Liu, Liwei Cai +3
Voice Assistants such as Alexa, Siri, and Google Assistant typically use a two-stage Spoken Language Understanding pipeline; first, an Automatic Speech Recognition (ASR) component…
Delexicalized Paraphrase Generation
Boya Yu, Konstantine Arkoudas, Wael Hamza
We present a neural model for paraphrasing and train it to generate delexicalized sentences. We achieve this by creating training data in which each input is paired with a number o…
Grammatical Sequence Prediction for Real-Time Neural Semantic Parsing
Chunyang Xiao, Christoph Teichmann, Konstantine Arkoudas
While sequence-to-sequence (seq2seq) models achieve state-of-the-art performance in many natural language processing tasks, they can be too slow for real-time applications. One per…
Semantically Driven Auto-completion
Konstantine Arkoudas, Mohamed Yahya
The Bloomberg Terminal has been a leading source of financial data and analytics for over 30 years. Through its thousands of functions, the Terminal allows its users to query and r…