activity
20172022
most citedDistilling Task-Specific Knowledge from BERT into Simple Neural Networks

335 citations · 506 across the 12 of their papers we have counts for

collaborators
Showing cs.CLShow all

14 papers · 1 filter

cs.CL2022

SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale

Raphael Tang, Karun Kumar, Gefei Yang +7

End-to-end automatic speech recognition systems represent the state of the art, but they rely on thousands of hours of manually annotated speech for training, as well as heavyweigh…

cs.CL2020

Inserting Information Bottlenecks for Attribution in Transformers

Zhiying Jiang, Raphael Tang, Ji Xin +1

Pretrained transformers achieve the state of the art across tasks in natural language processing, motivating researchers to investigate their inner mechanisms. One common direction…

cs.CL20202 cited

Howl: A Deployed, Open-Source Wake Word Detection System

Raphael Tang, Jaejun Lee, Afsaneh Razi +4

We describe Howl, an open-source wake word detection toolkit with native support for open speech datasets, like Mozilla Common Voice and Google Speech Commands. We report benchmark…

cs.CL20201 cited

Showing Your Work Doesn't Always Work

Raphael Tang, Jaejun Lee, Ji Xin +3

In natural language processing, a recently popular line of work explores how to best report the experimental results of neural networks. One exemplar publication, titled "Show Your…

cs.CL202019 cited

DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

Ji Xin, Raphael Tang, Jaejun Lee +2

Large-scale pre-trained language models such as BERT have brought significant improvements to NLP applications. However, they are also notorious for being slow in inference, which…

cs.CL202062 cited

Rapidly Bootstrapping a Question Answering Dataset for COVID-19

Raphael Tang, Rodrigo Nogueira, Edwin Zhang +4

We present CovidQA, the beginnings of a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered from Kaggle's COVID-19 Open Research Da…