183 citations · 511 across the 12 of their papers we have counts for
13 papers
Continuous Optimization for p-adic Models
Julian Salazar, Dimitri Kanevsky, Matt Harvey +2
We present the first method for native, continuous gradient descent for machine learning models with -adic parameters. Existing native optimizers are discrete, mostly combinator…
Zero-Shot End-to-End Spoken Language Understanding via Cross-Modal Selective Self-Training
Jianfeng He, Julian Salazar, Kaisheng Yao +2
End-to-end (E2E) spoken language understanding (SLU) is constrained by the cost of collecting speech-semantics pairs, especially when label domains change. Hence, we explore \texti…
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation
Zejiang Hou, Julian Salazar, George Polovets
Large pretrained language models (PLMs) are often domain- or task-adapted via fine-tuning or prompting. Finetuning requires modifying all of the parameters and having enough data t…
Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment
Ethan A. Chi, Julian Salazar, Katrin Kirchhoff
Non-autoregressive models greatly improve decoding speed over typical sequence-to-sequence models, but suffer from degraded performance. Infilling and iterative refinement models m…
Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings
Phillip Keung, Julian Salazar, Yichao Lu +1
We describe an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text. We use multilingual BERT to create source and target sentence…
Don't Use English Dev: On the Zero-Shot Cross-Lingual Evaluation of Contextual Embeddings
Phillip Keung, Yichao Lu, Julian Salazar +1
Multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning, where multilingual BERT is fine-tuned on one source…
Attentional Speech Recognition Models Misbehave on Out-of-domain Utterances
Phillip Keung, Wei Niu, Yichao Lu +2
We discuss the problem of echographic transcription in autoregressive sequence-to-sequence attentional architectures for automatic speech recognition, where a model produces very l…
Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition
Shaoshi Ling, Yuzong Liu, Julian Salazar +1
We propose a novel approach to semi-supervised automatic speech recognition (ASR). We first exploit a large amount of unlabeled audio data via representation learning, where we rec…
Masked Language Model Scoring
Julian Salazar, Davis Liang, Toan Q. Nguyen +1
Pretrained masked language models (MLMs) require finetuning for most NLP tasks. Instead, we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are co…
Transformers without Tears: Improving the Normalization of Self-Attention
Toan Q. Nguyen, Julian Salazar
We evaluate three simple, normalization-centric changes to improve Transformer training. First, we show that pre-norm residual connections (PreNorm) and smaller initializations ena…
BERTphone: Phonetically-Aware Encoder Representations for Utterance-Level Speaker and Language Recognition
Shaoshi Ling, Julian Salazar, Yuzong Liu +1
We introduce BERTphone, a Transformer encoder trained on large speech corpora that outputs phonetically-aware contextual representation vectors that can be used for both speaker an…
Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
Julian Salazar, Katrin Kirchhoff, Zhiheng Huang
The success of self-attention in NLP has led to recent applications in end-to-end encoder-decoder architectures for speech recognition. Separately, connectionist temporal classific…
Conway's subprime Fibonacci sequences
Richard K. Guy, Tanya Khovanova, Julian Salazar
It's the age-old recurrence with a twist: sum the last two terms and if the result is composite, divide by its smallest prime divisor to get the next term (e.g., 0, 1, 1, 2, 3, 5,…