8.8k citations · 10.6k across the 35 of their papers we have counts for
39 papers
Scaling Laws of Motion Forecasting and Planning -- Technical Report
Mustafa Baniodeh, Kratarth Goel, Scott Ettinger +14
We study the empirical scaling laws of a family of encoder-decoder autoregressive transformer models on the task of joint motion forecasting and planning in the autonomous driving…
Scaling Motion Forecasting Models with Ensemble Distillation
Scott Ettinger, Kratarth Goel, Avikalp Srivastava +1
Motion forecasting has become an increasingly critical component of autonomous robotic systems. Onboard compute budgets typically limit the accuracy of real-time systems. In this w…
MoST: Multi-modality Scene Tokenization for Motion Prediction
Norman Mu, Jingwei Ji, Zhenpei Yang +11
Many existing motion prediction approaches rely on symbolic perception outputs to generate agent trajectories, such as bounding boxes, road graph information and traffic lights. Th…
Let Your Graph Do the Talking: Encoding Structured Data for LLMs
Bryan Perozzi, Bahare Fatemi, Dustin Zelle +4
How can we best encode structured data into sequential form for use in large language models (LLMs)? In this work, we introduce a parameter-efficient method to explicitly represent…
MotionLM: Multi-Agent Motion Forecasting as Language Modeling
Ari Seff, Brian Cera, Dian Chen +6
Reliable forecasting of the future behavior of road agents is a critical component to safe planning in autonomous vehicles. Here, we represent continuous trajectories as sequences…
Fine-Tashkeel: Finetuning Byte-Level Models for Accurate Arabic Text Diacritization
Bashar Al-Rfooh, Gheith Abandah, Rami Al-Rfou
Most of previous work on learning diacritization of the Arabic language relied on training models from scratch. In this paper, we investigate how to leverage pre-trained language m…
Wayformer: Motion Forecasting via Simple & Efficient Attention Networks
Nigamaa Nayakanti, Rami Al-Rfou, Aurick Zhou +3
Motion forecasting for autonomous driving is a challenging task because complex driving scenarios result in a heterogeneous mix of static and dynamic inputs. It is an open problem…
Narrowing the Coordinate-frame Gap in Behavior Prediction Models: Distillation for Efficient and Accurate Scene-centric Motion Forecasting
DiJia Su, Bertrand Douillard, Rami Al-Rfou +2
Behavior prediction models have proliferated in recent years, especially in the popular real-world robotics application of autonomous driving, where representing the distribution o…
VN-Transformer: Rotation-Equivariant Attention for Vector Neurons
Serge Assaad, Carlton Downey, Rami Al-Rfou +2
Rotation equivariance is a desirable property in many practical applications such as motion forecasting and 3D perception, where it can offer benefits like sample efficiency, bette…
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
Tu Vu, Brian Lester, Noah Constant +2
There has been growing interest in parameter-efficient methods to apply pre-trained language models to downstream tasks. Building on the Prompt Tuning approach of Lester et al. (20…
nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?
Mihir Kale, Aditya Siddhant, Noah Constant +3
Recently, mT5 - a massively multilingual version of T5 - leveraged a unified text-to-text format to attain state-of-the-art results on a wide variety of multilingual NLP tasks. In…
ByT5: Towards a token-free future with pre-trained byte-to-byte models
Linting Xue, Aditya Barua, Noah Constant +5
Most widely-used pre-trained language models operate on sequences of tokens corresponding to word or subword units. By comparison, token-free models that operate directly on raw te…
The Power of Scale for Parameter-Efficient Prompt Tuning
Brian Lester, Rami Al-Rfou, Noah Constant
In this work, we explore "prompt tuning", a simple yet effective mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. Unl…
Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training
Oshin Agarwal, Heming Ge, Siamak Shakeri +1
Prior work on Data-To-Text Generation, the task of converting knowledge graph (KG) triples into natural text, focused on domain-specific benchmark datasets. In this paper, however,…
mT5: A massively multilingual pre-trained text-to-text transformer
Linting Xue, Noah Constant, Adam Roberts +5
The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP t…
LAReQA: Language-agnostic answer retrieval from a multilingual pool
Uma Roy, Noah Constant, Rami Al-Rfou +3
We present LAReQA, a challenging new benchmark for language-agnostic answer retrieval from a multilingual candidate pool. Unlike previous cross-lingual tasks, LAReQA tests for "str…
Bridging the Gap for Tokenizer-Free Language Models
Dokook Choe, Rami Al-Rfou, Mandy Guo +2
Purely character-based language models (LMs) have been lagging in quality on large scale datasets, and current state-of-the-art LMs rely on word tokenization. It has been assumed t…
DDGK: Learning Graph Representations for Deep Divergence Graph Kernels
Rami Al-Rfou, Dustin Zelle, Bryan Perozzi
Can neural networks learn to compare graphs without feature engineering? In this paper, we show that it is possible to learn representations for graph similarity with neither domai…
A Tutorial on Network Embeddings
Haochen Chen, Bryan Perozzi, Rami Al-Rfou +1
Network embedding methods aim at learning low-dimensional latent representation of nodes in a network. These representations can be used as features for a wide range of tasks on gr…
Character-Level Language Modeling with Deeper Self-Attention
Rami Al-Rfou, Dokook Choe, Noah Constant +2
LSTMs and other RNN variants have shown strong performance on character-level language modeling. These models are typically trained using truncated backpropagation through time, an…
Watch Your Step: Learning Node Embeddings via Graph Attention
Sami Abu-El-Haija, Bryan Perozzi, Rami Al-Rfou +1
Graph embedding methods represent nodes in a continuous vector space, preserving information from the graph (e.g. by sampling random walks). There are many hyper-parameters to thes…
CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networks
Mustafa Mustafa, Deborah Bard, Wahid Bhimji +3
Inferring model parameters from experimental data is a grand challenge in many sciences, including cosmology. This often relies critically on high fidelity numerical simulations, w…
Learning Edge Representations via Low-Rank Asymmetric Projections
Sami Abu-El-Haija, Bryan Perozzi, Rami Al-Rfou
We propose a new method for embedding graphs while preserving directed edge information. Learning such continuous-space vector representations (or embeddings) of nodes in a graph i…
Efficient Natural Language Response Suggestion for Smart Reply
Matthew Henderson, Rami Al-Rfou, Brian Strope +6
This paper presents a computationally efficient machine-learned method for natural language response suggestion. Feed-forward neural networks using n-gram embedding features encode…