567 citations · 646 across the 17 of their papers we have counts for
7 papers · 1 filter
How to Train Data-Efficient LLMs
Noveen Sachdeva, Benjamin Coleman, Wang-Cheng Kang +6
The training of large language models (LLMs) is expensive. In this paper, we study data-efficient approaches for pre-training LLMs, i.e., techniques that aim to optimize the Pareto…
Farzi Data: Autoregressive Data Distillation
Noveen Sachdeva, Zexue He, Wang-Cheng Kang +3
We study data distillation for auto-regressive machine learning tasks, where the input and output have a strict left-to-right causal structure. More specifically, we propose Farzi,…
Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems
Benjamin Coleman, Wang-Cheng Kang, Matthew Fahrbach +4
Learning high-quality feature embeddings efficiently and effectively is critical for the performance of web-scale machine learning systems. A typical model ingests hundreds of feat…
Learning to Embed Categorical Features without Embedding Tables for Recommendation
Wang-Cheng Kang, Derek Zhiyuan Cheng, Tiansheng Yao +4
Embedding learning of categorical features (e.g. user/item IDs) is at the core of various recommendation models including matrix factorization and neural collaborative filtering. T…
Beyond Point Estimate: Inferring Ensemble Prediction Variation from Neuron Activation Strength in Recommender Systems
Zhe Chen, Yuyan Wang, Dong Lin +4
Despite deep neural network (DNN)'s impressive prediction performance in various domains, it is well known now that a set of DNN models trained with the same model specification an…
Content-based Music Similarity with Triplet Networks
Joseph Cleveland, Derek Cheng, Michael Zhou +2
We explore the feasibility of using triplet neural networks to embed songs based on content-based music similarity. Our network is trained using triplets of songs such that two son…
Self-supervised Learning for Large-scale Item Recommendations
Tiansheng Yao, Xinyang Yi, Derek Zhiyuan Cheng +8
Large scale recommender models find most relevant items from huge catalogs, and they play a critical role in modern search and recommendation systems. To model the input space with…