57 citations · 129 across the 8 of their papers we have counts for
8 papers
CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models
Hailin Zhang, Zirui Liu, Boxuan Chen +4
Recently, the growing memory demands of embedding tables in Deep Learning Recommendation Models (DLRMs) pose great challenges for model training and deployment. Existing embedding…
Experimental Analysis of Large-scale Learnable Vector Storage Compression
Hailin Zhang, Penghao Zhao, Xupeng Miao +4
Learnable embedding vector is one of the most important applications in machine learning, and is widely used in various database-related domains. However, the high dimensionality o…
Model-enhanced Vector Index
Hailin Zhang, Yujing Wang, Qi Chen +16
Embedding-based retrieval methods construct vector indices to search for document representations that are most similar to the query representations. They are widely used in docume…
Adaptive Multi-Teacher Knowledge Distillation with Meta-Learning
Hailin Zhang, Defang Chen, Can Wang
Multi-Teacher knowledge distillation provides students with additional supervision from multiple pre-trained teachers with diverse information sources. Most existing methods explor…
Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism
Xupeng Miao, Yujie Wang, Youhe Jiang +4
Transformer models have achieved state-of-the-art performance on various domains of applications and gradually becomes the foundations of the advanced large deep learning (DL) mode…
Knowledge Distillation with the Reused Teacher Classifier
Defang Chen, Jian-Ping Mei, Hailin Zhang +3
Knowledge distillation aims to compress a powerful yet cumbersome teacher model into a lightweight student model without much sacrifice of performance. For this purpose, various ap…
Confidence-Aware Multi-Teacher Knowledge Distillation
Hailin Zhang, Defang Chen, Can Wang
Knowledge distillation is initially introduced to utilize additional supervision from a single teacher model for the student model training. To boost the student performance, some…
HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework
Xupeng Miao, Hailin Zhang, Yining Shi +4
Embedding models have been an effective learning paradigm for high-dimensional data. However, one open issue of embedding models is that their representations (latent factors) ofte…