571 citations · 585 across the 12 of their papers we have counts for
13 papers
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Shivang Chopra, Fotis Iliopoulos, Zsolt Kira +1
Click-Through Rate prediction, a core task in recommendation and advertising systems, relies on modeling interactions among sparse categorical features. Explicit cross networks are…
Gemma 4 Technical Report
Gemma Team, Sherif El Abd, Vaibhav Aggarwal +320
We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemm…
GUIDE: Guided Initialization and Distillation of Embeddings
Khoa Trinh, Gaurav Menghani, Erik Vee
Algorithmic efficiency techniques such as distillation (\cite{hinton2015distillation}) are useful in improving model quality without increasing serving costs, provided a larger tea…
LAuReL: Learned Augmented Residual Layer
Gaurav Menghani, Ravi Kumar, Sanjiv Kumar
One of the core pillars of efficient deep learning methods is architectural improvements such as the residual/skip connection, which has led to significantly better model convergen…
SLaM: Student-Label Mixing for Distillation with Unlabeled Examples
Vasilis Kontonis, Fotis Iliopoulos, Khoa Trinh +3
Knowledge distillation with unlabeled examples is a powerful training paradigm for generating compact and lightweight student models in applications where the amount of labeled dat…
Weighted Distillation with Unlabeled Examples
Fotis Iliopoulos, Vasilis Kontonis, Cenk Baykal +3
Distillation with unlabeled examples is a popular and powerful method for training deep neural networks in settings where the amount of labeled data is limited: A large ''teacher''…
Robust Active Distillation
Cenk Baykal, Khoa Trinh, Fotis Iliopoulos +2
Distilling knowledge from a large teacher model to a lightweight one is a widely successful approach for generating compact, powerful models in the semi-supervised learning setting…
Fast Bitmap Fit: A CPU Cache Line friendly memory allocator for single object allocations
Dhruv Matani, Gaurav Menghani
Applications making excessive use of single-object based data structures (such as linked lists, trees, etc...) can see a drop in efficiency over a period of time due to the randomi…
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
Gaurav Menghani
Deep Learning has revolutionized the fields of computer vision, natural language understanding, speech recognition, information retrieval and more. However, with the progressive im…
An Easy-to-Use-and-Deploy Grid Computing Framework
Gaurav Menghani, Anil Harwani, Yash Londhe +1
A few grid-computing tools are available for public use. However, such systems are usually quite complex and require several man-months to set up. In case the user wishes to set-up…
Genome Compression Against a Reference
Anirduddha Laud, Gaurav Menghani, Madhava Keralapura
Being able to store and transmit human genome sequences is an important part in genomic research and industrial applications. The complete human genome has 3.1 billion base pairs (…
Learning from a Teacher using Unlabeled Data
Gaurav Menghani, Sujith Ravi
Knowledge distillation is a widely used technique for model compression. We posit that the teacher model used in a distillation setup, captures relationships between classes, that…
A Simple Solution to the Level-Ancestor Problem
Gaurav Menghani, Dhruv Matani
A Level Ancestory query LA(, ) asks for the the ancestor of the node at a depth . We present a simple solution, which pre-processes the tree in time with …