Publications (20)
AdaCliP: Adaptive Clipping for Private SGD
Venkatadheeraj Pichapati, Ananda Theertha Suresh, Felix X. Yu +2
Privacy preserving machine learning algorithms are crucial for learning models over user data to protect sensitive information. Motivated by this, differentially private stochastic…
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling
Shanshan Wu, Alexandros G. Dimakis, Sujay Sanghavi +5
Linear encoding of sparse vectors is widely popular, but is commonly data-independent -- missing any possible extra (but a priori unknown) structure beyond sparsity. In this paper…
Learning Spread-out Local Feature Descriptors
Xu Zhang, Felix X. Yu, Sanjiv Kumar +1
We propose a simple, yet powerful regularization technique that can be used to significantly improve both the pairwise and triplet losses in learning local feature descriptors. The…
Compact Nonlinear Maps and Circulant Extensions
Felix X. Yu, Sanjiv Kumar, Henry Rowley +1
Kernel approximation via nonlinear random feature maps is widely used in speeding up kernel machines. There are two main challenges for the conventional kernel approximation method…
On Learning from Label Proportions
Felix X. Yu, Krzysztof Choromanski, Sanjiv Kumar +2
Learning from Label Proportions (LLP) is a learning setting, where the training data is provided in groups, or "bags", and only the proportion of each class in each bag is known. T…
Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation
Michal Lukasik, Lin Chen, Harikrishna Narasimhan +7
Bipartite ranking is a fundamental supervised learning problem, with the goal of learning a ranking over instances with maximal Area Under the ROC Curve (AUC) against a single bina…
Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces
Ankit Singh Rawat, Aditya Krishna Menon, Wittawat Jitkrittum +4
Negative sampling schemes enable efficient training given a large number of classes, by offering a means to approximate a computationally expensive loss function that takes all lab…
SVM for learning with label proportions
Felix X. Yu, Dong Liu, Sanjiv Kumar +2
We study the problem of learning with label proportions in which the training data is provided in groups and only the proportion of each class in each group is known. We propose a…
Federated Learning: Strategies for Improving Communication Efficiency
Jakub KoneÄný, H. Brendan McMahan, Felix X. Yu +3
Federated Learning is a machine learning setting where the goal is to train a high-quality centralized model while training data remains distributed over a large number of clients…
Orthogonal Random Features
Felix X. Yu, Ananda Theertha Suresh, Krzysztof Choromanski +2
We present an intriguing discovery related to Random Fourier Features: in Gaussian kernel approximation, replacing the random Gaussian matrix by a properly scaled random orthogonal…
Circulant Binary Embedding
Felix X. Yu, Sanjiv Kumar, Yunchao Gong +1
Binary embedding of high-dimensional data requires long codes to preserve the discriminative power of the input space. Traditional binary coding methods often suffer from very high…
Pre-training Tasks for Embedding-based Large-scale Retrieval
Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang +2
We consider the large-scale query-document retrieval problem: given a query (e.g., a question), return the set of relevant documents (e.g., paragraphs containing the answer) from a…
Automatic Engineering of Long Prompts
Cho-Jui Hsieh, Si Si, Felix X. Yu +1
Large language models (LLMs) have demonstrated remarkable capabilities in solving complex open-domain tasks, guided by comprehensive instructions and demonstrations provided in the…
An exploration of parameter redundancy in deep networks with circulant projections
Yu Cheng, Felix X. Yu, Rogerio S. Feris +3
We explore the redundancy of parameters in deep neural networks by replacing the conventional linear projection in fully-connected layers with the circulant projection. The circula…
On Binary Embedding using Circulant Matrices
Felix X. Yu, Aditya Bhaskara, Sanjiv Kumar +2
Binary embeddings provide efficient and powerful ways to perform operations on large scale data. However binary embedding typically requires long codes in order to preserve the dis…
FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients
Jianyu Wang, Hang Qi, Ankit Singh Rawat +4
In classical federated learning, the clients contribute to the overall training by communicating local updates for the underlying model on their private data to a coordinating serv…
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent +56
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a cen…
Distributed Mean Estimation with Limited Communication
Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar +1
Motivated by the need for distributed learning and optimization algorithms with low communication cost, we study communication efficient algorithms for distributed mean estimation.…
Federated Learning with Only Positive Labels
Felix X. Yu, Ankit Singh Rawat, Aditya Krishna Menon +1
We consider learning a multi-class classification model in the federated setting, where each user has access to the positive data associated with only a single class. As a result,…