7 citations · 16 across the 21 of their papers we have counts for
13 papers · 1 filter
Federation over Text: Insight Sharing for Multi-Agent Reasoning
Dixi Yao, Tahseen Rabbani, Manzil Zaheer +1
Modern agents specialize in varying domains while there is no clear approach combining different domain skills. We propose a federated learning-like framework, Federation over Text…
Asynchronous Heavy-Tailed Optimization
Junfei Sun, Dixi Yao, Xuchen Gong +3
Heavy-tailed stochastic gradient noise, commonly observed in transformer models, can destabilize the optimization process. Recent works mainly focus on developing and understanding…
DISCO: A Browser-Based Privacy-Preserving Framework for Distributed Collaborative Learning
Julien T. T. Vignoud, Valérian Rousset, Hugo El Guedj +28
Data is often impractical to share for a range of well considered reasons, such as concerns over privacy, intellectual property, and legal constraints. This not only fragments the…
Remote Labor Index: Measuring AI Automation of Remote Work
Mantas Mazeika, Alice Gatti, Cristina Menghini +44
AIs have made rapid progress on research-oriented benchmarks of knowledge and reasoning, but it remains unclear how these gains translate into economic value and automation. To mea…
Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs
Thierry Bossy, Julien Vignoud, Tahseen Rabbani +2
Federated learning (FL) is a popular paradigm for collaborative training which avoids direct data exposure between clients. However, data privacy issues still remain: FL-trained la…
HashEvict: A Pre-Attention KV Cache Eviction Strategy using Locality-Sensitive Hashing
Minghui Liu, Tahseen Rabbani, Tony O'Halloran +5
Transformer-based large language models (LLMs) use the key-value (KV) cache to significantly accelerate inference by storing the key and value embeddings of past tokens. However, t…
Balancing Label Imbalance in Federated Environments Using Only Mixup and Artificially-Labeled Noise
Kyle Sang, Tahseen Rabbani, Furong Huang
Clients in a distributed or federated environment will often hold data skewed towards differing subsets of labels. This scenario, referred to as heterogeneous or non-iid federated…
Sketch-GNN: Scalable Graph Neural Networks with Sublinear Training Complexity
Mucong Ding, Tahseen Rabbani, Bang An +2
Graph Neural Networks (GNNs) are widely applied to graph learning problems such as node classification. When scaling up the underlying graphs of GNNs to a larger size, we are force…
Calibrated Dataset Condensation for Faster Hyperparameter Search
Mucong Ding, Yuancheng Xu, Tahseen Rabbani +5
Dataset condensation can be used to reduce the computational cost of training multiple models on a large dataset by condensing the training dataset into a small synthetic set. Stat…
conv_einsum: A Framework for Representation and Fast Evaluation of Multilinear Operations in Convolutional Tensorial Neural Networks
Tahseen Rabbani, Jiahao Su, Xiaoyu Liu +3
Modern ConvNets continue to achieve state-of-the-art results over a vast array of vision and image classification tasks, but at the cost of increasing parameters. One strategy for…
Large-Scale Distributed Learning via Private On-Device Locality-Sensitive Hashing
Tahseen Rabbani, Marco Bornstein, Furong Huang
Locality-sensitive hashing (LSH) based frameworks have been used efficiently to select weight vectors in a dense hidden layer with high cosine similarity to an input, enabling dyna…
Comfetch: Federated Learning of Large Networks on Constrained Clients via Sketching
Tahseen Rabbani, Brandon Feng, Marco Bornstein +5
Federated learning (FL) is a popular paradigm for private and collaborative model training on the edge. In centralized FL, the parameters of a global architecture (such as a deep n…
Practical and Fast Momentum-Based Power Methods
Tahseen Rabbani, Apollo Jain, Arjun Rajkumar +1
The power method is a classical algorithm with broad applications in machine learning tasks, including streaming PCA, spectral clustering, and low-rank matrix approximation. The di…