3 citations · 3 across the 15 of their papers we have counts for
12 papers · 1 filter
BRANCH-MoE: Balance-Aware Tree Routing for Large Embedding Models
Gang Fu, Adel Javanmard, MohammadHossein Bateni +1
Mixture-of-experts (MoE) layers increase model capacity without a proportional increase in per-example computation. However, conventional flat routers can yield imbalanced expert u…
SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant
Adel Javanmard, David P. Woodruff, Vahab Mirrokni
Achieving local differential privacy in distributed optimization while maintaining low communication cost remains challenging. Existing vector quantization methods, such as vqSGD,…
Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data
Kareem Amin, Rudrajit Das, Alessandro Epasto +4
The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets. Ho…
Geometric Signatures of Reasoning: A Spectral Perspective on Task Hardness
Aria Masoomi, Mahsa Bazzaz, Adel Javanmard +1
Chain-of-thought (CoT) reasoning enables large language models (LLMs) to solve complex problems by generating intermediate reasoning steps. While much attention has been paid to th…
Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models
Adel Javanmard, Baharan Mirzasoleiman, Vahab Mirrokni
Large Language Models (LLMs) are pretrained on massive datasets and later instruction-tuned via supervised fine-tuning (SFT) or reinforcement learning (RL). Best practices emphasiz…
Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing
Adel Javanmard, Rudrajit Das, Alessandro Epasto +1
Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance. While prior works ha…
DeepCrossAttention: Supercharging Transformer Residual Connections
Mike Heddes, Adel Javanmard, Kyriakos Axiotis +3
Transformer networks have achieved remarkable success across diverse domains, leveraging a variety of architectural innovations, including residual connections. However, traditiona…
Retraining with Predicted Hard Labels Provably Increases Model Accuracy
Rudrajit Das, Inderjit S. Dhillon, Alessandro Epasto +5
The performance of a model trained with noisy labels is often improved by simply \textit{retraining} the model with its \textit{own predicted hard labels} (i.e., 1/0 labels). Yet,…
Optimistic Rates for Learning from Label Proportions
Gene Li, Lin Chen, Adel Javanmard +1
We consider a weakly supervised learning problem called Learning from Label Proportions (LLP), where examples are grouped into ``bags'' and only the average label within each bag i…
PriorBoost: An Adaptive Algorithm for Learning from Aggregate Responses
Adel Javanmard, Matthew Fahrbach, Vahab Mirrokni
This work studies algorithms for learning from aggregate responses. We focus on the construction of aggregation sets (called bags in the literature) for event-level loss functions.…
Learning from Aggregate responses: Instance Level versus Bag Level Loss Functions
Adel Javanmard, Lin Chen, Vahab Mirrokni +2
Due to the rise of privacy concerns, in many practical applications the training data is aggregated before being shared with the learner, in order to protect privacy of users' sens…
Anonymous Learning via Look-Alike Clustering: A Precise Analysis of Model Generalization
Adel Javanmard, Vahab Mirrokni
While personalized recommendations systems have become increasingly popular, ensuring user data protection remains a top concern in the development of these learning systems. A com…