23 citations · 106 across the 25 of their papers we have counts for
26 papers
Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
Xinyu Lian, Walid Krichene, Beichen Huang +4
Quantization is widely used to reduce the inference cost of large language models, but its effect on reasoning models is not fully captured by final-answer accuracy or per-token la…
Private Learning with Public Feature Conditioning
Shuli Jiang, Walid Krichene, Nicolas Mayoraz
We study differentially private (DP) regression in settings where each data sample includes public, non-sensitive features -- common in applications such as recommendation and adve…
MAC-Attention: a Match-Amend-Complete Scheme for Fast and Accurate Attention Computation
Jinghan Yao, Sam Adé Jacobs, Walid Krichene +2
Long-context decoding in LLMs is IO-bound: each token re-reads an ever-growing KV cache. Prior accelerations cut bytes via compression, which lowers fidelity, or selection/eviction…
Training Differentially Private Ad Prediction Models with Semi-Sensitive Features
Lynn Chua, Qiliang Cui, Badih Ghazi +9
Motivated by problems arising in digital advertising, we introduce the task of training differentially private (DP) machine learning models with semi-sensitive features. In this se…
Private Learning with Public Features
Walid Krichene, Nicolas Mayoraz, Steffen Rendle +3
We study a class of private learning problems in which the data is a join of private and public features. This is often the case in private personalization tasks such as recommenda…
Private Matrix Factorization with Public Item Features
Mihaela Curmei, Walid Krichene, Li Zhang +1
We consider the problem of training private recommendation models with access to public item features. Training with Differential Privacy (DP) offers strong privacy guarantees, at…
Multi-Task Differential Privacy Under Distribution Skew
Walid Krichene, Prateek Jain, Shuang Song +3
We study the problem of multi-task learning under user-level differential privacy, in which users contribute data to tasks, each involving a subset of users. One important…
Differentially Private Image Classification from Features
Harsh Mehta, Walid Krichene, Abhradeep Thakurta +2
Leveraging transfer learning has recently been shown to be an effective strategy for training large models with Differential Privacy (DP). Moreover, somewhat surprisingly, recent w…
Reciprocity in Machine Learning
Mukund Sundararajan, Walid Krichene
Machine learning is pervasive. It powers recommender systems such as Spotify, Instagram and YouTube, and health-care systems via models that predict sleep patterns, or the risk of…
ALX: Large Scale Matrix Factorization on TPUs
Harsh Mehta, Steffen Rendle, Walid Krichene +1
We present ALX, an open-source library for distributed matrix factorization using Alternating Least Squares, written in JAX. Our design allows for efficient use of the TPU architec…
iALS++: Speeding up Matrix Factorization with Subspace Optimization
Steffen Rendle, Walid Krichene, Li Zhang +1
iALS is a popular algorithm for learning matrix factorization models from implicit feedback with alternating least squares. This algorithm was invented over a decade ago but still…
Revisiting the Performance of iALS on Item Recommendation Benchmarks
Steffen Rendle, Walid Krichene, Li Zhang +1
Matrix factorization learned by implicit alternating least squares (iALS) is a popular baseline in recommender system research publications. iALS is known to be one of the most com…
Private Alternating Least Squares: Practical Private Matrix Completion with Tighter Rates
Steve Chien, Prateek Jain, Walid Krichene +4
We study the problem of differentially private (DP) matrix completion under user-level privacy. We design a joint differentially private variant of the popular Alternating-Least-Sq…
Global Convergence of Second-order Dynamics in Two-layer Neural Networks
Walid Krichene, Kenneth F. Caluya, Abhishek Halder
Recent results have shown that for two-layer fully connected neural networks, gradient flow converges to a global optimum in the infinite width limit, by making a connection betwee…
Neural Collaborative Filtering vs. Matrix Factorization Revisited
Steffen Rendle, Walid Krichene, Li Zhang +1
Embedding based models have been the state of the art in collaborative filtering for over a decade. Traditionally, the dot product or higher order equivalents have been used to com…
Superbloom: Bloom filter meets Transformer
John Anderson, Qingqing Huang, Walid Krichene +2
We extend the idea of word pieces in natural language models to machine learning tasks on opaque ids. This is achieved by applying hash functions to map each id to multiple hash to…
Scaling Up Collaborative Filtering Data Sets through Randomized Fractal Expansions
Francois Belletti, Karthik Lakshmanan, Walid Krichene +7
Recommender system research suffers from a disconnect between the size of academic data sets and the scale of industrial production systems. In order to bridge that gap, we propose…
Scalable Realistic Recommendation Datasets through Fractal Expansions
Francois Belletti, Karthik Lakshmanan, Walid Krichene +2
Recommender System research suffers currently from a disconnect between the size of academic data sets and the scale of industrial production systems. In order to bridge that gap w…
Efficient Training on Very Large Corpora via Gramian Estimation
Walid Krichene, Nicolas Mayoraz, Steffen Rendle +5
We study the problem of learning similarity functions over very large corpora using neural network embedding models. These models are typically trained using SGD with sampling of r…
Acceleration and Averaging in Stochastic Mirror Descent Dynamics
Walid Krichene, Peter L. Bartlett
We formulate and study a general family of (continuous-time) stochastic dynamics for accelerated first-order minimization of smooth convex functions. Building on an averaging formu…
Minimizing Regret on Reflexive Banach Spaces and Learning Nash Equilibria in Continuous Zero-Sum Games
Maximilian Balandat, Walid Krichene, Claire Tomlin +1
We study a general version of the adversarial online learning problem. We are given a decision set in a reflexive Banach space and a sequence of reward vectors in…
Differential Privacy of Populations in Routing Games
Roy Dong, Walid Krichene, Alexandre M. Bayen +1
As our ground transportation infrastructure modernizes, the large amount of data being measured, transmitted, and stored motivates an analysis of the privacy aspect of these emergi…
Dual Averaging on Compactly-Supported Distributions And Application to No-Regret Learning on a Continuum
Walid Krichene
We consider an online learning problem on a continuum. A decision maker is given a compact feasible set , and is faced with the following sequential problem: at iteration~, t…
Learning Nash Equilibria in Congestion Games
Walid Krichene, Benjamin Drighès, Alexandre M. Bayen
We study the repeated congestion game, in which multiple populations of players share resources, and make, at each iteration, a decentralized decision on which resources to utilize…