40 citations · 91 across the 22 of their papers we have counts for
25 papers
Stochastic Regret Guarantees for Online Zeroth- and First-Order Bilevel Optimization
Parvin Nazari, Bojian Hou, Davoud Ataee Tarzanagh +2
Online bilevel optimization (OBO) is a powerful framework for machine learning problems where both outer and inner objectives evolve over time, requiring dynamic updates. Current O…
Truncated Matrix Completion - An Empirical Study
Rishhabh Naik, Nisarg Trivedi, Davoud Ataee Tarzanagh +1
Low-rank Matrix Completion (LRMC) describes the problem where we wish to recover missing entries of partially observed low-rank matrix. Most existing matrix completion work deals w…
Clustering Alzheimer's Disease Subtypes via Similarity Learning and Graph Diffusion
Tianyi Wei, Shu Yang, Davoud Ataee Tarzanagh +6
Alzheimer's disease (AD) is a complex neurodegenerative disorder that affects millions of people worldwide. Due to the heterogeneous nature of AD, its diagnosis and treatment pose…
Optimizing Attention with Mirror Descent: Generalized Max-Margin Token Selection
Addison Kristanto Julistiono, Davoud Ataee Tarzanagh, Navid Azizan
Attention mechanisms have revolutionized several domains of artificial intelligence, such as natural language processing and computer vision, by enabling models to selectively focu…
Fairness-Aware Estimation of Graphical Models
Zhuoping Zhou, Davoud Ataee Tarzanagh, Bojian Hou +2
This paper examines the issue of fairness in the estimation of graphical models (GMs), particularly Gaussian, Covariance, and Ising models. These models play a vital role in unders…
Fair Canonical Correlation Analysis
Zhuoping Zhou, Davoud Ataee Tarzanagh, Bojian Hou +5
This paper investigates fairness and bias in Canonical Correlation Analysis (CCA), a widely used statistical technique for examining the relationship between two sets of variables.…
Transformers as Support Vector Machines
Davoud Ataee Tarzanagh, Yingcong Li, Christos Thrampoulidis +1
Since its inception in "Attention Is All You Need", transformer architecture has led to revolutionary advancements in NLP. The attention layer within the transformer admits a seque…
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation
Davoud Ataee Tarzanagh, Mingchen Li, Pranay Sharma +1
Stochastic approximation with multiple coupled sequences (MSA) has found broad applications in machine learning as it encompasses a rich class of problems including bilevel optimiz…
Max-Margin Token Selection in Attention Mechanism
Davoud Ataee Tarzanagh, Yingcong Li, Xuechen Zhang +1
Attention mechanism is a central component of the transformer architecture which led to the phenomenal success of large language models. However, the theoretical principles underly…
A Penalty-Based Method for Communication-Efficient Decentralized Bilevel Programming
Parvin Nazari, Ahmad Mousavi, Davoud Ataee Tarzanagh +1
Bilevel programming has recently received attention in the literature due to its wide range of applications, including reinforcement learning and hyper-parameter optimization. Howe…
A Bilevel Optimization Method for Tensor Recovery Under Metric Learning Constraints
Maryam Bagherian, Davoud A. Tarzanagh, Ivo Dinov +1
Tensor completion and tensor decomposition are important problems in many domains. In this work, we leverage the connection between these problems to learn a distance metric that i…
Online Bilevel Optimization: Regret Analysis of Online Alternating Gradient Methods
Davoud Ataee Tarzanagh, Parvin Nazari, Bojian Hou +2
This paper introduces \textit{online bilevel optimization} in which a sequence of time-varying bilevel problems is revealed one after the other. We extend the known regret bounds f…
FedNest: Federated Bilevel, Minimax, and Compositional Optimization
Davoud Ataee Tarzanagh, Mingchen Li, Christos Thrampoulidis +1
Standard federated optimization methods successfully apply to stochastic problems with single-level structure. However, many contemporary ML problems -- including adversarial robus…
Fair Community Detection and Structure Learning in Heterogeneous Graphical Models
Davoud Ataee Tarzanagh, Laura Balzano, Alfred O. Hero
Inference of community structure in probabilistic graphical models may not be consistent with fairness constraints when nodes have demographic attributes. Certain demographics may…
Identification and Adaptive Control of Markov Jump Systems: Sample Complexity and Regret Bounds
Yahya Sattar, Zhe Du, Davoud Ataee Tarzanagh +3
Learning how to effectively control unknown dynamical systems is crucial for intelligent autonomous systems. This task becomes a significant challenge when the underlying dynamics…
Certainty Equivalent Quadratic Control for Markov Jump Systems
Zhe Du, Yahya Sattar, Davoud Ataee Tarzanagh +3
Real-world control applications often involve complex dynamics subject to abrupt changes or variations. Markov jump linear systems (MJS) provide a rich framework for modeling such…
Solving a class of non-convex min-max games using adaptive momentum methods
Babak Barazandeh, Davoud Ataee Tarzanagh, George Michailidis
Adaptive momentum methods have recently attracted a lot of attention for training of deep neural networks. They use an exponential moving average of past gradients of the objective…
A Newton-Type Active Set Method for Nonlinear Optimization with Polyhedral Constraints
William W. Hager, Davoud Ataee Tarzanagh
A Newton-type active set algorithm for large-scale minimization subject to polyhedral constraints is proposed. The algorithm consists of a gradient projection step, a second-order…
Adaptive First-and Zeroth-order Methods for Weakly Convex Stochastic Optimization Problems
Parvin Nazari, Davoud Ataee Tarzanagh, George Michailidis
In this paper, we design and analyze a new family of adaptive subgradient methods for solving an important class of weakly convex (possibly nonsmooth) stochastic optimization probl…
Grassmannian Optimization for Online Tensor Completion and Tracking with the t-SVD
Kyle Gilman, Davoud Ataee Tarzanagh, Laura Balzano
We propose a new fast streaming algorithm for the tensor completion problem of imputing missing entries of a low-tubal-rank tensor using the tensor singular value decomposition (t-…
Regularized and Smooth Double Core Tensor Factorization for Heterogeneous Data
Davoud Ataee Tarzanagh, George Michailidis
We introduce a general tensor model suitable for data analytic tasks for {\em heterogeneous} datasets, wherein there are joint low-rank structures within groups of observations, bu…
Online Distributed Estimation of Principal Eigenspaces
Davoud Ataee Tarzanagh, Mohamad Kazem Shirani Faradonbeh, George Michailidis
Principal components analysis (PCA) is a widely used dimension reduction technique with an extensive range of applications. In this paper, an online distributed algorithm is propos…
DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online Optimization
Parvin Nazari, Davoud Ataee Tarzanagh, George Michailidis
Adaptive gradient-based optimization methods such as \textsc{Adagrad}, \textsc{Rmsprop}, and \textsc{Adam} are widely used in solving large-scale machine learning problems includin…
Fast Randomized Algorithms for t-Product Based Tensor Operations and Decompositions with Applications to Imaging Data
Davoud Ataee Tarzanagh, George Michailidis
Tensors of order three or higher have found applications in diverse fields, including image and signal processing, data mining, biomedical engineering and link analysis, to name a…