10 citations · 20 across the 15 of their papers we have counts for
8 papers · 1 filter
DeepCrossAttention: Supercharging Transformer Residual Connections
Mike Heddes, Adel Javanmard, Kyriakos Axiotis +3
Transformer networks have achieved remarkable success across diverse domains, leveraging a variety of architectural innovations, including residual connections. However, traditiona…
Data-Efficient Learning via Clustering-Based Sensitivity Sampling: Foundation Models and Beyond
Kyriakos Axiotis, Vincent Cohen-Addad, Monika Henzinger +5
We study the data selection problem, whose aim is to select a small representative subset of data that can be used to efficiently train a machine learning model. We present a new d…
SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization
Taisuke Yasuda, Kyriakos Axiotis, Gang Fu +2
Neural network pruning is a key technique towards engineering large yet scalable, interpretable, and generalizable models. Prior work on the subject has developed largely along two…
Greedy PIG: Adaptive Integrated Gradients
Kyriakos Axiotis, Sami Abu-al-haija, Lin Chen +2
Deep learning has become the standard approach for most machine learning tasks. While its impact is undeniable, interpreting the predictions of deep learning models from a human pe…
Performance of Regularization for Sparse Convex Optimization
Kyriakos Axiotis, Taisuke Yasuda
Despite widespread adoption in practice, guarantees for the LASSO and Group LASSO are strikingly lacking in settings beyond statistical problems, and these algorithms are usually c…
Gradient Descent Converges Linearly for Logistic Regression on Separable Data
Kyriakos Axiotis, Maxim Sviridenko
We show that running gradient descent with variable learning rate guarantees loss for the logistic regression objective, where the error decays…
Local Search Algorithms for Rank-Constrained Convex Optimization
Kyriakos Axiotis, Maxim Sviridenko
We propose greedy and local search algorithms for rank-constrained convex optimization, namely solving given a convex function $R…
Sparse Convex Optimization via Adaptively Regularized Hard Thresholding
Kyriakos Axiotis, Maxim Sviridenko
The goal of Sparse Convex Optimization is to optimize a convex function under a sparsity constraint , where is the target number of non-zero entries in a feas…