154 citations · 861 across the 94 of their papers we have counts for
6 papers · 1 filter
High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws
M. Emrullah Ildiz, Halil Alperen Gozeten, Ege Onur Taga +2
A growing number of machine learning scenarios rely on knowledge distillation where one uses the output of a surrogate model as labels to supervise the training of a target model.…
System Identification via Nuclear Norm Regularization
Yue Sun, Samet Oymak, Maryam Fazel
This paper studies the problem of identifying low-order linear systems via Hankel nuclear norm regularization. Hankel regularization encourages the low-rankness of the Hankel matri…
Generalization Guarantees for Neural Architecture Search with Train-Validation Split
Samet Oymak, Mingchen Li, Mahdi Soltanolkotabi
Neural Architecture Search (NAS) is a popular method for automatically designing optimized architectures for high-performance deep learning. In this approach, it is common to use b…
High Dimensional Data Enrichment: Interpretable, Fast, and Data-Efficient
Amir Asiaee, Samet Oymak, Kevin R. Coombes +1
We consider the problem of multi-task learning in the high dimensional setting. In particular, we introduce an estimator and investigate its statistical and computational propertie…
Fast and Reliable Parameter Estimation from Nonlinear Observations
Samet Oymak, Mahdi Soltanolkotabi
In this paper we study the problem of recovering a structured but unknown parameter from nonlinear observations of the form $y_i=f(\langle {\bf{x}}_i,{\bfθ}^*\rangle…
Finding Dense Clusters via "Low Rank + Sparse" Decomposition
Samet Oymak, Babak Hassibi
Finding "densely connected clusters" in a graph is in general an important and well studied problem in the literature \cite{Schaeffer}. It has various applications in pattern recog…