51 citations · 127 across the 23 of their papers we have counts for
26 papers
SLaNC: Static LayerNorm Calibration
Mahsa Salmani, Nikita Trukhanov, Ilya Soloveychik
The ever increasing sizes of Large Language Models (LLMs) beyond hundreds of billions of parameters have generated enormous pressure on the manufacturers of dedicated hardware acce…
Accurate Block Quantization in LLMs with Outliers
Nikita Trukhanov, Ilya Soloveychik
The demand for inference on extremely large scale LLMs has seen enormous growth in the recent months. It made evident the colossal shortage of dedicated hardware capable of efficie…
Error Probability Bounds for Invariant Causal Prediction via Multiple Access Channels
Austin Goddard, Yu Xiang, Ilya Soloveychik
We consider the problem of lower bounding the error probability under the invariant causal prediction (ICP) framework. To this end, we examine and draw connections between ICP and…
Identifying Direct Causes using Intervened Target Variable
Kang Du, Yu Xiang, Ilya Soloveychik
Identifying the direct causes or causal parents of a target variable is crucial for scientific discovery. Focusing on linear models, the invariant prediction framework was built up…
Block Format Error Bounds and Optimal Block Size Selection
Ilya Soloveychik, Ilya Lyubomirsky, Xin Wang +1
The amounts of data that need to be transmitted, processed, and stored by the modern deep neural networks have reached truly enormous volumes in the last few years calling for the…
Lower Bounds on the Error Probability for Invariant Causal Prediction
Austin Goddard, Yu Xiang, Ilya Soloveychik
It is common practice to collect observations of feature and response pairs from different environments. A natural question is how to identify features that have consistent predict…
Central Limit Theorem for Symmetric Exchangeable Random Variables
Ilya Soloveychik
A central limit theorem for arrays of symmetric row-wise exchangeable random variables is presented. The result is valid for finite and infinite extendable and non-extendable seque…
A Robust Test for Elliptical Symmetry
Ilya Soloveychik
Most signal processing and statistical applications heavily rely on specific data distribution models. The Gaussian distributions, although being the most common choice, are inadeq…
Asymptotically Pseudo-Independent Matrices
Ilya Soloveychik, Vahid Tarokh
We show that the family of pseudo-random matrices recently discovered by Soloveychik, Xiang, and Tarokh in their work `Symmetric Pseudo-Random Matrices' exhibits asymptotic indepen…
Stationary Geometric Graphical Model Selection
Ilya Soloveychik, Vahid Tarokh
We consider the problem of model selection in Gaussian Markov fields in the sample deficient scenario. In many practically important cases, the underlying networks are embedded int…
Region Detection in Markov Random Fields: Gaussian Case
Ilya Soloveychik, Vahid Tarokh
We consider the problem of model selection in Gaussian Markov fields in the sample deficient scenario. The benchmark information-theoretic results in the case of d-regular graphs r…
Large Deviations of Convex Polyominoes
Ilya Soloveychik, Vahid Tarokh
Enumeration of various types of lattice polygons and in particular polyominoes is of primary importance in many machine learning, pattern recognition, and geometric analysis proble…
On the Spectral Norms of Pseudo-Wigner and Related Matrices
Ilya Soloveychik, Vahid Tarokh
We investigate the spectral norms of symmetric matrices from two pseudo-random ensembles. The first is the pseudo-Wigner ensemble introduced in "Pseudo-Wigner Matrices…
A Note on the Concentration of Spectral Measure of Wigner's Matrices
Ilya Soloveychik, Vahid Tarokh
In this short note we derive concentration inequalities for the empirical absolute moments of square symmetric matrices with independent symmetrically distributed +/-1 entries. Mos…
Symmetric Pseudo-Random Matrices
Ilya Soloveychik, Yu Xiang, Vahid Tarokh
We consider the problem of generating symmetric pseudo-random sign (+/-1) matrices based on the similarity of their spectra to Wigner's semicircular law. Using binary m-sequences (…
Pseudo-Wigner Matrices
Ilya Soloveychik, Yu Xiang, Vahid Tarokh
We consider the problem of generating pseudo-random matrices based on the similarity of their spectra to Wigner's semicircular law. We introduce the notion of an r-independent pseu…
Simultaneous penalized M-estimation of covariance matrices using geodesically convex optimization
Esa Ollila, Ilya Soloveychik, David E. Tyler +1
A common assumption when sampling -dimensional observations from distinct group is the equality of the covariance matrices. In this paper, we propose two penalized -estim…
Gaussian and Robust Kronecker Product Covariance Estimation: Existence and Uniqueness
Ilya Soloveychik, Dmitry Trushin
We study the Gaussian and robust covariance estimation, assuming the true covariance matrix to be a Kronecker product of two lower dimensional square matrices. In both settings we…
Joint Inverse Covariances Estimation with Mutual Linear Structure
Ilya Soloveychik, Ami Wiesel
We consider the problem of joint estimation of structured inverse covariance matrices. We perform the estimation using groups of measurements with different covariances of the same…
Joint Covariance Estimation with Mutual Linear Structure
Ilya Soloveychik, Ami Wiesel
We consider the problem of joint estimation of structured covariance matrices. Assuming the structure is unknown, estimation is achieved using heterogeneous training sets. Namely,…
Group Symmetric Robust Covariance Estimation
Ilya Soloveychik, Dmitry Trushin, Ami Wiesel
In this paper we consider Tyler's robust covariance M-estimator under group symmetry constraints. We assume that the covariance matrix is invariant to the conjugation action of a u…
Tyler's Covariance Matrix Estimator in Elliptical Models with Convex Structure
Ilya Soloveychik, Ami Wiesel
We address structured covariance estimation in elliptical distributions by assuming that the covariance is a priori known to belong to a given convex set, e.g., the set of Toeplitz…
Error Bound for Compound Wishart Matrices
Ilya Soloveychik
In this paper we consider non-asymptotic behavior of the real compound Wishart matrices that generalize the classical real Wishart distribution. In particular, we consider matrices…
Performance Analysis of Tyler's Covariance Estimator
Ilya Soloveychik, Ami Wiesel
This paper analyzes the performance of Tyler's M-estimator of the scatter matrix in elliptical populations. We focus on the non-asymptotic setting and derive the estimation error b…