64 citations · 243 across the 40 of their papers we have counts for
55 papers
OSFP4: Joint Optimization of Diagonal Smoothing and Block Scales for NVFP4 Quantization
Neriah Ben David, Ori Meir, Or Ordentlich
NVFP4 is an attractive datatype for large language model (LLM) inference, offering compact storage and native tensor-core acceleration. However, preserving accuracy using NVFP4 req…
Maximal correlation under cardinality constraints
Dror Drach, Tomer Berg, Or Ordentlich +1
In this paper, we define and analyze the quantized maximal correlation, an extension of the notion of maximal correlation restricted to functions taking values in sets of bounded c…
High-Rate Quantized Matrix Multiplication II
Or Ordentlich, Yury Polyanskiy
This is the second part of the work investigating quantized matrix multiplication (MatMul). In part I we considered the case of calibration-free quantization, whereas here we discu…
WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization
Egor Lifar, Semyon Savkin, Or Ordentlich +1
This paper considers the problem of converting a given dense linear layer to low precision. The tradeoff between compressed length and output discrepancy is analyzed information th…
Price of metric universality in vector quantization is at most 0.11 bit
Alina Harbuzova, Or Ordentlich, Yury Polyanskiy
Fast computation of a matrix product is a workhorse of modern LLMs. To make their deployment more efficient, a popular approach is that of using a low-precision approxim…
High-Rate Quantized Matrix Multiplication I
Or Ordentlich, Yury Polyanskiy
This paper investigates the problem of quantized matrix multiplication (MatMul), which has become crucial for the efficient deployment of large language models (LLMs). We consider…
The Voronoi Spherical CDF for Lattices and Linear Codes: New Bounds for Quantization and Coding
Or Ordentlich
For a lattice/linear code, we define the Voronoi spherical cumulative density function (CDF) as the CDF of the -norm/Hamming weight of a random vector uniformly distributed…
High-Rate Nested-Lattice Quantized Matrix Multiplication with Small Lookup Tables
Iris Kaplan, Or Ordentlich
Recent work have shown that the quantization for matrix multiplication problem can be optimally solved by quantizing each column in each matrix using a nested lattice code, and the…
NestQuant: Nested Lattice Quantization for Matrix Products and LLMs
Semyon Savkin, Eitan Porat, Or Ordentlich +1
Post-training quantization (PTQ) has emerged as a critical technique for efficient deployment of large language models (LLMs). This work proposes NestQuant, a novel PTQ scheme for…
Optimal Online Bookmaking for Binary Games
Alankrita Bhatt, Or Ordentlich, Oron Sabag
In online betting, the bookmaker can update the payoffs it offers on a particular event many times before the event takes place, and the updated payoffs may depend on the bets accu…
Optimal Quantization for Matrix Multiplication
Or Ordentlich, Yury Polyanskiy
Recent work in machine learning community proposed multiple methods for performing lossy compression (quantization) of large matrices. This quantization is important for accelerati…
The strong data processing inequality under the heat flow
Bo'az Klartag, Or Ordentlich
Let and be probability distributions on , and be their evolution under the heat flow, that is, the probability distributions resulting from convolvi…
Memory Complexity of Estimating Entropy and Mutual Information
Tomer Berg, Or Ordentlich, Ofer Shayevitz
We observe an infinite sequence of independent identically distributed random variables drawn from an unknown distribution over , and our goal is to estim…
Lower Bounds on Mutual Information for Linear Codes Transmitted over Binary Input Channels, and for Information Combining
Uri Erez, Or Ordentlich, Shlomo Shamai
It has been known for a long time that the mutual information between the input sequence and output of a binary symmetric channel (BSC) is upper bounded by the mutual information b…
Statistical Inference with Limited Memory: A Survey
Tomer Berg, Or Ordentlich, Ofer Shayevitz
The problem of statistical inference in its various forms has been the subject of decades-long extensive research. Most of the effort has been focused on characterizing the behavio…
Bounds on the density of smooth lattice coverings
Or Ordentlich, Oded Regev, Barak Weiss
Let be a convex body in , let be a lattice with covolume one, and let . We say that and form an -smooth cover if each point $x \in \mathbb{R}^…
On The Memory Complexity of Uniformity Testing
Tomer Berg, Or Ordentlich, Ofer Shayevitz
In this paper we consider the problem of uniformity testing with limited memory. We observe a sequence of independent identically distributed random variables drawn from a distribu…
Deterministic Finite-Memory Bias Estimation
Tomer Berg, Or Ordentlich, Ofer Shayevitz
In this paper we consider the problem of estimating a Bernoulli parameter using finite memory. Let be a sequence of independent identically distributed Bernoulli r…
On the Role of Channel Capacity in Learning Gaussian Mixture Models
Elad Romanov, Tamir Bendory, Or Ordentlich
This paper studies the sample complexity of learning the unknown centers of a balanced Gaussian mixture model (GMM) in with spherical covariance matrix $σ^2\math…
Blind Modulo Analog-to-Digital Conversion of Vector Processes
Amir Weiss, Everest Huang, Or Ordentlich +1
In a growing number of applications, there is a need to digitize a (possibly high) number of correlated signals whose spectral characteristics are challenging for traditional analo…
Spiked Covariance Estimation from Modulo-Reduced Measurements
Elad Romanov, Or Ordentlich
Consider the rank-1 spiked model: , where is the spike intensity, is an unknown direction and $ξ\sim \mathcal{N}(0,1),…
Blind Modulo Analog-to-Digital Conversion
Amir Weiss, Everest Huang, Or Ordentlich +1
In a growing number of applications, there is a need to digitize signals whose spectral characteristics are challenging for traditional Analog-to-Digital Converters (ADCs). Example…
On Compressed Sensing of Binary Signals for the Unsourced Random Access Channel
Elad Romanov, Or Ordentlich
Motivated by applications in unsourced random access, this paper develops a novel scheme for the problem of compressed sensing of binary signals. In this problem, the goal is to de…
Critical Slowing Down Near Topological Transitions in Rate-Distortion Problems
Shlomi Agmon, Etam Benger, Or Ordentlich +1
In rate-distortion (RD) problems one seeks reduced representations of a source that meet a target distortion constraint. Such optimal representations undergo topological transition…