12 papers
Acceleration of Diffusion Language Model through Discrete Average Generator
Yidong Ouyang, Zhengyan Wan, Themis Haris +8
Discrete diffusion models and flow matching have emerged as powerful frameworks for generative modeling over discrete state spaces, yet efficient few-step generation remains a fund…
Elastic Threshold Attention: Learned Contextual Sparsity for Long-Context Decoding
Themistoklis Haris, Henry Li, Maryam Karimzadehgan
Massive KV caches can cause severe memory-bandwidth bottlenecks during long-context decoding. Sparse attention methods mitigate this problem, but often drop necessary context, lead…
Testing k-submodularity
Themistoklis Haris, Diptaksho Palit
We initiate the study of property testing for -submodular functions, a higher-dimensional analogue of submodular functions defined on partial partitions of a ground set. While $…
Efficient Algorithms for Influence Maximization in General Models and Observed Cascades
Fabian Spaeh, Themistoklis Haris, Alina Ene +1
We study influence maximization in general stochastic models, the observed cascades model, and the independent cascade (IC) model. For general stochastic models with only black-box…
Symmetry Reveals Layerwise Dynamics: How Transformers Perform In-Context Classification
Patrick Lutz, Themistoklis Haris, Arjun Chandra +2
Transformers can perform in-context classification from a few labeled examples, yet the inference-time algorithm remains opaque. We study multi-class linear classification in the h…
Noise Stability of Transformer Models
Themistoklis Haris, Zihan Zhang, Yuichi Yoshida
Understanding simplicity biases in deep learning offers a promising path toward developing reliable AI. A common metric for this, inspired by Boolean function analysis, is average…
Fast-MWEM: Private Data Release in Sublinear Time
Themistoklis Haris, Steve Choi, Mutiraj Laksanawisit
The Multiplicative Weights Exponential Mechanism (MWEM) is a fundamental iterative framework for private data analysis, with broad applications such as answering linear queries…
Efficient Algorithms for Adversarially Robust Approximate Nearest Neighbor Search
Alexandr Andoni, Themistoklis Haris, Esty Kelman +1
We study the Approximate Nearest Neighbor (ANN) problem under a powerful adaptive adversary that controls both the dataset and a sequence of queries. Primarily, for the high-di…
Estimating Hitting Times Locally At Scale
Themistoklis Haris, Fabian Spaeh, Spyros Dragazis +1
Hitting times provide a fundamental measure of distance in random processes, quantifying the expected number of steps for a random walk starting at node to reach node . They…
Compression Barriers for Autoregressive Transformers
Themistoklis Haris, Krzysztof Onak
A key limitation of autoregressive Transformers is the large memory needed at inference-time to cache all previous key-value (KV) embeddings. Prior works address this by compressin…
NN Attention Demystified: A Theoretical Exploration for Scalable Transformers
Themistoklis Haris
Despite their power, Transformers face challenges with long sequences due to the quadratic complexity of self-attention. To address this limitation, methods like -Nearest-Neighb…
Counting Simplices in Hypergraph Streams
Amit Chakrabarti, Themistoklis Haris
We consider the problem of space-efficiently estimating the number of simplices in a hypergraph stream. This is the most natural hypergraph generalization of the highly-studied pro…