4 citations · 9 across the 13 of their papers we have counts for
17 papers
DStill: Trajectory Consistency Distillation for Accelerated One-Step Drafting in Block Diffusion Speculative Decoding
Shreyas Kulkarni, Uttam Gurram, Madhav Kanda +5
Block-diffusion speculative decoding accelerates large language model inference by drafting multiple tokens in parallel. However, one-step drafting in discrete diffusion is limited…
Representation of syntax in LLMs through the lens of linear distance and similarity-aware entropy
Juan Pablo Vigneaux, Mary Kennedy, Khalil Iskarous +2
Structural probes were introduced by Hewitt and Manning to reconstruct syntactic trees from a neural language model's latent representations. They are evaluated by calculating the…
Probing neural audio codecs for distinctions among English nuclear tunes
Juan Pablo Vigneaux, Jennifer Cole
State-of-the-art spoken dialogue models (Défossez et al. 2024; Schalkwyk et al. 2025) use neural audio codecs to "tokenize" audio signals into a lower-frequency stream of vectorial…
Entropies associated with orbits of finite groups
Ryan Leal, Jingtong Sun, Juan Pablo Vigneaux
For many groups, parabolic subgroups arise as stabilizers of flags of sets or vector spaces; the quotients by these subgroups parametrize orbits of flags, and entropies appear as r…
The magnitude of categories of texts enriched by language models
Tai-Danae Bradley, Juan Pablo Vigneaux
The purpose of this article is twofold. Firstly, we use the next-token probabilities given by a language model to explicitly define a category of texts in natural language enriched…
A combinatorial approach to categorical Möbius inversion and pseudoinversion
Juan Pablo Vigneaux
We use Cramer's formula for the inverse of a matrix and a combinatorial expression for the determinant in terms of paths of an associated digraph (which can be traced back to Coate…
On the entropy of rectifiable and stratified measures
Juan Pablo Vigneaux
We summarize some results of geometric measure theory concerning rectifiable sets and measures. Combined with the entropic chain rule for disintegrations (Vigneaux, 2021), they acc…
A formula for the categorical magnitude in terms of the Moore-Penrose pseudoinverse
Stephanie Chen, Juan Pablo Vigneaux
The magnitude of finite categories is a generalization of the Euler characteristic. It is defined using the coarse incidence algebra of rational-valued functions on the given finit…
Categorical magnitude and entropy
Stephanie Chen, Juan Pablo Vigneaux
Given any finite set equipped with a probability measure, one may compute its Shannon entropy or information content. The entropy becomes the logarithm of the cardinality of the se…
Typicality for stratified measures
Juan Pablo Vigneaux
Stratified measures on Euclidean space are defined here as convex combinations of rectifiable measures. They are possibly singular with respect to the Lebesgue measure and generali…
Information cohomology of classical vector-valued observables
Juan Pablo Vigneaux
We provide here a novel algebraic characterization of two information measures associated with a vector-valued random variable, its differential entropy and the dimension of the un…
Entropy under disintegrations
Juan Pablo Vigneaux
We consider the differential entropy of probability measures absolutely continuous with respect to a given -finite reference measure on an arbitrary measurable space. We state t…
Extra-fine sheaves and interaction decompositions
Daniel Bennequin, Olivier Peltre, Grégoire Sergeant-Perthuis +1
We introduce an original notion of (upper) extra-fine sheaf on a topological space, for which Čech cohomology in strictly positive degree vanishes. We apply this notion when the to…
A homological characterization of generalized multinomial coefficients related to the entropic chain rule
Juan Pablo Vigneaux
There is an asymptotic relationship between the multiplicative relations among multinomial coefficients and the (additive) recurrence property of Shannon entropy known as the chain…
A functional equation related to generalized entropies and the modular group
Daniel Bennequin, Juan Pablo Vigneaux
We solve a functional equation connected to the algebraic characterization of generalized information functions. To prove the symmetry of the solution, we study a related system of…
Information theory with finite vector spaces
Juan Pablo Vigneaux
Whereas Shannon entropy is related to the growth rate of multinomial coefficients, we show that the quadratic entropy (Tsallis 2-entropy) is connected to their -deformation; whe…
Information structures and their cohomology
Juan Pablo Vigneaux
We introduce the category of information structures, whose objects are suitable diagrams of measurable sets that encode the possible outputs of a given family of observables and th…