331 citations · 614 across the 11 of their papers we have counts for
12 papers
TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings
Alexander Shabalin, Viacheslav Meshchaninov, Egor Chimbulatov +6
This paper presents the Text Encoding Diffusion Model (TEncDM), a novel approach to diffusion modeling that operates in the space of pre-trained language model encodings. In contra…
Star-Shaped Denoising Diffusion Probabilistic Models
Andrey Okhotin, Dmitry Molchanov, Vladimir Arkhipkin +4
Denoising Diffusion Probabilistic Models (DDPMs) provide the foundation for the recent breakthroughs in generative modeling. Their Markovian structure makes it difficult to define…
Reintroducing Straight-Through Estimators as Principled Methods for Stochastic Binary Networks
Alexander Shekhovtsov, Viktor Yanush
Training neural networks with binary weights and activations is a challenging problem due to the lack of gradients and difficulty of optimization over discrete weights. Many succes…
Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation
Dmitry Molchanov, Alexander Lyzhov, Yuliya Molchanova +2
Test-time data augmentationaveraging the predictions of a machine learning model across multiple augmented samples of datais a widely used technique that improves the predict…
Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning
Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov +1
Uncertainty estimation and ensembling methods go hand-in-hand. Uncertainty estimation is one of the main benchmarks for assessment of ensembling performance. At the same time, deep…
Variational Dropout via Empirical Bayes
Valery Kharitonov, Dmitry Molchanov, Dmitry Vetrov
We study the Automatic Relevance Determination procedure applied to deep neural networks. We show that ARD applied to Bayesian DNNs with Gaussian approximate posterior distribution…
Doubly Semi-Implicit Variational Inference
Dmitry Molchanov, Valery Kharitonov, Artem Sobolev +1
We extend the existing framework of semi-implicit variational inference (SIVI) and introduce doubly semi-implicit variational inference (DSIVI), a way to perform variational infere…
Bayesian Incremental Learning for Deep Neural Networks
Max Kochurov, Timur Garipov, Dmitry Podoprikhin +3
In industrial machine learning pipelines, data often arrive in parts. Particularly in the case of deep neural networks, it may be too expensive to train the model from scratch each…
Uncertainty Estimation via Stochastic Batch Normalization
Andrei Atanov, Arsenii Ashukha, Dmitry Molchanov +2
In this work, we investigate Batch Normalization technique and propose its probabilistic interpretation. We propose a probabilistic model and show that Batch Normalization maximaze…
Variance Networks: When Expectation Does Not Meet Your Expectations
Kirill Neklyudov, Dmitry Molchanov, Arsenii Ashukha +1
Ordinary stochastic neural networks mostly rely on the expected values of their weights to make predictions, whereas the induced noise is mostly used to capture the uncertainty, pr…
Structured Bayesian Pruning via Log-Normal Multiplicative Noise
Kirill Neklyudov, Dmitry Molchanov, Arsenii Ashukha +1
Dropout-based regularization methods can be regarded as injecting random noise with pre-defined magnitude to different parts of the neural network during training. It was recently…
Variational Dropout Sparsifies Deep Neural Networks
Dmitry Molchanov, Arsenii Ashukha, Dmitry Vetrov
We explore a recently proposed Variational Dropout technique that provided an elegant Bayesian interpretation to Gaussian Dropout. We extend Variational Dropout to the case when dr…