18 citations · 63 across the 11 of their papers we have counts for
7 papers · 1 filter
How Does Information Bottleneck Help Deep Learning?
Kenji Kawaguchi, Zhun Deng, Xu Ji +1
Numerous deep learning algorithms have been inspired by and understood via the notion of information bottleneck, where unnecessary information is (often implicitly) minimized while…
GFlowOut: Dropout with Generative Flow Networks
Dianbo Liu, Moksh Jain, Bonaventure Dossou +10
Bayesian Inference offers principled tools to tackle many critical problems with modern neural networks such as poor calibration and generalization, and data inefficiency. However,…
GFlowNets and variational inference
Esmeralda S. Whitammer, Salem Lahlou, Tristan Deleu +5
This paper builds bridges between two families of probabilistic algorithms: (hierarchical) variational inference (VI), which is typically used to model distributions over continuou…
Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization
Dianbo Liu, Alex Lamb, Xu Ji +4
Vector Quantization (VQ) is a method for discretizing latent representations and has become a major part of the deep learning toolkit. It has been theoretically and empirically sho…
Properties of Minimizing Entropy
Xu Ji, Lena Nehale-Ezzine, Maksym Korablyov
Compact data representations are one approach for improving generalization of learned functions. We explicitly illustrate the relationship between entropy and cardinality, both mea…
Test Sample Accuracy Scales with Training Sample Density in Neural Networks
Xu Ji, Razvan Pascanu, Devon Hjelm +2
Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to correlate with how densely the samples are surrounded by seen training samples in…
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
Xu Ji, Joao Henriques, Tinne Tuytelaars +1
Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a me…