18 citations · 63 across the 11 of their papers we have counts for
12 papers
How Does Information Bottleneck Help Deep Learning?
Kenji Kawaguchi, Zhun Deng, Xu Ji +1
Numerous deep learning algorithms have been inspired by and understood via the notion of information bottleneck, where unnecessary information is (often implicitly) minimized while…
Sources of Richness and Ineffability for Phenomenally Conscious States
Xu Ji, Eric Elmoznino, George Deane +5
Conscious states (states that there is something it is like to be in) seem both rich or full of detail, and ineffable or hard to fully describe or recall. The problem of ineffabili…
GFlowOut: Dropout with Generative Flow Networks
Dianbo Liu, Moksh Jain, Bonaventure Dossou +10
Bayesian Inference offers principled tools to tackle many critical problems with modern neural networks such as poor calibration and generalization, and data inefficiency. However,…
GFlowNets and variational inference
Esmeralda S. Whitammer, Salem Lahlou, Tristan Deleu +5
This paper builds bridges between two families of probabilistic algorithms: (hierarchical) variational inference (VI), which is typically used to model distributions over continuou…
Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization
Dianbo Liu, Alex Lamb, Xu Ji +4
Vector Quantization (VQ) is a method for discretizing latent representations and has become a major part of the deep learning toolkit. It has been theoretically and empirically sho…
Properties of Minimizing Entropy
Xu Ji, Lena Nehale-Ezzine, Maksym Korablyov
Compact data representations are one approach for improving generalization of learned functions. We explicitly illustrate the relationship between entropy and cardinality, both mea…
Test Sample Accuracy Scales with Training Sample Density in Neural Networks
Xu Ji, Razvan Pascanu, Devon Hjelm +2
Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to correlate with how densely the samples are surrounded by seen training samples in…
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
Xu Ji, Joao Henriques, Tinne Tuytelaars +1
Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a me…
There and Back Again: Revisiting Backpropagation Saliency Methods
Sylvestre-Alvise Rebuffi, Ruth Fong, Xu Ji +1
Saliency methods seek to explain the predictions of a model by producing an importance map across each input sample. A popular class of such methods is based on backpropagating a s…
NormGrad: Finding the Pixels that Matter for Training
Sylvestre-Alvise Rebuffi, Ruth Fong, Xu Ji +2
The different families of saliency methods, either based on contrastive signals, closed-form formulas mixing gradients with activations or on perturbation masks, all focus on which…
On the role of neurogenesis in overcoming catastrophic forgetting
German I. Parisi, Xu Ji, Stefan Wermter
Lifelong learning capabilities are crucial for artificial autonomous agents operating on real-world data, which is typically non-stationary and temporally correlated. In this work,…
Invariant Information Clustering for Unsupervised Image Classification and Segmentation
Xu Ji, João F. Henriques, Andrea Vedaldi
We present a novel clustering objective that learns a neural network classifier from scratch, given only unlabelled data samples. The model discovers clusters that accurately match…