280 citations · 1.1k across the 43 of their papers we have counts for
43 papers
Training Discrete Energy-Based Models with Energy Discrepancy
Tobias Schröder, Zijing Ou, Yingzhen Li +1
Training energy-based models (EBMs) on discrete spaces is challenging because sampling over such spaces can be difficult. We propose to train discrete EBMs with energy discrepancy…
Energy Discrepancies: A Score-Independent Loss for Energy-Based Models
Tobias Schröder, Zijing Ou, Jen Ning Lim +3
Energy-based models are a simple yet powerful class of probabilistic models, but their widespread adoption has been limited by the computational burden of training them. We propose…
On the Identifiability of Switching Dynamical Systems
Carles Balsells-Rodas, Yixin Wang, Yingzhen Li
The identifiability of latent variable models has received increasing attention due to its relevance in interpretability and out-of-distribution generalisation. In this work, we st…
ESD: Expected Squared Difference as a Tuning-Free Trainable Calibration Measure
Hee Suk Yoon, Joshua Tian Jin Tee, Eunseop Yoon +4
Studies have shown that modern neural networks tend to be poorly calibrated due to over-confident predictions. Traditionally, post-processing methods have been used to calibrate th…
Calibrating Transformers via Sparse Gaussian Processes
Wenlong Chen, Yingzhen Li
Transformer models have achieved profound success in prediction tasks in a wide range of applications in natural language processing, speech recognition and computer vision. Extend…
Scalable Infomin Learning
Yanzhi Chen, Weihao Sun, Yingzhen Li +1
The task of infomin learning aims to learn a representation with high utility while being uninformative about a specified target, with the latter achieved by minimising the mutual…
Markovian Gaussian Process Variational Autoencoders
Harrison Zhu, Carles Balsells Rodas, Yingzhen Li
Sequential VAEs have been successfully considered for many high-dimensional time series modelling problems, with many variant models relying on discrete-time mechanisms such as rec…
Repairing Neural Networks by Leaving the Right Past Behind
Ryutaro Tanno, Melanie F. Pradier, Aditya Nori +1
Prediction failures of machine learning models often arise from deficiencies in training data, such as incorrect labels, outliers, and selection biases. However, such data points t…
Learning Neural Set Functions Under the Optimal Subset Oracle
Zijing Ou, Tingyang Xu, Qinliang Su +3
Learning neural set functions becomes increasingly more important in many applications like product recommendation and compound selection in AI-aided drug discovery. The majority o…
A Principled Approach to Failure Analysis and Model Repairment: Demonstration in Medical Imaging
Thomas Henn, Yasukazu Sakamoto, Clément Jacquet +8
Machine learning models commonly exhibit unexpected failures post-deployment due to either data shifts or uncommon situations in the training environment. Domain experts typically…
Interpreting diffusion score matching using normalizing flow
Wenbo Gong, Yingzhen Li
Scoring matching (SM), and its related counterpart, Stein discrepancy (SD) have achieved great success in model training and evaluations. However, recent research shows their limit…
Sparse Uncertainty Representation in Deep Learning with Inducing Weights
Hippolyt Ritter, Martin Kukla, Cheng Zhang +1
Bayesian neural networks and deep ensembles represent two modern paradigms of uncertainty quantification in deep learning. Yet these approaches struggle to scale mainly due to memo…
Contextual HyperNetworks for Novel Feature Adaptation
Angus Lamb, Evgeny Saveliev, Yingzhen Li +7
While deep learning has obtained state-of-the-art results in many applications, the adaptation of neural network architectures to incorporate new output features remains a challeng…
Active Slices for Sliced Stein Discrepancy
Wenbo Gong, Kaibo Zhang, Yingzhen Li +1
Sliced Stein discrepancy (SSD) and its kernelized variants have demonstrated promising successes in goodness-of-fit tests and model learning in high dimensions. Despite their theor…
Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification
Yi Zhu, Ehsan Shareghi, Yingzhen Li +2
Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP. Nonetheless, th…
Reinforcement Learning with Efficient Active Feature Acquisition
Haiyan Yin, Yingzhen Li, Sinno Jialin Pan +2
Solving real-life sequential decision making problems under partial observability involves an exploration-exploitation problem. To be successful, an agent needs to efficiently gath…
A Study on Efficiency in Continual Learning Inspired by Human Learning
Philip J. Ball, Yingzhen Li, Angus Lamb +1
Humans are efficient continual learning systems; we continually learn new skills from birth with finite cells and resources. Our learning is highly optimized both in terms of capac…
Learning Sparse Sentence Encoding without Supervision: An Exploration of Sparsity in Variational Autoencoders
Victor Prokhorov, Yingzhen Li, Ehsan Shareghi +1
It has been long known that sparsity is an effective inductive bias for learning efficient representation of data in vectors with fixed dimensionality, and it has been explored in…
Interpreting Spatially Infinite Generative Models
Chaochao Lu, Richard E. Turner, Yingzhen Li +1
Traditional deep generative models of images and other spatial modalities can only generate fixed sized outputs. The generated images have exactly the same resolution as the traini…
Meta-Learning Divergences of Variational Inference
Ruqi Zhang, Yingzhen Li, Christopher De Sa +2
Variational inference (VI) plays an essential role in approximate Bayesian inference due to its computational efficiency and broad applicability. Crucial to the performance of VI i…
Sliced Kernelized Stein Discrepancy
Wenbo Gong, Yingzhen Li, José Miguel Hernández-Lobato
Kernelized Stein discrepancy (KSD), though being extensively used in goodness-of-fit tests and model learning, suffers from the curse-of-dimensionality. We address this issue by pr…
A Causal View on Robustness of Neural Networks
Cheng Zhang, Kun Zhang, Yingzhen Li
We present a causal view on the robustness of neural networks against input manipulations, which applies not only to traditional classification tasks but also to general measuremen…
Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data
Sebastian Lunz, Yingzhen Li, Andrew Fitzgibbon +1
Recent work has shown the ability to learn generative models for 3D shapes from only unstructured 2D images. However, training such models requires differentiating through the rast…
Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
Maximilian Igl, Kamil Ciosek, Yingzhen Li +4
The ability for policies to generalize to new environments is key to the broad application of RL agents. A promising approach to prevent an agent's policy from overfitting to a lim…