49 citations · 241 across the 42 of their papers we have counts for
22 papers · 1 filter
Revisiting the Robustness of the Minimum Error Entropy Criterion: A Transfer Learning Case Study
Luis Pedro Silvestrin, Shujian Yu, Mark Hoogendoorn
Coping with distributional shifts is an important part of transfer learning methods in order to perform well in real-life tasks. However, most of the existing approaches in this ar…
Causal Recurrent Variational Autoencoder for Medical Time Series Generation
Hongming Li, Shujian Yu, Jose Principe
We propose causal recurrent variational autoencoder (CR-VAE), a novel generative model that is able to learn a Granger causal graph from a multivariate time series x and incorporat…
The Conditional Cauchy-Schwarz Divergence with Applications to Time-Series Data and Sequential Decision Making
Shujian Yu, Hongming Li, Sigurd Løkse +2
The Cauchy-Schwarz (CS) divergence was developed by Príncipe et al. in 2000. In this paper, we extend the classic CS divergence to quantify the closeness between two conditional di…
Robust and Fast Measure of Information via Low-rank Representation
Yuxin Dong, Tieliang Gong, Shujian Yu +2
The matrix-based Rényi's entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution. This i…
Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval
Yufeng Shi, Shujian Yu, Duanquan Xu +1
Zero-shot cross-modal retrieval (ZS-CMR) deals with the retrieval problem among heterogenous data from unseen classes. Typically, to guarantee generalization, the pre-defined class…
Principle of Relevant Information for Graph Sparsification
Shujian Yu, Francesco Alesiani, Wenzhe Yin +2
Graph sparsification aims to reduce the number of edges of a graph while maintaining its structural properties. In this paper, we propose the first general and effective informatio…
Multi-view Information Bottleneck Without Variational Approximation
Qi Zhang, Shujian Yu, Jingmin Xin +1
By "intelligently" fusing the complementary information across different views, multi-view learning is able to improve the performance of classification tasks. In this work, we ext…
Gated Information Bottleneck for Generalization in Sequential Environments
Francesco Alesiani, Shujian Yu, Xi Yu
Deep neural networks suffer from poor generalization to unseen environments when the underlying data distribution is different from that in the training set. By learning minimum su…
Information Theoretic Structured Generative Modeling
Bo Hu, Shujian Yu, Jose C. Principe
Rényi's information provides a theoretical foundation for tractable and data-efficient non-parametric density estimation, based on pair-wise evaluations in a reproducing kernel Hil…
Learning to Transfer with von Neumann Conditional Divergence
Ammar Shaker, Shujian Yu, Daniel Oñoro-Rubio
The similarity of feature representations plays a pivotal role in the success of problems related to domain adaptation. Feature similarity includes both the invariance of marginal…
Deep Deterministic Information Bottleneck with Matrix-based Entropy Functional
Xi Yu, Shujian Yu, Jose C. Principe
We introduce the matrix-based Renyi's -order entropy functional to parameterize Tishby et al. information bottleneck (IB) principle with a neural network. We term our methodolog…
Measuring Dependence with Matrix-based Entropy Functional
Shujian Yu, Francesco Alesiani, Xi Yu +2
Measuring the dependence of data plays a central role in statistics and machine learning. In this work, we summarize and generalize the main idea of existing information-theoretic…
Bilevel Continual Learning
Ammar Shaker, Francesco Alesiani, Shujian Yu +1
Continual learning (CL) studies the problem of learning a sequence of tasks, one at a time, such that the learning of each new task does not lead to the deterioration in performanc…
Modular-Relatedness for Continual Learning
Ammar Shaker, Shujian Yu, Francesco Alesiani
In this paper, we propose a continual learning (CL) technique that is beneficial to sequential task learners by improving their retained accuracy and reducing catastrophic forgetti…
Learning an Interpretable Graph Structure in Multi-Task Learning
Shujian Yu, Francesco Alesiani, Ammar Shaker +1
We present a novel methodology to jointly perform multi-task learning and infer intrinsic relationship among tasks by an interpretable and sparse graph. Unlike existing multi-task…
Towards Interpretable Multi-Task Learning Using Bilevel Programming
Francesco Alesiani, Shujian Yu, Ammar Shaker +1
Interpretable Multi-Task Learning can be expressed as learning a sparse graph of the task relationship based on the prediction performance of the learned models. Since many natural…
PRI-VAE: Principle-of-Relevant-Information Variational Autoencoders
Yanjun Li, Shujian Yu, Jose C. Principe +2
Although substantial efforts have been made to learn disentangled representations under the variational autoencoder (VAE) framework, the fundamental properties to the dynamics of l…
Multiscale Principle of Relevant Information for Hyperspectral Image Classification
Yantao Wei, Shujian Yu, Luis Sanchez Giraldo +1
This paper proposes a novel architecture, termed multiscale principle of relevant information (MPRI), to learn discriminative spectral-spatial features for hyperspectral image (HSI…
Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels
Shujian Yu, Xiaoyang Wang, Jose C. Principe
One important assumption underlying common classification models is the stationarity of the data. However, in real-world streaming applications, the data concept indicated by the j…
Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration
Shujian Yu, Kristoffer Wickstrøm, Robert Jenssen +1
The matrix-based Renyi's α-entropy functional and its multivariate extension were recently developed in terms of the normalized eigenspectrum of a Hermitian matrix of the projected…
Understanding Autoencoders with Information Theoretic Concepts
Shujian Yu, Jose C. Principe
Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks. In this paper, we illustrate an…
On Kernel Method-Based Connectionist Models and Supervised Deep Learning Without Backpropagation
Shiyu Duan, Shujian Yu, Yunmei Chen +1
We propose a novel family of connectionist models based on kernel machines and consider the problem of learning layer-by-layer a compositional hypothesis class, i.e., a feedforward…