780 citations · 1.8k across the 76 of their papers we have counts for
35 papers · 1 filter
Universal Recurrent Event Memories for Streaming Data
Ran Dou, Jose Principe
In this paper, we propose a new event memory architecture (MemNet) for recurrent neural networks, which is universal for different types of time series data such as scalar, multiva…
Dynamic Analysis and an Eigen Initializer for Recurrent Neural Networks
Ran Dou, Jose Principe
In recurrent neural networks, learning long-term dependency is the main difficulty due to the vanishing and exploding gradient problem. Many researchers are dedicated to solving th…
Causal Recurrent Variational Autoencoder for Medical Time Series Generation
Hongming Li, Shujian Yu, Jose Principe
We propose causal recurrent variational autoencoder (CR-VAE), a novel generative model that is able to learn a Granger causal graph from a multivariate time series x and incorporat…
Adapting the Exploration Rate for Value-of-Information-Based Reinforcement Learning
Isaac J. Sledge, Jose C. Principe
In this paper, we consider the problem of adjusting the exploration rate when using value-of-information-based exploration. We do this by converting the value-of-information optimi…
The Normalized Cross Density Functional: A Framework to Quantify Statistical Dependence for Random Processes
Bo Hu, Jose C. Principe
This paper presents a novel approach to measuring statistical dependence between two random processes (r.p.) using a positive-definite function called the Normalized Cross Density…
Principle of Relevant Information for Graph Sparsification
Shujian Yu, Francesco Alesiani, Wenzhe Yin +2
Graph sparsification aims to reduce the number of edges of a graph while maintaining its structural properties. In this paper, we propose the first general and effective informatio…
Information Theoretic Structured Generative Modeling
Bo Hu, Shujian Yu, Jose C. Principe
Rényi's information provides a theoretical foundation for tractable and data-efficient non-parametric density estimation, based on pair-wise evaluations in a reproducing kernel Hil…
A Physics inspired Functional Operator for Model Uncertainty Quantification in the RKHS
Rishabh Singh, Jose C. Principe
Accurate uncertainty quantification of model predictions is a crucial problem in machine learning. Existing Bayesian methods, being highly iterative, are expensive to implement and…
Uncertainty quantification for multiclass data description
Leila Kalantari, Jose Principe, Kathryn E. Sieving
In this manuscript, we propose a multiclass data description model based on kernel Mahalanobis distance (MDD-KM) with self-adapting hyperparameter setting. MDD-KM provides uncertai…
Labels, Information, and Computation: Efficient Learning Using Sufficient Labels
Shiyu Duan, Spencer Chang, Jose C. Principe
In supervised learning, obtaining a large set of fully-labeled training data is expensive. We show that we do not always need full label information on every single training exampl…
A Kernel Framework to Quantify a Model's Local Predictive Uncertainty under Data Distributional Shifts
Rishabh Singh, Jose C. Principe
Traditional Bayesian approaches for model uncertainty quantification rely on notoriously difficult processes of marginalization over each network parameter to estimate its probabil…
Annotating Motion Primitives for Simplifying Action Search in Reinforcement Learning
Isaac J. Sledge, Darshan W. Bryner, Jose C. Principe
Reinforcement learning in large-scale environments is challenging due to the many possible actions that can be taken in specific situations. We have previously developed a means of…
Deep Deterministic Information Bottleneck with Matrix-based Entropy Functional
Xi Yu, Shujian Yu, Jose C. Principe
We introduce the matrix-based Renyi's -order entropy functional to parameterize Tishby et al. information bottleneck (IB) principle with a neural network. We term our methodolog…
Measuring Dependence with Matrix-based Entropy Functional
Shujian Yu, Francesco Alesiani, Xi Yu +2
Measuring the dependence of data plays a central role in statistics and machine learning. In this work, we summarize and generalize the main idea of existing information-theoretic…
Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods
Shiyu Duan, Jose C. Principe
This tutorial paper surveys provably optimal alternatives to end-to-end backpropagation (E2EBP) -- the de facto standard for training deep architectures. Modular training refers to…
Unsupervised Foveal Vision Neural Networks with Top-Down Attention
Ryan Burt, Nina N. Thigpen, Andreas Keil +1
Deep learning architectures are an extremely powerful tool for recognizing and classifying images. However, they require supervised learning and normally work on vectors the size o…
PRI-VAE: Principle-of-Relevant-Information Variational Autoencoders
Yanjun Li, Shujian Yu, Jose C. Principe +2
Although substantial efforts have been made to learn disentangled representations under the variational autoencoder (VAE) framework, the fundamental properties to the dynamics of l…
Towards a Kernel based Uncertainty Decomposition Framework for Data and Models
Rishabh Singh, Jose C. Principe
This paper introduces a new framework for quantifying predictive uncertainty for both data and models that relies on projecting the data into a Gaussian reproducing kernel Hilbert…
Fast Estimation of Information Theoretic Learning Descriptors using Explicit Inner Product Spaces
Kan Li, Jose C. Principe
Kernel methods form a theoretically-grounded, powerful and versatile framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies o…
No-Trick (Treat) Kernel Adaptive Filtering using Deterministic Features
Kan Li, Jose C. Principe
Kernel methods form a powerful, versatile, and theoretically-grounded unifying framework to solve nonlinear problems in signal processing and machine learning. The standard approac…
Multiscale Principle of Relevant Information for Hyperspectral Image Classification
Yantao Wei, Shujian Yu, Luis Sanchez Giraldo +1
This paper proposes a novel architecture, termed multiscale principle of relevant information (MPRI), to learn discriminative spectral-spatial features for hyperspectral image (HSI…
An Exact Reformulation of Feature-Vector-based Radial-Basis-Function Networks for Graph-based Observations
Isaac J. Sledge, Jose C. Principe
Radial-basis-function networks are traditionally defined for sets of vector-based observations. In this short paper, we reformulate such networks so that they can be applied to adj…
Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels
Shujian Yu, Xiaoyang Wang, Jose C. Principe
One important assumption underlying common classification models is the stationarity of the data. However, in real-world streaming applications, the data concept indicated by the j…
A Taxonomy for Neural Memory Networks
Ying Ma, Jose Principe
In this paper, a taxonomy for memory networks is proposed based on their memory organization. The taxonomy includes all the popular memory networks: vanilla recurrent neural networ…