780 citations · 1.8k across the 76 of their papers we have counts for
82 papers
Value of Information in the Mean-Square Case and its Application to the Analysis of Financial Time-Series Forecast
Roman Belavkin, Panos Pardalos, Jose Principe
The advances and development of various machine learning techniques has lead to practical solutions in various areas of science, engineering, medicine and finance. The great choice…
Universal Recurrent Event Memories for Streaming Data
Ran Dou, Jose Principe
In this paper, we propose a new event memory architecture (MemNet) for recurrent neural networks, which is universal for different types of time series data such as scalar, multiva…
Dynamic Analysis and an Eigen Initializer for Recurrent Neural Networks
Ran Dou, Jose Principe
In recurrent neural networks, learning long-term dependency is the main difficulty due to the vanishing and exploding gradient problem. Many researchers are dedicated to solving th…
Causal Recurrent Variational Autoencoder for Medical Time Series Generation
Hongming Li, Shujian Yu, Jose Principe
We propose causal recurrent variational autoencoder (CR-VAE), a novel generative model that is able to learn a Granger causal graph from a multivariate time series x and incorporat…
The Functional Wiener Filter
Benjamin Colburn, Luis G. Sanchez Giraldo, Jose C. Principe
This paper presents a close form solution in Reproducing Kernel Hilbert Space (RKHS) for the famed Wiener filter, which we called the functional Wiener filter(FWF). Instead of usin…
Adapting the Exploration Rate for Value-of-Information-Based Reinforcement Learning
Isaac J. Sledge, Jose C. Principe
In this paper, we consider the problem of adjusting the exploration rate when using value-of-information-based exploration. We do this by converting the value-of-information optimi…
The Normalized Cross Density Functional: A Framework to Quantify Statistical Dependence for Random Processes
Bo Hu, Jose C. Principe
This paper presents a novel approach to measuring statistical dependence between two random processes (r.p.) using a positive-definite function called the Normalized Cross Density…
Robust Dependence Measure using RKHS based Uncertainty Moments and Optimal Transport
Rishabh Singh, Jose C. Principe
Reliable measurement of dependence between variables is essential in many applications of statistics and machine learning. Current approaches for dependence estimation, especially…
Quantifying Model Uncertainty for Semantic Segmentation using Operators in the RKHS
Rishabh Singh, Jose C. Principe
Deep learning models for semantic segmentation are prone to poor performance in real-world applications due to the highly challenging nature of the task. Model uncertainty quantifi…
Principle of Relevant Information for Graph Sparsification
Shujian Yu, Francesco Alesiani, Wenzhe Yin +2
Graph sparsification aims to reduce the number of edges of a graph while maintaining its structural properties. In this paper, we propose the first general and effective informatio…
Hierarchical Linear Dynamical System for Representing Notes from Recorded Audio
Leila Kalantari, Jose Principe, Kathryn E. Sieving
We seek to develop simultaneous segmentation and classification of notes from audio recordings in presence of outliers. The selected architecture for modeling time series is hierar…
Deep Deterministic Independent Component Analysis for Hyperspectral Unmixing
Hongming Li, Shujian Yu, Jose C. Principe
We develop a new neural network based independent component analysis (ICA) method by directly minimizing the dependence amongst all extracted components. Using the matrix-based R{é…
Information Theoretic Structured Generative Modeling
Bo Hu, Shujian Yu, Jose C. Principe
Rényi's information provides a theoretical foundation for tractable and data-efficient non-parametric density estimation, based on pair-wise evaluations in a reproducing kernel Hil…
Estimating Rényi's -Cross-Entropies in a Matrix-Based Way
Isaac J. Sledge, Jose C. Principe
Conventional information-theoretic quantities assume access to probability distributions. Estimating such distributions is not trivial. Here, we consider function-based formulation…
Analysis of Intra-Operative Physiological Responses Through Complex Higher-Order SVD for Long-Term Post-Operative Pain Prediction
Raheleh Baharloo, Jose C. Principe, Parisa Rashidi +1
Long-term pain conditions after surgery and patients' responses to pain relief medications are not yet fully understood. While recent studies developed an index for nociception lev…
A Physics inspired Functional Operator for Model Uncertainty Quantification in the RKHS
Rishabh Singh, Jose C. Principe
Accurate uncertainty quantification of model predictions is a crucial problem in machine learning. Existing Bayesian methods, being highly iterative, are expensive to implement and…
Uncertainty quantification for multiclass data description
Leila Kalantari, Jose Principe, Kathryn E. Sieving
In this manuscript, we propose a multiclass data description model based on kernel Mahalanobis distance (MDD-KM) with self-adapting hyperparameter setting. MDD-KM provides uncertai…
External-Memory Networks for Low-Shot Learning of Targets in Forward-Looking-Sonar Imagery
Isaac J. Sledge, Christopher D. Toole, Joseph A. Maestri +1
We propose a memory-based framework for real-time, data-efficient target analysis in forward-looking-sonar (FLS) imagery. Our framework relies on first removing non-discriminative…
An Information-Theoretic Approach for Automatically Determining the Number of States when Aggregating Markov Chains
Isaac J. Sledge, Jose C. Principe
A fundamental problem when aggregating Markov chains is the specification of the number of state groups. Too few state groups may fail to sufficiently capture the pertinent dynamic…
Labels, Information, and Computation: Efficient Learning Using Sufficient Labels
Shiyu Duan, Spencer Chang, Jose C. Principe
In supervised learning, obtaining a large set of fully-labeled training data is expensive. We show that we do not always need full label information on every single training exampl…
A Kernel Framework to Quantify a Model's Local Predictive Uncertainty under Data Distributional Shifts
Rishabh Singh, Jose C. Principe
Traditional Bayesian approaches for model uncertainty quantification rely on notoriously difficult processes of marginalization over each network parameter to estimate its probabil…
Annotating Motion Primitives for Simplifying Action Search in Reinforcement Learning
Isaac J. Sledge, Darshan W. Bryner, Jose C. Principe
Reinforcement learning in large-scale environments is challenging due to the many possible actions that can be taken in specific situations. We have previously developed a means of…
Deep Deterministic Information Bottleneck with Matrix-based Entropy Functional
Xi Yu, Shujian Yu, Jose C. Principe
We introduce the matrix-based Renyi's -order entropy functional to parameterize Tishby et al. information bottleneck (IB) principle with a neural network. We term our methodolog…
Measuring Dependence with Matrix-based Entropy Functional
Shujian Yu, Francesco Alesiani, Xi Yu +2
Measuring the dependence of data plays a central role in statistics and machine learning. In this work, we summarize and generalize the main idea of existing information-theoretic…