910 citations · 2.1k across the 53 of their papers we have counts for
41 papers · 1 filter
A Dual-Perspective Approach to Evaluating Feature Attribution Methods
Yawei Li, Yang Zhang, Kenji Kawaguchi +3
Feature attribution methods attempt to explain neural network predictions by identifying relevant features. However, establishing a cohesive framework for assessing feature attribu…
Tackling the Curse of Dimensionality with Physics-Informed Neural Networks
Zheyuan Hu, Khemraj Shukla, George Em Karniadakis +1
The curse-of-dimensionality taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solvin…
IF2Net: Innately Forgetting-Free Networks for Continual Learning
Depeng Li, Tianqi Wang, Bingrong Xu +3
Continual learning can incrementally absorb new concepts without interfering with previously learned knowledge. Motivated by the characteristics of neural networks, in which inform…
Multi-View Class Incremental Learning
Depeng Li, Tianqi Wang, Junwei Chen +3
Multi-view learning (MVL) has gained great success in integrating information from multiple perspectives of a dataset to improve downstream task performance. To make MVL methods mo…
How Does Information Bottleneck Help Deep Learning?
Kenji Kawaguchi, Zhun Deng, Xu Ji +1
Numerous deep learning algorithms have been inspired by and understood via the notion of information bottleneck, where unnecessary information is (often implicitly) minimized while…
Last-Layer Fairness Fine-tuning is Simple and Effective for Neural Networks
Yuzhen Mao, Zhun Deng, Huaxiu Yao +3
As machine learning has been deployed ubiquitously across applications in modern data science, algorithmic fairness has become a great concern. Among them, imposing fairness constr…
D4FT: A Deep Learning Approach to Kohn-Sham Density Functional Theory
Tianbo Li, Min Lin, Zheyuan Hu +6
Kohn-Sham Density Functional Theory (KS-DFT) has been traditionally solved by the Self-Consistent Field (SCF) method. Behind the SCF loop is the physics intuition of solving a syst…
MixupE: Understanding and Improving Mixup from Directional Derivative Perspective
Yingtian Zou, Vikas Verma, Sarthak Mittal +6
Mixup is a popular data augmentation technique for training deep neural networks where additional samples are generated by linearly interpolating pairs of inputs and their labels.…
Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph
Haonan Wang, Jieyu Zhang, Qi Zhu +3
Existing graph contrastive learning (GCL) techniques typically require two forward passes for a single instance to construct the contrastive loss, which is effective for capturing…
Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology
Zheyuan Hu, Ameya D. Jagtap, George Em Karniadakis +1
In this paper, we propose the augmented physics-informed neural network (APINN), which adopts soft and trainable domain decomposition and flexible parameter sharing to further impr…
Neural Active Learning on Heteroskedastic Distributions
Savya Khosla, Chew Kin Whye, Jordan T. Ash +3
Models that can actively seek out the best quality training data hold the promise of more accurate, adaptable, and efficient machine learning. Active learning techniques often tend…
Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning
Riashat Islam, Hongyu Zang, Anirudh Goyal +6
Goal-conditioned reinforcement learning (RL) is a promising direction for training agents that are capable of solving multiple tasks and reach a diverse set of objectives. How to \…
MGNNI: Multiscale Graph Neural Networks with Implicit Layers
Juncheng Liu, Bryan Hooi, Kenji Kawaguchi +1
Recently, implicit graph neural networks (GNNs) have been proposed to capture long-range dependencies in underlying graphs. In this paper, we introduce and justify two weaknesses o…
GFlowOut: Dropout with Generative Flow Networks
Dianbo Liu, Moksh Jain, Bonaventure Dossou +10
Bayesian Inference offers principled tools to tackle many critical problems with modern neural networks such as poor calibration and generalization, and data inefficiency. However,…
Scalable Set Encoding with Universal Mini-Batch Consistency and Unbiased Full Set Gradient Approximation
Jeffrey Willette, Seanie Lee, Bruno Andreis +3
Recent work on mini-batch consistency (MBC) for set functions has brought attention to the need for sequentially processing and aggregating chunks of a partitioned set while guaran…
Discrete Key-Value Bottleneck
Frederik Träuble, Anirudh Goyal, Nasim Rahaman +4
Deep neural networks perform well on classification tasks where data streams are i.i.d. and labeled data is abundant. Challenges emerge with non-stationary training data streams su…
Set-based Meta-Interpolation for Few-Task Meta-Learning
Seanie Lee, Bruno Andreis, Kenji Kawaguchi +2
Meta-learning approaches enable machine learning systems to adapt to new tasks given few examples by leveraging knowledge from related tasks. However, a large number of meta-traini…
Simplicial Embeddings in Self-Supervised Learning and Downstream Classification
Samuel Lavoie, Christos Tsirigotis, Max Schwarzer +4
Simplicial Embeddings (SEM) are representations learned through self-supervised learning (SSL), wherein a representation is projected into simplices of dimensions each usin…
EIGNN: Efficient Infinite-Depth Graph Neural Networks
Juncheng Liu, Kenji Kawaguchi, Bryan Hooi +2
Graph neural networks (GNNs) are widely used for modelling graph-structured data in numerous applications. However, with their inherently finite aggregation layers, existing GNN mo…
Training Free Graph Neural Networks for Graph Matching
Zhiyuan Liu, Yixin Cao, Fuli Feng +4
We present a framework of Training Free Graph Matching (TFGM) to boost the performance of Graph Neural Networks (GNNs) based graph matching, providing a fast promising solution wit…
Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization
Dianbo Liu, Alex Lamb, Xu Ji +4
Vector Quantization (VQ) is a method for discretizing latent representations and has become a major part of the deep learning toolkit. It has been theoretically and empirically sho…
Multi-Task Learning as a Bargaining Game
Aviv Navon, Aviv Shamsian, Idan Achituve +4
In Multi-task learning (MTL), a joint model is trained to simultaneously make predictions for several tasks. Joint training reduces computation costs and improves data efficiency;…
ExpertNet: A Symbiosis of Classification and Clustering
Shivin Srivastava, Kenji Kawaguchi, Vaibhav Rajan
A widely used paradigm to improve the generalization performance of high-capacity neural models is through the addition of auxiliary unsupervised tasks during supervised training.…
Noether Networks: Meta-Learning Useful Conserved Quantities
Ferran Alet, Dylan Doblar, Allan Zhou +3
Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often explo…