Graphs for deep learning representations

Abstract

In recent years, Deep Learning methods have achieved state of the art performance in a vast range of machine learning tasks, including image classification and multilingual automatic text translation. These architectures are trained to solve machine learning tasks in an end-to-end fashion. In order to reach top-tier performance, these architectures often require a very large number of trainable parameters. There are multiple undesirable consequences, and in order to tackle these issues, it is desired to be able to open the black boxes of deep learning architectures. Problematically, doing so is difficult due to the high dimensionality of representations and the stochasticity of the training process. In this thesis, we investigate these architectures by introducing a graph formalism based on the recent advances in Graph Signal Processing (GSP). Namely, we use graphs to represent the latent spaces of deep neural networks. We showcase that this graph formalism allows us to answer various questions including: ensuring generalization abilities, reducing the amount of arbitrary choices in the design of the learning process, improving robustness to small perturbations added to the inputs, and reducing computational complexity

arXiv commentPhD. Thesis

References in corpus (35)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Community structure in social and biological networks
• Distilling the Knowledge in a Neural Network
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• Neural Message Passing for Quantum Chemistry
• Bootstrap your own latent: A new approach to self-supervised Learning
• Language Models are Few-Shot Learners
• In Defense of the Triplet Loss for Person Re-Identification
• Wide Residual Networks
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Deep Learning using Linear Support Vector Machines
• Understanding Deep Convolutional Networks
• Pitfalls of Graph Neural Network Evaluation
• Efficient Neural Architecture Search via Parameter Sharing
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Manifold Mixup: Better Representations by Interpolating Hidden States
• Connecting the Dots: Identifying Network Structure via Graph Signal Processing
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Deep Reinforcement Learning that Matters
• GSPBOX: A toolbox for signal processing on graphs
• Autoregressive Moving Average Graph Filtering
• Topology Adaptive Graph Convolutional Networks
• LanczosNet: Multi-Scale Deep Graph Convolutional Networks
• A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
• A Fair Comparison of Graph Neural Networks for Graph Classification
• Evaluating and Understanding the Robustness of Adversarial Logit Pairing
• Hyperbolic Graph Neural Networks
• Invariance-Preserving Localized Activation Functions for Graph Neural Networks
• Flow Smoothing and Denoising: Graph Signal Processing in the Edge-Space
• Certifiable Robustness to Graph Perturbations
• How far can we go without convolution: Improving fully-connected networks
• Improved Visual Localization via Graph Smoothing
• A Unified Deep Learning Formalism For Processing Graph Signals