168 citations · 421 across the 13 of their papers we have counts for
13 papers
When Are Solutions Connected in Deep Networks?
Quynh Nguyen, Pierre Brechet, Marco Mondelli
The question of how and why the phenomenon of mode connectivity occurs in training deep neural networks has gained remarkable attention in the research community. From a theoretica…
A Note on Connectivity of Sublevel Sets in Deep Learning
Quynh Nguyen
It is shown that for deep neural networks, a single wide layer of width ( being the number of training samples) suffices to prove the connectivity of sublevel sets of the…
On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths
Quynh Nguyen
We give a simple proof for the global convergence of gradient descent in training deep ReLU networks with the standard square loss, and show some of its improvements over the state…
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks
Quynh Nguyen, Marco Mondelli, Guido Montufar
A recent line of work has analyzed the theoretical properties of deep neural networks via the Neural Tangent Kernel (NTK). In particular, the smallest eigenvalue of the NTK has bee…
Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology
Quynh Nguyen, Marco Mondelli
Recent works have shown that gradient descent can find a global minimum for over-parameterized neural networks where the widths of all the hidden layers scale polynomially with …
On Connected Sublevel Sets in Deep Learning
Quynh Nguyen
This paper shows that every sublevel set of the loss function of a class of deep over-parameterized neural nets with piecewise linear activation functions is connected and unbounde…
On the loss landscape of a class of deep neural networks with no bad local valleys
Quynh Nguyen, Mahesh Chandra Mukkamala, Matthias Hein
We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that…
Neural Networks Should Be Wide Enough to Learn Disconnected Decision Regions
Quynh Nguyen, Mahesh Chandra Mukkamala, Matthias Hein
In the recent literature the important role of depth in deep learning has been emphasized. In this paper we argue that sufficient width of a feedforward network is equally importan…
Optimization Landscape and Expressivity of Deep CNNs
Quynh Nguyen, Matthias Hein
We analyze the loss landscape and expressiveness of practical deep convolutional neural networks (CNNs) with shared weights and max pooling layers. We show that such CNNs produce l…
The loss surface of deep and wide neural networks
Quynh Nguyen, Matthias Hein
While the optimization problem behind deep neural networks is highly non-convex, it is frequently observed in practice that training deep networks seems possible without getting st…
Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods
Antoine Gautier, Quynh Nguyen, Matthias Hein
The optimization problem behind neural networks is highly non-convex. Training with stochastic gradient descent and variants requires careful parameter tuning and provides no guara…
Latent Embeddings for Zero-shot Classification
Yongqin Xian, Zeynep Akata, Gaurav Sharma +3
We present a novel latent embedding model for learning a compatibility function between image and class embeddings, in the context of zero-shot classification. The proposed method…
An Efficient Multilinear Optimization Framework for Hypergraph Matching
Quynh Nguyen, Francesco Tudisco, Antoine Gautier +1
Hypergraph matching has recently become a popular approach for solving correspondence problems in computer vision as it allows to integrate higher-order geometric information. Hype…