5 citations · 6 across the 2 of their papers we have counts for
5 papers
Phenomenology of Double Descent in Finite-Width Neural Networks
Sidak Pal Singh, Aurelien Lucchi, Thomas Hofmann +1
`Double descent' delineates the generalization behaviour of models depending on the regime they belong to: under- or over-parameterized. The current theoretical understanding behin…
Analytic Insights into Structure and Rank of Neural Network Hessian Maps
Sidak Pal Singh, Gregor Bachmann, Thomas Hofmann
The Hessian of a neural network captures parameter interactions through second-order derivatives of the loss. It is a fundamental object of study, closely tied to various problems…
WoodFisher: Efficient Second-Order Approximation for Neural Network Compression
Sidak Pal Singh, Dan Alistarh
Second-order information, in the form of Hessian- or Inverse-Hessian-vector products, is a fundamental tool for solving optimization problems. Recently, there has been significant…
GLOSS: Generative Latent Optimization of Sentence Representations
Sidak Pal Singh, Angela Fan, Michael Auli
We propose a method to learn unsupervised sentence representations in a non-compositional manner based on Generative Latent Optimization. Our approach does not impose any assumptio…
Context Mover's Distance & Barycenters: Optimal Transport of Contexts for Building Representations
Sidak Pal Singh, Andreas Hug, Aymeric Dieuleveut +1
We present a framework for building unsupervised representations of entities and their compositions, where each entity is viewed as a probability distribution rather than a vector…