2 citations · 2 across the 3 of their papers we have counts for
4 papers · 1 filter
Task Priors: Enhancing Model Evaluation by Considering the Entire Space of Downstream Tasks
Niket Patel, Randall Balestriero
The grand goal of AI research, and particularly Self Supervised Learning (SSL), is to produce systems that can successfully solve any possible task. In contrast, current evaluation…
Layer by Layer: Uncovering Hidden Representations in Language Models
Oscar Skean, Md Rifat Arefin, Dan Zhao +4
From extracting features to generating text, the outputs of large language models (LLMs) typically rely on the final layers, following the conventional wisdom that earlier layers c…
On the Local Complexity of Linear Regions in Deep ReLU Networks
Niket Patel, Guido Montufar
We define the local complexity of a neural network with continuous piecewise linear activations as a measure of the density of linear regions over an input data distribution. We sh…
Learning to Compress: Local Rank and Information Compression in Deep Neural Networks
Niket Patel, Ravid Shwartz-Ziv
Deep neural networks tend to exhibit a bias toward low-rank solutions during training, implicitly learning low-dimensional feature representations. This paper investigates how deep…