4 citations · 7 across the 6 of their papers we have counts for
5 papers · 1 filter
Layer by Layer: Uncovering Hidden Representations in Language Models
Oscar Skean, Md Rifat Arefin, Dan Zhao +4
From extracting features to generating text, the outputs of large language models (LLMs) typically rely on the final layers, following the conventional wisdom that earlier layers c…
Does Representation Matter? Exploring Intermediate Layers in Large Language Models
Oscar Skean, Md Rifat Arefin, Yann LeCun +1
Understanding what defines a good representation in large language models (LLMs) is fundamental to both theoretical understanding and practical applications. In this paper, we inve…
FroSSL: Frobenius Norm Minimization for Efficient Multiview Self-Supervised Learning
Oscar Skean, Aayush Dhakal, Nathan Jacobs +1
Self-supervised learning (SSL) is a popular paradigm for representation learning. Recent multiview methods can be classified as sample-contrastive, dimension-contrastive, or asymme…
DiME: Maximizing Mutual Information by a Difference of Matrix-Based Entropies
Oscar Skean, Jhoan Keider Hoyos Osorio, Austin J. Brockmeier +1
We introduce an information-theoretic quantity with similar properties to mutual information that can be estimated from data without making explicit assumptions on the underlying d…
The Representation Jensen-Rényi Divergence
Jhoan Keider Hoyos Osorio, Oscar Skean, Austin J. Brockmeier +1
We introduce a divergence measure between data distributions based on operators in reproducing kernel Hilbert spaces defined by kernels. The empirical estimator of the divergence i…