6 papers
Spectral Outliers Reveal Dominant Learned Structure in Transformer Attention
Kasun Dewage, Marianna Pensky, Suranadi De Silva +1
We apply Marchenko-Pastur (MP) random matrix theory to pre-trained attention weights in order to separate each projection matrix into a random-like bulk and a set of spectral outli…
KRAFTY: Khatri-Rao Framework for Joint Cluster Recovery
Siyi Gao, Zachary Lubberts, Marianna Pensky
When multiple datasets describe complementary information about the same set of entities, for example, brain scans of an individual over time, global trade network across years, or…
LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights
Kasun Dewage, Marianna Pensky, Suranadi De Silva +1
We introduce LoRA-CRAFT (\textbf{C}ross-layer \textbf{R}ank \textbf{A}daptation via \textbf{F}rozen \textbf{T}ucker), abbreviated CRAFT throughout, an extremely parameter-efficient…
Perfect Clustering in Very Sparse Diverse Multiplex Networks
Marianna Pensky
The paper studies the DIverse MultiPLEx Signed Generalized Random Dot Product Graph (DIMPLE-SGRDPG) network model (Pensky (2024)), where all layers of the network have the same col…
Scalable community detection in massive networks via predictive assignment
Subhankar Bhadra, Marianna Pensky, Srijan Sengupta
Massive network datasets are becoming increasingly common in scientific applications. Existing community detection methods encounter significant computational challenges for such m…
Davis-Kahan Theorem in the two-to-infinity norm and its application to perfect clustering
Marianna Pensky
Many statistical applications, such as the Principal Component Analysis, matrix completion, tensor regression and many others, rely on accurate estimation of leading eigenvectors o…