10 papers
Monitoring latent reasoning in looped language models
Pawel Olszowiec, Paulina Tomaszewska, Maciej Żelaszczyk +2
Looped language models (LMs) reuse one block of layers several times per token generation, promising more latent computation at no additional storage cost. Despite their rising pop…
The Ghost Couple: Correlated LLM Name Priors and Their Haunting of the Web and Academic Publishing
Michał Brzozowski, Neo Christopher Chung
These names do not exist. Elena Vasquez and Marcus Chen have appeared as volcano experts, astronauts, thriller protagonists, podcast hosts, and academic co-authors across hundreds…
Reading the Finetuning Prior: Verbatim Content Recovery via Contrastive Decoding Diffing
Michał Brzozowski, Zuzanna Dubanowska, Enrico Cassano +1
Narrowly finetuned language models memorize implanted content verbatim, but auditing what a deployed model has been taught, without access to its weights or training data, remains…
Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)
Michał Brzozowski, Neo Christopher Chung
Sparse autoencoders (SAEs) are one of the main methods to interpret the inner workings of deep neural networks (DNNs), decomposing activations into higher-dimensional features. How…
Semantic Adapter Routing with Fine-Tuning Task Embeddings
Enrico Cassano, Michał Brzozowski, Paolo Mandica +2
Parameter-efficient fine-tuning (PEFT) has led to model ecosystems in which a single backbone is paired with many task-specialized adapters. Given such a library, routing aims to s…
Ablating Archetypes: The Stability of Archetypal SAEs is an Artifact of Initialization and Metric Design
Michał Brzozowski, Neo Christopher Chung
Dictionary learning with sparse autoencoders (SAEs) produces overcomplete bases from neural network activations that are often interpretable and reduces polysemanticity. However, f…
GPart: End-to-End Isometric Fine-Tuning via Global Parameter Partitioning
Paolo Mandica, Michał Brzozowski, Zuzanna Dubanowska +1
Low-rank adaptation (LoRA) has become a dominant paradigm for parameter-efficient fine-tuning (PEFT) of large-scale deep learning models. However, its bilinear parameterization ind…
Regularizing Attention Scores with Bootstrapping
Neo Christopher Chung, Maxim Laletin
Vision transformers (ViT) rely on attention mechanism to weigh input features, and therefore attention scores have naturally been considered as explanations for its decision-making…
Explain and Monitor Deep Learning Models for Computer Vision using Obz AI
Neo Christopher Chung, Jakub Binda
Deep learning has transformed computer vision (CV), achieving outstanding performance in classification, segmentation, and related tasks. Such AI-based CV systems are becoming prev…
Safeguarding Generative AI Applications in Preclinical Imaging through Hybrid Anomaly Detection
Jakub Binda, Valentina Paneta, Vasileios Eleftheriadis +3
Generative AI holds great potentials to automate and enhance data synthesis in nuclear medicine. However, the high-stakes nature of biomedical imaging necessitates robust mechanism…