1 citations · 1 across the 14 of their papers we have counts for
8 papers · 1 filter
Fast LapSum: Exact Differentiable Top- at Million Scale
Łukasz Struski, Joanna Wojciechowicz, Jakub Antczak +3
Selecting the top- elements is a fundamental operation for inducing sparsity in large-scale models and optimization problems, enabling robust expert activation, token routing or…
LaPrune: Controllable Differentiable Sparsity at Million Scale
Jakub Antczak, Joanna Wojciechowicz, Łukasz Struski +1
Top- selection determines which components of a sparse model remain active. Hard selection blocks gradients, while continuous relaxations often couple mask hardness to the selec…
Interpretability-Guided Soft Pruning of Attention Heads in Vision Transformers
Kamil Książek, Piotr Suszyński, Michał Jan Włodarczyk +2
Vision foundation models, such as DINOv2, learn highly expressive representations but rely on massive, opaque architectures that demand substantial computational power and memory.…
What Pixels Are Enough? SEAMS: Sufficiency Saliency via MSE-Preservation Soft-Masks
Magdalena Trędowicz, Łukasz Struski, Arkadiusz Lewicki +4
Saliency maps are most useful when they identify the image regions that are sufficient to preserve a model's behaviour. We introduce SEAMS, a sufficiency-based saliency method that…
SoftMoE: Soft Differentiable Routing for Mixture-of-Experts in LLMs
Mikołaj Zasada, Łukasz Struski, Jacek Tabor +1
Sparse Mixture-of-Experts (MoE) architectures enable scaling LLM parameters under a fixed inference budget by activating only a small subset of experts via top- routing. While t…
LAPLEX: The FFT of Learnable Laplace Kernels
Łukasz Struski, Hanna Blazhko, Piotr Kubaty +1
Fast linear algebra in deep learning usually comes with a choice: fixed geometry and exact computation, as in the Fourier transform, or adaptive geometry paid for by dense paramete…
Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models
Piotr Kubaty, Patryk Marszałek, Łukasz Struski +3
Vision-language models learn powerful multimodal embeddings, yet their internal semantics remain opaque. While sparse autoencoders (SAEs) can extract interpretable features, they r…
DAVE: Distribution-aware Attribution via ViT Gradient Decomposition
Adam Wróbel, Siddhartha Gairola, Jacek Tabor +3
Vision Transformers (ViTs) have become a dominant architecture in computer vision, yet producing stable and high-resolution attribution maps for these models remains challenging. A…