12 citations · 15 across the 6 of their papers we have counts for
7 papers
The Logical Implication Steering Method for Conditional Interventions on Transformer Generation
Damjan Kalajdzievski
The field of mechanistic interpretability in pre-trained transformer models has demonstrated substantial evidence supporting the ''linear representation hypothesis'', which is the…
Scaling Laws for Forgetting When Fine-Tuning Large Language Models
Damjan Kalajdzievski
We study and quantify the problem of forgetting when fine-tuning pre-trained large language models (LLMs) on a downstream task. We find that parameter-efficient fine-tuning (PEFT)…
A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA
Damjan Kalajdzievski
As large language models (LLMs) have become increasingly compute and memory intensive, parameter-efficient fine-tuning (PEFT) methods are now a common strategy to fine-tune LLMs. A…
Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer
Damjan Kalajdzievski, Ximeng Mao, Pascal Fortier-Poisson +2
When presented with a data stream of two statistically dependent variables, predicting the future of one of the variables (the target stream) can benefit from information about bot…
The ultrafilter and almost disjointness numbers
Osvaldo Guzman, Damjan Kalajdzievski
We prove that every MAD family can be destroyed by a proper forcing that preserves -points. With this result, we prove that it is consistent that $ω_{1}=\mathfrak{u}<\mathfrak{a…
Forcing and Construction Schemes
Damjan Kalajdzievski, Fulgencio Lopez
We investigate forcing and independence questions relating to construction schemes. We show that adding Cohen reals adds a capturing construction scheme. We study the we…
Measurability Aspects of the Compactness Theorem for Sample Compression Schemes
Damjan Kalajdzievski
It was proved in 1998 by Ben-David and Litman that a concept space has a sample compression scheme of size d if and only if every finite subspace has a sample compression scheme of…