2 citations · 2 across the 2 of their papers we have counts for
5 papers
The Logical Implication Steering Method for Conditional Interventions on Transformer Generation
Damjan Kalajdzievski
The field of mechanistic interpretability in pre-trained transformer models has demonstrated substantial evidence supporting the ''linear representation hypothesis'', which is the…
Scaling Laws for Forgetting When Fine-Tuning Large Language Models
Damjan Kalajdzievski
We study and quantify the problem of forgetting when fine-tuning pre-trained large language models (LLMs) on a downstream task. We find that parameter-efficient fine-tuning (PEFT)…
A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA
Damjan Kalajdzievski
As large language models (LLMs) have become increasingly compute and memory intensive, parameter-efficient fine-tuning (PEFT) methods are now a common strategy to fine-tune LLMs. A…
The ultrafilter and almost disjointness numbers
Osvaldo Guzman, Damjan Kalajdzievski
We prove that every MAD family can be destroyed by a proper forcing that preserves -points. With this result, we prove that it is consistent that $ω_{1}=\mathfrak{u}<\mathfrak{a…
Forcing and Construction Schemes
Damjan Kalajdzievski, Fulgencio Lopez
We investigate forcing and independence questions relating to construction schemes. We show that adding Cohen reals adds a capturing construction scheme. We study the we…