4 citations · 7 across the 11 of their papers we have counts for
5 papers · 1 filter
How Depth Growth and Looping Shape In-Context Learning in Language Models
Ferdinand Kapl, Kaitlin Maile, Stefan Bauer +1
Looped and depth-grown language models reuse computations across depth and have attracted interest for their reasoning capabilities. Building on prior suggestive evidence of improv…
Growing to See: Efficient Pretraining of Self-Supervised Vision Transformers
Amir Mohammad Karimi Mamaghan, Ferdinand Kapl, Andrea Dittadi +4
Progressive depth growth builds large networks by starting with shallow models and inserting layers during training. Recent advances in large language models have renewed interest…
Evolution With Purpose: Hierarchy-Informed Optimization of Whole-Brain Models
Hormoz Shahrzad, Niharika Gajawelli, Kaitlin Maile +2
Evolutionary search is well suited for large-scale biophysical brain modeling, where many parameters with nonlinear interactions and no tractable gradients need to be optimized. St…
From Growing to Looping: A Unified View of Iterative Computation in LLMs
Ferdinand Kapl, Emmanouil Angelis, Kaitlin Maile +2
Looping, reusing a block of layers across depth, and depth growing, training shallow-to-deep models by duplicating middle layers, have both been linked to stronger reasoning, but t…
From Words to Amino Acids: Does the Curse of Depth Persist?
Aleena Siji, Amir Mohammad Karimi Mamaghan, Ferdinand Kapl +9
Protein language models (PLMs) have become widely adopted as general-purpose models, demonstrating strong performance in protein engineering and de novo design. Like large language…