4 papers
Diminishing Returns in Self-Supervised Learning
Oli Bridge, Huey Sun, Botond Branyicskai-Nagy +2
Transformer-based architectures have become a dominant paradigm in vision and language, but their success is often attributed to large model capacity and massive training data. In…
Enhancing Instruction-Following Capabilities in Seq2Seq Models: DoLA Adaptations for T5
Huey Sun, Anabel Yong, Lorenzo Gilly +1
Encoder-decoder models such as FLAN-T5 are finetuned to follow instructions, but often fail when the instructions conflict with memorized continuations ingrained during training. T…
Limits of Transfer Learning
Jake Williams, Abel Tadesse, Tyler Sam +2
Transfer learning involves taking information and insight from one problem domain and applying it to a new problem domain. Although widely used in practice, theory for transfer lea…
Decomposable Probability-of-Success Metrics in Algorithmic Search
Tyler Sam, Jake Williams, Abel Tadesse +2
Previous studies have used a specific success metric within an algorithmic search framework to prove machine learning impossibility results. However, this specific success metric p…