3 papers
cs.LG2024
Can Custom Models Learn In-Context? An Exploration of Hybrid Architecture Performance on In-Context Learning Tasks
Ryan Campbell, Nelson Lojo, Kesava Viswanadha +5
In-Context Learning (ICL) is a phenomenon where task learning occurs through a prompt sequence without the necessity of parameter updates. ICL in Multi-Headed Attention (MHA) with…
cs.LG2023
Automatic Curriculum Learning with Gradient Reward Signals
Ryan Campbell, Junsang Yoon
This paper investigates the impact of using gradient norm reward signals in the context of Automatic Curriculum Learning (ACL) for deep reinforcement learning (DRL). We introduce a…
cs.LG2023
Can Transformers Learn Sequential Function Classes In Context?
Ryan Campbell, Emma Guo, Evan Hu +2
In-context learning (ICL) has revolutionized the capabilities of transformer models in NLP. In our project, we extend the understanding of the mechanisms underpinning ICL by explor…