In-context Learning and Gradient Descent Revisited

Gilad Deutch, Nadav Magar, Tomer Bar Natan +1

In-context learning (ICL) has shown impressive results in few-shot learning tasks, yet its underlying mechanism is still not fully understood. A recent line of work suggests that I…