4 papers
On Surprising Effectiveness of Masking Updates in Adaptive Optimizers
Taejong Joo, Wenhan Xia, Cheolmin Kim +2
Training large language models (LLMs) relies almost exclusively on dense adaptive optimizers with increasingly sophisticated preconditioners. We challenge this by showing that rand…
Technical Debt in In-Context Learning: Diminishing Efficiency in Long Context
Taejong Joo, Diego Klabjan
Transformers have demonstrated remarkable in-context learning (ICL) capabilities, adapting to new tasks by simply conditioning on demonstrations without parameter updates. Compelli…
Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaboration
Taejong Joo, Shu Ishida, Ivan Sosnovik +4
As a model-agnostic approach to long context modeling, multi-agent systems can process inputs longer than a large language model's context window without retraining or architectura…
Improving self-training under distribution shifts via anchored confidence with theoretical guarantees
Taejong Joo, Diego Klabjan
Self-training often falls short under distribution shifts due to an increased discrepancy between prediction confidence and actual accuracy. This typically necessitates computation…