2 papers
cs.LG2026
Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning
Joana Pasquali, Ramiro N. Barros, Arthur S. Bianchessi +7
LoRA is widely adopted for continual fine-tuning of Large Language Models due to its parameter efficiency, modularity across tasks, and compatibility with replay strategies. Howeve…
cs.CL2026
Bayesian Attention Mechanism: A Probabilistic Framework for Positional Encoding and Context Length Extrapolation
Arthur S. Bianchessi, Yasmin C. Aguirre, Rodrigo C. Barros +1
Transformer-based language models rely on positional encoding (PE) to handle token order and support context length extrapolation. However, existing PE methods lack theoretical cla…