4 papers
DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation
Peyman Hosseini, Ondrej Bohdal, Ahmed Alajrami +6
Large Language Model (LLM)-based agents can solve complex procedural tasks by interacting with environments over multiple turns, but this ability typically depends on large models,…
CG-TTRL: Context-Guided Test-Time Reinforcement Learning for On-Device Large Language Models
Peyman Hosseini, Ondrej Bohdal, Taha Ceritli +4
Test-time Reinforcement Learning (TTRL) has shown promise in adapting foundation models for complex tasks at test-time, resulting in large performance improvements. TTRL leverages…
Cost-Effective Attention Mechanisms for Low Resource Settings: Necessity & Sufficiency of Linear Transformations
Peyman Hosseini, Mehran Hosseini, Ignacio Castro +1
From natural language processing to vision, Scaled Dot Product Attention (SDPA) is the backbone of most modern deep learning applications. Unfortunately, its memory and computation…
Efficient Solutions For An Intriguing Failure of LLMs: Long Context Window Does Not Mean LLMs Can Analyze Long Sequences Flawlessly
Peyman Hosseini, Ignacio Castro, Iacopo Ghinassi +1
Large Language Models (LLMs) have demonstrated remarkable capabilities in comprehending and analyzing lengthy sequential inputs, owing to their extensive context windows that allow…