4 papers
Move by Move: Measuring and Steering How LLMs Conduct Psychotherapy
Afonso Baldo, Hugo Pitorro, Areti Vassilopoulos +5
As the demand for mental health services outpaces clinical workforce capacity, users are increasingly turning to large language models (LLMs) for accessible and private support. Al…
AdaSplash-2: Faster Differentiable Sparse Attention
Nuno Gonçalves, Hugo Pitorro, Vlad Niculae +4
Sparse attention has been proposed as a way to alleviate the quadratic cost of transformers, a central bottleneck in long-context training. A promising line of work is -entmax a…
LaTIM: Measuring Latent Token-to-Token Interactions in Mamba Models
Hugo Pitorro, Marcos Treviso
State space models (SSMs), such as Mamba, have emerged as an efficient alternative to transformers for long-context sequence modeling. However, despite their growing adoption, SSMs…
How Effective are State Space Models for Machine Translation?
Hugo Pitorro, Pavlo Vasylenko, Marcos Treviso +1
Transformers are the current architecture of choice for NLP, but their attention layers do not scale well to long contexts. Recent works propose to replace attention with linear re…