8 papers
Reflective Reasoning for SQL Generation
Isabelle Mohr, Joao Gandarela, John Dujany +1
Robust text-to-SQL over complex, real-world databases remains brittle even with modern LLMs: iterative refinement often introduces syntactic and semantic drift, corrections tend to…
Emergence and Localisation of Semantic Role Circuits in LLMs
Nura Aljaafari, Danilo S. Carvalho, André Freitas
Despite displaying semantic competence, large language models' internal mechanisms that ground abstract semantic structure remain insufficiently characterised. We propose a method…
elsciRL: Integrating Language Solutions into Reinforcement Learning Problem Settings
Philip Osborne, Danilo S. Carvalho, André Freitas
We present elsciRL, an open-source Python library to facilitate the application of language solutions on reinforcement learning problems. We demonstrate the potential of our softwa…
TRACE: Training and Inference-Time Interpretability Analysis for Language Models
Nura Aljaafari, Danilo S. Carvalho, André Freitas
Understanding when and how linguistic knowledge emerges during language model training remains a central challenge for interpretability. Most existing tools are post hoc, rely on s…
Learning to Disentangle Latent Reasoning Rules with Language VAEs: A Systematic Study
Yingji Zhang, Marco Valentino, Danilo S. Carvalho +1
Incorporating explicit reasoning rules within the latent space of language models (LMs) offers a promising pathway to enhance generalisation, interpretability, and controllability.…
TRACE for Tracking the Emergence of Semantic Representations in Transformers
Nura Aljaafari, Danilo S. Carvalho, André Freitas
Modern transformer models exhibit phase transitions during training, distinct shifts from memorisation to abstraction, but the mechanisms underlying these transitions remain poorly…