1 citations · 1 across the 7 of their papers we have counts for
7 papers
Encoded Early, Used Late: Where Transformers Begin to Act on an Inferred Partner's Expertise
Mika Okamoto, Gabriele Sarti
A transformer can make an attribute linearly decodable in its residual stream at a depth where that attribute does not yet influence the output. This gap between where information…
Demystifying Entropy-based Selection for Chain-of-Thought Compression in Large Reasoning Models
Sara Candussio, Daniel Scalena, Luca Bortolussi +3
Entropy-based pruning has been proposed as an effective method for compressing Chain-of-Thought (CoT) reasoning with negligible accuracy loss. We test the robustness of low- and hi…
Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
Daniel Scalena, Sara Candussio, Luca Bortolussi +3
Chain-of-thought (CoT) reasoning is the dominant paradigm for inference-time scaling in language models, yet the causal influence of individual steps on the final answer remains po…
Predicting Future Behaviors in Reasoning Models Enables Better Steering
Evgenii Kortukov, Piotr Komorowski, Florian Klein +5
Deployed large reasoning models (LRMs) often behave unexpectedly. Test-time steering controls LRM outputs by intervening on their hidden representations, but it can degrade output…
Distilling Formal Logic into Neural Spaces: A Kernel Alignment Approach for Signal Temporal Logic
Sara Candussio, Gabriele Sarti, Gaia Saveri +1
We introduce a framework for learning continuous neural representations of formal specifications by distilling the geometry of their semantics into a latent space. Existing approac…
Interpreto: An Explainability Library for Transformers
Antonin Poché, Thomas Mullor, Gabriele Sarti +8
Interpreto is an open-source Python library for interpreting HuggingFace language models, from early BERT variants to LLMs. It provides two complementary families of methods: attri…
Bridging Logic and Learning: Decoding Temporal Logic Embeddings via Transformers
Sara Candussio, Gaia Saveri, Gabriele Sarti +1
Continuous representations of logic formulae allow us to integrate symbolic knowledge into data-driven learning algorithms. If such embeddings are semantically consistent, i.e. if…