32 citations · 85 across the 50 of their papers we have counts for
38 papers · 1 filter
Is Inference Mediated by Distinct Semantic Structures in LLMs? A Mechanistic Interpretation
Nura Aljaafari, Marco Valentino, André Freitas
Predicting a label correctly does not necessarily require representing the operation that produces it. Transformer representations are known to carry label-level information, but w…
Compliance vs. Sensibility: On the Reasoning Controllability in Large Language Models
Xingwei Tan, Marco Valentino, Mahmud Elahi Akhter +3
Large Language Models (LLMs) acquire reasoning capabilities through shared inference patterns in pre-training data, which are further elicited via Chain-of-Thought (CoT). However,…
Decompose-and-Formalise: Recursively Verifiable Natural Language Inference
Xin Quan, Marco Valentino, Louise A. Dennis +1
Recent work has shown that integrating large language models (LLMs) with theorem provers (TPs) in neuro-symbolic pipelines helps with entailment verification and proof-guided refin…
Inferring Latent Intentions: Attributional Natural Language Inference in LLM Agents
Xin Quan, Jiafeng Xiong, Marco Valentino +1
Attributional inference, the ability to predict latent intentions behind observed actions, is a critical yet underexplored capability for large language models (LLMs) operating in…
Monotonic Reference-Free Refinement for Autoformalization
Lan Zhang, Marco Valentino, André Freitas
While statement autoformalization has advanced rapidly, full-theorem autoformalization remains largely unexplored. Existing iterative refinement methods in statement autoformalizat…
MASA: LLM-Driven Multi-Agent Systems for Autoformalization
Lan Zhang, Marco Valentino, André Freitas
Autoformalization serves a crucial role in connecting natural language and formal reasoning. This paper presents MASA, a novel framework for building multi-agent systems for autofo…
Dissecting Clinical Reasoning in Language Models: A Comparative Study of Prompts and Model Adaptation Strategies
Mael Jullien, Marco Valentino, Leonardo Ranaldi +1
Recent works on large language models (LLMs) have demonstrated the impact of prompting strategies and fine-tuning techniques on their reasoning capabilities. Yet, their effectivene…
Beyond Gold Standards: Epistemic Ensemble of LLM Judges for Formal Mathematical Reasoning
Lan Zhang, Marco Valentino, Jordan Meadows +1
Statement autoformalization plays a crucial role in formal mathematical reasoning by enabling the automatic translation of natural language statements into formal languages. While…
Faithful and Robust LLM-Driven Theorem Proving for NLI Explanations
Xin Quan, Marco Valentino, Louise A. Dennis +1
Natural language explanations play a fundamental role in Natural Language Inference (NLI) by revealing how premises logically entail hypotheses. Recent work has shown that the inte…
Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions
Leonardo Ranaldi, Marco Valentino, Andrè Freitas
Chain-of-Though (CoT) represents a common strategy for reasoning in Large Language Models (LLMs) by decomposing complex tasks into intermediate inference steps. However, explanatio…
Autoformalization in the Wild: Assessing LLMs on Real-World Mathematical Definitions
Lan Zhang, Marco Valentino, Andre Freitas
Thanks to their linguistic capabilities, LLMs offer an opportunity to bridge the gap between informal mathematics and formal languages through autoformalization. However, it is sti…
Eliciting Critical Reasoning in Retrieval-Augmented Language Models via Contrastive Explanations
Leonardo Ranaldi, Marco Valentino, Andrè Freitas
Retrieval-augmented generation (RAG) has emerged as a critical mechanism in contemporary NLP to support Large Language Models(LLMs) in systematically accessing richer factual conte…
SylloBio-NLI: Evaluating Large Language Models on Biomedical Syllogistic Reasoning
Magdalena Wysocka, Danilo Carvalho, Oskar Wysocki +2
Syllogistic reasoning is crucial for Natural Language Inference (NLI). This capability is particularly significant in specialized domains such as biomedicine, where it can support…
Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference
Geonhee Kim, Marco Valentino, André Freitas
Recent studies on reasoning in language models (LMs) have sparked a debate on whether they can learn systematic inferential principles or merely exploit superficial patterns in the…
Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Proving
Xin Quan, Marco Valentino, Louise A. Dennis +1
Natural language explanations represent a proxy for evaluating explanation-based and multi-step Natural Language Inference (NLI) models. However, assessing the validity of explanat…
SemEval-2024 Task 2: Safe Biomedical Natural Language Inference for Clinical Trials
Mael Jullien, Marco Valentino, André Freitas
Large Language Models (LLMs) are at the forefront of NLP achievements but fall short in dealing with shortcut learning, factual inconsistency, and vulnerability to adversarial inpu…
A Differentiable Integer Linear Programming Solver for Explanation-Based Natural Language Inference
Mokanarangan Thayaparan, Marco Valentino, André Freitas
Integer Linear Programming (ILP) has been proposed as a formalism for encoding precise structural and semantic constraints for Natural Language Inference (NLI). However, traditiona…
Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models
Julia Rozanova, Marco Valentino, André Freitas
Rigorous evaluation of the causal effects of semantic features on language model predictions can be hard to achieve for natural language reasoning problems. However, this is such a…
Inference to the Best Explanation in Large Language Models
Dhairya Dalal, Marco Valentino, André Freitas +1
While Large Language Models (LLMs) have found success in real-world applications, their underlying explanatory process is still poorly understood. This paper proposes IBE-Eval, a f…
Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement
Xin Quan, Marco Valentino, Louise A. Dennis +1
An increasing amount of research in Natural Language Inference (NLI) focuses on the application and evaluation of Large Language Models (LLMs) and their reasoning capabilities. Des…
Improving Semantic Control in Discrete Latent Spaces with Transformer Quantized Variational Autoencoders
Yingji Zhang, Danilo S. Carvalho, Marco Valentino +2
Achieving precise semantic control over the latent spaces of Variational AutoEncoders (VAEs) holds significant value for downstream tasks in NLP as the underlying generative mechan…
Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders
Yingji Zhang, Marco Valentino, Danilo S. Carvalho +2
The injection of syntactic information in Variational AutoEncoders (VAEs) has been shown to result in an overall improvement of performances and generalisation. An effective strate…
Controlling Equational Reasoning in Large Language Models with Prompt Interventions
Jordan Meadows, Marco Valentino, Andre Freitas
This paper investigates how hallucination rates in Large Language Models (LLMs) may be controlled via a symbolic data generation framework, exploring a fundamental relationship bet…
Estimating the Causal Effects of Natural Logic Features in Neural NLI Models
Julia Rozanova, Marco Valentino, Andre Freitas
Rigorous evaluation of the causal effects of semantic features on language model predictions can be hard to achieve for natural language reasoning problems. However, this is such a…