7 papers · 1 filter
Logic Before Language: Pre-pretraining on Formal Derivations Fosters Skill Acquisition and Compressibility
Jo-Ku Cheng, Nikolaos Aletras, Marco Valentino
Pre-pretraining language models (LMs) on symbolic data can accelerate and improve natural language acquisition. However, existing pre-pretraining tasks, such as Dyck and procedural…
Fundamental Reasoning Paradigms Induce Out-of-Domain Generalization in Language Models
Mingzi Cao, Xingwei Tan, Mahmud Elahi Akhter +4
Deduction, induction, and abduction are fundamental reasoning paradigms, core for human logical thinking. Although improving Large Language Model (LLM) reasoning has attracted sign…
Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Models
Gabriele Maraia, Marco Valentino, Fabio Massimo Zanzotto +1
Large Language Models (LLMs) often struggle with deductive judgment in syllogistic reasoning, systematically conflating semantic plausibility with formal validity a phenomenon know…
OpenSIR: Open-Ended Self-Improving Reasoner
Wai-Chung Kwan, Joshua Ong Jun Leang, Pavlos Vougiouklis +3
Recent advances in large language model (LLM) reasoning through reinforcement learning rely on annotated datasets for verifiable rewards, which may limit models' ability to surpass…
Adaptive LLM-Symbolic Reasoning via Dynamic Logical Solver Composition
Lei Xu, Pierre Beckmann, Marco Valentino +1
Neuro-symbolic NLP methods aim to leverage the complementary strengths of large language models and formal logical solvers. However, current approaches are mostly static in nature,…
Learning to Disentangle Latent Reasoning Rules with Language VAEs: A Systematic Study
Yingji Zhang, Marco Valentino, Danilo S. Carvalho +1
Incorporating explicit reasoning rules within the latent space of language models (LMs) offers a promising pathway to enhance generalisation, interpretability, and controllability.…
Enhancing Logical Reasoning in Language Models via Symbolically-Guided Monte Carlo Process Supervision
Xingwei Tan, Marco Valentino, Mahmud Akhter +2
Large language models (LLMs) have shown strong performance in many reasoning benchmarks. However, recent studies have pointed to memorization, rather than generalization, as one of…