13 papers
Scaling Laws for Hypernetwork-Based Knowledge Injection in Large Language Models
Nischay Dhankhar, Dos Baha, Abulhair Saparov
Injecting factual knowledge into large language models (LLMs) reliably and at scale remains an open challenge. Hypernetworks provide a promising solution to large-scale knowledge i…
Learning to Reason Efficiently with A* Post-Training
Andreas Opedal, Francesco Ignazio Re, Abulhair Saparov +3
Many applications of large language models (LLMs) require deductive reasoning, yet models frequently produce incorrect or redundant inference steps. We frame natural language infer…
Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
Tianle Wang, Zhaoyang Wang, Guangchen Lan +4
Reinforcement learning (RL) has been applied to improve large language model (LLM) reasoning, yet the systematic study of how training scales with task difficulty has been hampered…
Language Models Might Not Understand You: Evaluating Theory of Mind via Story Prompting
Nathaniel Getachew, Abulhair Saparov
We introduce StorySim, a programmable framework for synthetically generating stories to evaluate the theory of mind (ToM) and world modeling (WM) capabilities of large language mod…
Transformers Can Learn Connectivity in Some Graphs but Not Others
Amit Roy, Abulhair Saparov
Reasoning capability is essential to ensure the factual correctness of the responses of transformer-based Large Language Models (LLMs), and robust reasoning about transitive relati…
Do Language Models Follow Occam's Razor? An Evaluation of Parsimony in Inductive and Abductive Reasoning
Yunxin Sun, Abulhair Saparov
Non-deductive reasoning, encompassing inductive and abductive reasoning, is essential in addressing complex real-world questions. One key feature of inductive and abductive reasoni…