2 citations · 3 across the 15 of their papers we have counts for
16 papers
Beyond Surface Forms: Symbolic Edits as a Test for Logical Reasoning with LLMs
Ramya Keerthy Thatikonda, Wray Buntine, Ehsan Shareghi
Logical reasoning with large language models (LLMs) is a critical capability, as it reflects a system's ability to correctly deduce hypotheses from a given context using faithful d…
GTS: Inference-Time Scaling of Latent Reasoning with a Learnable Gaussian Thought Sampler
Minghan Wang, Ye Bai, Thuy-Trang Vu +2
Inference-time scaling (ITS) in latent reasoning models typically relies on heuristic perturbations, such as dropout or fixed Gaussian noise, to generate diverse candidate trajecto…
Improving Symbolic Translation of Language Models for Logical Reasoning
Ramya Keerthy Thatikonda, Jiuzhou Han, Wray Buntine +1
The use of formal language for deductive logical reasoning aligns well with language models (LMs), where translating natural language (NL) into first-order logic (FOL) and employin…
Cube Bench: A Benchmark for Spatial Visual Reasoning in MLLMs
Dhruv Anand, Ehsan Shareghi
We introduce Cube Bench, a Rubik's-cube benchmark for evaluating spatial and sequential reasoning in multimodal large language models (MLLMs). The benchmark decomposes performance…
Towards Inference-time Scaling for Continuous Space Reasoning
Minghan Wang, Thuy-Trang Vu, Ehsan Shareghi +1
Inference-time scaling through multiple sample generation in combination with Process- or Outcome-Reward Model (PRM or ORM) re-ranking has proven effective for text-based reasoning…
Logical Reasoning with Outcome Reward Models for Test-Time Scaling
Ramya Keerthy Thatikonda, Wray Buntine, Ehsan Shareghi
Logical reasoning is a critical benchmark for evaluating the capabilities of large language models (LLMs), as it reflects their ability to derive valid conclusions from given premi…
Uncertainty-Based Methods for Automated Process Reward Data Construction and Output Aggregation in Mathematical Reasoning
Jiuzhou Han, Wray Buntine, Ehsan Shareghi
Large language models have demonstrated remarkable capabilities in complex mathematical reasoning tasks, but they inevitably generate errors throughout multi-step solutions. Proces…
TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law
Zheng Hui, Yijiang River Dong, Ehsan Shareghi +1
As large language models (LLMs) are increasingly deployed in high-risk domains such as law, finance, and medicine, systematically evaluating their domain-specific safety and compli…
VerifiAgent: a Unified Verification Agent in Language Model Reasoning
Jiuzhou Han, Wray Buntine, Ehsan Shareghi
Large language models demonstrate remarkable reasoning capabilities but often produce unreliable or incorrect responses. Existing verification methods are typically model-specific…
Assessing the Sensitivity and Alignment of FOL Closeness Metrics
Ramya Keerthy Thatikonda, Wray Buntine, Ehsan Shareghi
The recent successful paradigm of solving logical reasoning problems with tool-augmented large language models (LLMs) leverages translation of natural language (NL) statements into…
Evaluating LLM-based Approaches to Legal Citation Prediction: Domain-specific Pre-training, Fine-tuning, or RAG? A Benchmark and an Australian Law Case Study
Jiuzhou Han, Paul Burgess, Ehsan Shareghi
Large Language Models (LLMs) have demonstrated strong potential across legal tasks, yet the problem of legal citation prediction remains under-explored. At its core, this task dema…
One STEP at a time: Language Agents are Stepwise Planners
Minh Nguyen, Ehsan Shareghi
Language agents have shown promising adaptability in dynamic environments to perform complex tasks. However, despite the versatile knowledge embedded in large language models, thes…
Strategies for Improving NL-to-FOL Translation with LLMs: Data Generation, Incremental Fine-Tuning, and Verification
Ramya Keerthy Thatikonda, Jiuzhou Han, Wray Buntine +1
Logical reasoning is a fundamental task in natural language processing that presents significant challenges to Large Language Models (LLMs). The inherent characteristics of logical…
The Compressor-Retriever Architecture for Language Model OS
Yuan Yang, Siheng Xiong, Ehsan Shareghi +1
Recent advancements in large language models (LLMs) have significantly enhanced their capacity to aggregate and process information across multiple modalities, enabling them to per…
A Closer Look at Logical Reasoning with LLMs: The Choice of Tool Matters
Long Hei Matthew Lam, Ramya Keerthy Thatikonda, Ehsan Shareghi
The emergence of Large Language Models (LLMs) has demonstrated promising progress in solving logical reasoning tasks effectively. Several recent approaches have proposed to change…
Can LLMs Reason in the Wild with Programs?
Yuan Yang, Siheng Xiong, Ali Payani +2
Large Language Models (LLMs) have shown superior capability to solve reasoning problems with programs. While being a promising direction, most of such frameworks are trained and ev…