10 papers
Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability
Yash Aggarwal, Atmika Gorti, Vinija Jain +3
Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply "biased" or "…
Experiments or Outcomes? Probing Scientific Feasibility in Large Language Models
Seyedali Mohammadi, Manas Gaur, Francis Ferraro
Scientific feasibility assessment asks whether a claim is consistent with established knowledge and whether experimental evidence could support or refute it. We frame feasibility a…
NeuroSymbolic AI for Legal AI-TRISM: Trustworthy, Reliable, Interpretable, Safe Models
Deepa Tilwani, Yash Saxena, Ankur Padia +2
Large Language Models (LLMs) have transformed natural language processing, but their lack of interpretable reasoning and tendency to hallucinate pose significant challenges for leg…
Flying Pigs, FaR and Beyond: Evaluating LLM Reasoning in Counterfactual Worlds
Anish R Joishy, Ishwar B Balappanawar, Vamshi Krishna Bonagiri +3
A fundamental challenge in reasoning is navigating hypothetical, counterfactual worlds where logic may conflict with ingrained knowledge. We investigate this frontier for Large Lan…
Beyond Memorization: Testing LLM Reasoning on Unseen Theory of Computation Tasks
Shlok Shelat, Jay Raval, Souvik Roy +1
Large language models (LLMs) have demonstrated strong performance on formal language tasks, yet whether this reflects genuine symbolic reasoning or pattern matching on familiar con…
SymLoc: Symbolic Localization of Hallucination across HaluEval and TruthfulQA
Naveen Lamba, Sanju Tiwari, Manas Gaur
LLMs still struggle with hallucination, especially when confronted with symbolic triggers like modifiers, negation, numbers, exceptions, and named entities. Yet, we lack a clear un…