4 papers
Cross-Examiner: Evaluating Consistency of Large Language Model-Generated Explanations
Danielle Villa, Maria Chang, Keerthiram Murugesan +2
Large Language Models (LLMs) are often asked to explain their outputs to enhance accuracy and transparency. However, evidence suggests that these explanations can misrepresent the…
Few-shot Policy (de)composition in Conversational Question Answering
Kyle Erwin, Guy Axelrod, Maria Chang +8
The task of policy compliance detection (PCD) is to determine if a scenario is in compliance with respect to a set of written policies. In a conversational setting, the results of…
Reasoning about concepts with LLMs: Inconsistencies abound
Rosario Uceda-Sosa, Karthikeyan Natesan Ramamurthy, Maria Chang +1
The ability to summarize and organize knowledge into abstract concepts is key to learning and reasoning. Many industrial applications rely on the consistent and systematic use of c…
Alignment Studio: Aligning Large Language Models to Particular Contextual Regulations
Swapnaja Achintalwar, Ioana Baldini, Djallel Bouneffouf +16
The alignment of large language models is usually done by model providers to add or control behaviors that are common or universally understood across use cases and contexts. In co…