90 citations · 138 across the 31 of their papers we have counts for
34 papers
Thoughts without Thinking: Reconsidering the Explanatory Value of Chain-of-Thought Reasoning in LLMs through Agentic Pipelines
Ramesh Manuvinakurike, Emanuel Moss, Elizabeth Anne Watkins +3
Agentic pipelines present novel challenges and opportunities for human-centered explainability. The HCXAI community is still grappling with how best to make the inner workings of L…
AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons
Shaona Ghosh, Heather Frase, Adina Williams +99
The rapid advancement and deployment of AI systems have created an urgent need for standard safety-evaluation frameworks. This paper introduces AILuminate v1.0, the first comprehen…
QA-TOOLBOX: Conversational Question-Answering for process task guidance in manufacturing
Ramesh Manuvinakurike, Elizabeth Watkins, Celal Savur +7
In this work we explore utilizing LLMs for data augmentation for manufacturing task guidance system. The dataset consists of representative samples of interactions with technicians…
Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging
Hua Farn, Hsuan Su, Shachi H Kumar +3
Fine-tuning large language models (LLMs) for downstream tasks often leads to catastrophic forgetting, notably degrading the safety of originally aligned models. While some existing…
Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models
Shachi H Kumar, Saurav Sahay, Sahisnu Mazumder +6
Large Language Models (LLMs) have excelled at language understanding and generating human-level text. However, even with supervised training and human alignment, these LLMs are sus…
Introducing v0.5 of the AI Safety Benchmark from MLCommons
Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed +97
This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safe…
Zero-shot Conversational Summarization Evaluations with small Large Language Models
Ramesh Manuvinakurike, Saurav Sahay, Sangeeta Manepalli +1
Large Language Models (LLMs) exhibit powerful summarization abilities. However, their capabilities on conversational summarization remains under explored. In this work we evaluate…
Learning from Red Teaming: Gender Bias Provocation and Mitigation in Large Language Models
Hsuan Su, Cheng-Chu Cheng, Hua Farn +4
Recently, researchers have made considerable improvements in dialogue systems with the progress of large language models (LLMs) such as ChatGPT and GPT-4. These LLM-based chatbots…
Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home
Eda Okur, Roddy Fuentes Alba, Saurav Sahay +1
Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becomin…
Sample Efficient Multimodal Semantic Augmentation for Incremental Summarization
Sumanta Bhattacharyya, Ramesh Manuvinakurike, Sahisnu Mazumder +1
In this work, we develop a prompting approach for incremental summarization of task videos. We develop a sample-efficient few-shot approach for extracting semantic concepts as an i…
Position Matters! Empirical Study of Order Effect in Knowledge-grounded Dialogue
Hsuan Su, Shachi H Kumar, Sahisnu Mazumder +7
With the power of large pretrained language models, various research works have integrated knowledge into dialogue systems. The traditional techniques treat knowledge as part of th…
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning
Shih-Cheng Huang, Shih-Heng Wang, Min-Han Shih +2
Parameter-efficient (PE) methods (like Prompts or Adapters) for adapting pre-trained language models (PLM) to downstream tasks have been popular recently. However, hindrances still…
End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics
Eda Okur, Saurav Sahay, Roddy Fuentes Alba +1
The advances in language-based Artificial Intelligence (AI) technologies applied to build educational applications can present AI for social-good opportunities with a broader posit…
Human in the loop approaches in multi-modal conversational task guidance system development
Ramesh Manuvinakurike, Sovan Biswas, Giuseppe Raffa +6
Development of task guidance systems for aiding humans in a situated task remains a challenging problem. The role of search (information retrieval) and conversational systems for t…
Learning to Generate Prompts for Dialogue Generation through Reinforcement Learning
Hsuan Su, Pohan Chi, Shih-Cheng Huang +4
Much literature has shown that prompt-based learning is an efficient method to make use of the large pre-trained language model. Recent works also exhibit the possibility of steeri…
NLU for Game-based Learning in Real: Initial Evaluations
Eda Okur, Saurav Sahay, Lama Nachman
Intelligent systems designed for play-based interactions should be contextually aware of the users and their surroundings. Spoken Dialogue Systems (SDS) are critical for these inte…
Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System
Eda Okur, Saurav Sahay, Lama Nachman
Contextually aware intelligent agents are often required to understand the users and their surroundings in real-time. Our goal is to build Artificial Intelligence (AI) systems that…
Seamlessly Integrating Factual Information and Social Content with Persuasive Dialogue
Maximillian Chen, Weiyan Shi, Feifan Yan +4
Complex conversation settings such as persuasion involve communicating changes in attitude or behavior, so users' perspectives need to be addressed, even when not directly related…
Controllable Response Generation for Assistive Use-cases
Shachi H Kumar, Hsuan Su, Ramesh Manuvinakurike +2
Conversational agents have become an integral part of the general population for simple task enabling situations. However, these systems are yet to have any social impact on the di…
Semi-supervised Interactive Intent Labeling
Saurav Sahay, Eda Okur, Nagib Hakim +1
Building the Natural Language Understanding (NLU) modules of task-oriented Spoken Dialogue Systems (SDS) involves a definition of intents and entities, collection of task-relevant…
Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention
Hsuan Su, Jiun-Hao Jhan, Fan-yun Sun +2
Most chatbot literature that focuses on improving the fluency and coherence of a chatbot, is dedicated to making chatbots more human-like. However, very little work delves into wha…
Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration
Weiyan Shi, Yu Li, Saurav Sahay +1
Persuasion dialogue systems reflect the machine's ability to make strategic moves beyond verbal communication, and therefore differentiate themselves from task-oriented or open-dom…
Audio-Visual Understanding of Passenger Intents for In-Cabin Conversational Agents
Eda Okur, Shachi H Kumar, Saurav Sahay +1
Building multimodal dialogue understanding capabilities situated in the in-cabin context is crucial to enhance passenger comfort in autonomous vehicle (AV) interaction systems. To…
Low Rank Fusion based Transformers for Multimodal Sequences
Saurav Sahay, Eda Okur, Shachi H Kumar +1
Our senses individually work in a coordinated fashion to express our emotional intentions. In this work, we experiment with modeling modality-specific sensory signals to attend to…