379 citations · 780 across the 76 of their papers we have counts for
92 papers
ISCA: A Framework for Interview-Style Conversational Agents
Charles Welch, Allison Lahnala, Vasudha Varadarajan +4
We present a low-compute non-generative system for implementing interview-style conversational agents which can be used to facilitate qualitative data collection through controlled…
Language Model Alignment in Multilingual Trolley Problems
Zhijing Jin, Max Kleiman-Weiner, Giorgio Piatti +9
We evaluate the moral alignment of LLMs with human preferences in multilingual trolley problems. Building on the Moral Machine experiment, which captures over 40 million human judg…
Towards Region-aware Bias Evaluation Metrics
Angana Borah, Aparna Garimella, Rada Mihalcea
When exposed to human-generated data, language models are known to learn and amplify societal biases. While previous works introduced benchmarks that can be used to assess the bias…
CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark
David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo +73
Visual Question Answering (VQA) is an important task in multimodal AI, and it is often used to test the ability of vision-language models to understand and reason on knowledge pres…
Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense
Siqi Shen, Lajanugen Logeswaran, Moontae Lee +3
Large language models (LLMs) have demonstrated substantial commonsense understanding through numerous benchmark evaluations. However, their understanding of cultural commonsense re…
Quriosity: Analyzing Human Questioning Behavior and Causal Inquiry through Curiosity-Driven Queries
Roberto Ceraolo, Dmitrii Kharlapenko, Ahmad Khan +6
Recent progress in Large Language Model (LLM) technology has changed our role in interacting with these models. Instead of primarily testing these models with questions we already…
Implicit Personalization in Language Models: A Systematic Study
Zhijing Jin, Nils Heil, Jiarui Liu +5
Implicit Personalization (IP) is a phenomenon of language models inferring a user's background from the implicit cues in the input prompts and tailoring the response based on this…
Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
Giorgio Piatti, Zhijing Jin, Max Kleiman-Weiner +3
As AI systems pervade human life, ensuring that large language models (LLMs) make safe decisions remains a significant challenge. We introduce the Governance of the Commons Simulat…
MAiDE-up: Multilingual Deception Detection of GPT-generated Hotel Reviews
Oana Ignat, Xiaomeng Xu, Rada Mihalcea
Deceptive reviews are becoming increasingly common, especially given the increase in performance and the prevalence of LLMs. While work to date has addressed the development of mod…
Cross-cultural Inspiration Detection and Analysis in Real and LLM-generated Social Media Data
Oana Ignat, Gayathri Ganesh Lakshmy, Rada Mihalcea
Inspiration is linked to various positive outcomes, such as increased creativity, productivity, and happiness. Although inspiration has great potential, there has been limited effo…
Do LLMs Think Fast and Slow? A Causal Study on Sentiment Analysis
Zhiheng Lyu, Zhijing Jin, Fernando Gonzalez +3
Sentiment analysis (SA) aims to identify the sentiment expressed in a text, such as a product review. Given a review and the sentiment associated with it, this work formulates SA a…
Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization
Navonil Majumder, Chia-Yu Hung, Deepanway Ghosal +3
Generative multimodal content is increasingly prevalent in much of the content creation arena, as it has the potential to allow artists and media personnel to create pre-production…
The Generation Gap: Exploring Age Bias in the Value Systems of Large Language Models
Siyang Liu, Trish Maturi, Bowen Yi +2
We explore the alignment of values in Large Language Models (LLMs) with specific age groups, leveraging data from the World Value Survey across thirteen categories. Through a diver…
Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Data
Shinka Mori, Oana Ignat, Andrew Lee +1
Synthetic data generation has the potential to impact applications and domains with scarce data. However, before such data is used for sensitive tasks such as mental health, we nee…
Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation
Do June Min, Veronica Perez-Rosas, Kenneth Resnicow +1
In this paper, we study the problem of multi-reward reinforcement learning to jointly optimize for multiple text qualities for natural language generation. We focus on the task of…
Annotations on a Budget: Leveraging Geo-Data Similarity to Balance Model Performance and Annotation Cost
Oana Ignat, Longju Bai, Joan Nwatu +1
Current foundation models have shown impressive performance across various tasks. However, several studies have revealed that these models are not effective for everyone due to the…
Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMs
Naihao Deng, Zhenjie Sun, Ruiqi He +5
In this paper, we investigate the effectiveness of various LLMs in interpreting tabular data through different prompting strategies and data formats. Our analyses extend across six…
A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity
Andrew Lee, Xiaoyan Bai, Itamar Pres +3
While alignment algorithms are now commonly used to tune pre-trained language models towards a user's preferences, we lack explanations for the underlying mechanisms in which model…
Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions
Pengfei Hong, Navonil Majumder, Deepanway Ghosal +3
Recent advancements in Large Language Models (LLMs) have showcased striking results on existing logical reasoning benchmarks, with some models even surpassing human performance. Ho…
Whose wife is it anyway? Assessing bias against same-gender relationships in machine translation
Ian Stewart, Rada Mihalcea
Machine translation often suffers from biased data and algorithms that can lead to unacceptable errors in system output. While bias in gender norms has been investigated, less is k…
Task-Adaptive Tokenization: Enhancing Long-Form Text Generation Efficacy in Mental Health and Beyond
Siyang Liu, Naihao Deng, Sahand Sabour +3
We propose task-adaptive tokenization as a way to adapt the generation pipeline to the specifics of a downstream task and enhance long-form generation in mental health. Inspired by…
Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models
Joan Nwatu, Oana Ignat, Rada Mihalcea
Despite the impressive performance of current AI models reported across various tasks, performance reports often do not include evaluations of how these models perform on the speci…
Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networks
Panfeng Li, Mohamed Abouelenien, Rada Mihalcea +3
Deception detection is gaining increasing interest due to ethical and security concerns. This paper explores the application of convolutional neural networks for the purpose of mul…
VERVE: Template-based ReflectiVE Rewriting for MotiVational IntErviewing
Do June Min, Verónica Pérez-Rosas, Kenneth Resnicow +1
Reflective listening is a fundamental skill that counselors must acquire to achieve proficiency in motivational interviewing (MI). It involves responding in a manner that acknowled…