106 citations · 150 across the 13 of their papers we have counts for
13 papers
On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance
Etienne Casanova, Rafal Kocielnik, R. Michael Alvarez
Large Language Models (LLMs) are increasingly used for zero-shot annotation and LLM-as-a-judge tasks, yet their reliability hinges on how model-internalized priors interact with us…
Automating Feedback Analysis in Surgical Training: Detection, Categorization, and Assessment
Firdavs Nasriddinov, Rafal Kocielnik, Arushi Gupta +4
This work introduces the first framework for reconstructing surgical dialogue from unstructured real-world recordings, which is crucial for characterizing teaching tasks. In surgic…
Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment
Arushi Gupta, Rafal Kocielnik, Jiayun Wang +5
During surgical training, real-time feedback from trainers to trainees is important for preventing errors and enhancing long-term skill acquisition. Accurately predicting the effec…
Online Moderation in Competitive Action Games: How Intervention Affects Player Behaviors
Zhuofang Li, Rafal Kocielnik, Mitchell Linegar +7
Online competitive action games have flourished as a space for entertainment and social connections, yet they face challenges from a small percentage of players engaging in disrupt…
Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposium
Hyewon Jeong, Sarah Jabbour, Yuzhe Yang +40
The third ML4H symposium was held in person on December 10, 2023, in New Orleans, Louisiana, USA. The symposium included research roundtable sessions to foster discussions between…
ChatGPT Based Data Augmentation for Improved Parameter-Efficient Debiasing of LLMs
Pengrui Han, Rafal Kocielnik, Adhithya Saravanan +3
Large Language models (LLMs), while powerful, exhibit harmful social biases. Debiasing is often challenging due to computational costs, data constraints, and potential degradation…
Deep Multimodal Fusion for Surgical Feedback Classification
Rafal Kocielnik, Elyssa Y. Wong, Timothy N. Chu +5
Quantification of real-time informal feedback delivered by an experienced surgeon to a trainee during surgery is important for skill improvements in surgical training. Such feedbac…
Exploring Social Bias in Downstream Applications of Text-to-Image Foundation Models
Adhithya Prakash Saravanan, Rafal Kocielnik, Roy Jiang +2
Text-to-image diffusion models have been adopted into key commercial workflows, such as art generation and image editing. Characterising the implicit social biases they exhibit, su…
BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models
Rafal Kocielnik, Shrimai Prabhumoye, Vivian Zhang +3
Pretrained Language Models (PLMs) harbor inherent social biases that can result in harmful real-world implications. Such social biases are measured through the probability values t…
Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructions
Rafal Kocielnik, Sara Kangaslahti, Shrimai Prabhumoye +3
Labeling social-media data for custom dimensions of toxicity and social bias is challenging and labor-intensive. Existing transfer and active learning approaches meant to reduce an…
From Who You Know to What You Read: Augmenting Scientific Recommendations with Implicit Social Networks
Hyeonsu B. Kang, Rafal Kocielnik, Andrew Head +6
The ever-increasing pace of scientific publication necessitates methods for quickly identifying relevant papers. While neural recommenders trained on user interests can help, they…
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases
Shrimai Prabhumoye, Rafal Kocielnik, Mohammad Shoeybi +2
Detecting social bias in text is challenging due to nuance, subjectivity, and difficulty in obtaining good quality labeled datasets at scale, especially given the evolving nature o…
Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop
Justin Cranshaw, Emad Elwany, Todd Newman +5
Although information workers may complain about meetings, they are an essential part of their work life. Consequently, busy people spend a significant amount of time scheduling mee…