393 citations · 694 across the 19 of their papers we have counts for
20 papers
OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
Negin Baghbanzadeh, Pritam Sarkar, Michael Colacci +6
High-stakes clinical use of large vision-language models (LVLMs) requires reasoning that is grounded in visual evidence and clinical knowledge, not just correct final answers. We i…
VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models
Pritam Sarkar, Ali Etemad
Despite recent advances in video understanding, the capabilities of Large Video Language Models (LVLMs) to perform video-based causal reasoning remains underexplored, largely due t…
Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization
Pritam Sarkar, Ali Etemad
Despite recent advances in Large Video Language Models (LVLMs), they still struggle with fine-grained temporal understanding, hallucinate, and often make simple mistakes on even si…
Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment
Pritam Sarkar, Sayna Ebrahimi, Ali Etemad +3
Despite their significant advancements, Multimodal Large Language Models (MLLMs) often generate factually inaccurate information, referred to as hallucination. In this work, we add…
CLARE: Cognitive Load Assessment in REaltime with Multimodal Data
Anubhav Bhatti, Prithila Angkan, Behnam Behinaein +11
We present a novel multimodal dataset for Cognitive Load Assessment in REal-time (CLARE). The dataset contains physiological and gaze data from 24 participants with self-reported c…
Speech Emotion Recognition with Distilled Prosodic and Linguistic Affect Representations
Debaditya Shome, Ali Etemad
We propose EmoDistill, a novel speech emotion recognition (SER) framework that leverages cross-modal knowledge distillation during training to learn strong linguistic and prosodic…
Region-Disentangled Diffusion Model for High-Fidelity PPG-to-ECG Translation
Debaditya Shome, Pritam Sarkar, Ali Etemad
The high prevalence of cardiovascular diseases (CVDs) calls for accessible and cost-effective continuous cardiac monitoring tools. Despite Electrocardiography (ECG) being the gold…
Uncovering the Hidden Dynamics of Video Self-supervised Learning under Distribution Shifts
Pritam Sarkar, Ahmad Beirami, Ali Etemad
Video self-supervised learning (VSSL) has made significant progress in recent years. However, the exact behavior and dynamics of these models under different forms of distribution…
XKD: Cross-modal Knowledge Distillation with Domain Alignment for Video Representation Learning
Pritam Sarkar, Ali Etemad
We present XKD, a novel self-supervised framework to learn meaningful representations from unlabelled videos. XKD is trained with two pseudo objectives. First, masked data reconstr…
AVCAffe: A Large Scale Audio-Visual Dataset of Cognitive Load and Affect for Remote Work
Pritam Sarkar, Aaron Posen, Ali Etemad
We introduce AVCAffe, the first Audio-Visual dataset consisting of Cognitive load and Affect attributes. We record AVCAffe by simulating remote work scenarios over a video-conferen…
Gaze Estimation with Eye Region Segmentation and Self-Supervised Multistream Learning
Zunayed Mahmud, Paul Hungler, Ali Etemad
We present a novel multistream network that learns robust eye representations for gaze estimation. We first create a synthetic dataset containing eye region masks detailing the vis…
Self-Supervised Audio-Visual Representation Learning with Relaxed Cross-Modal Synchronicity
Pritam Sarkar, Ali Etemad
We present CrissCross, a self-supervised framework for learning audio-visual representations. A novel notion is introduced in our framework whereby in addition to learning the intr…
A Transformer Architecture for Stress Detection from ECG
Behnam Behinaein, Anubhav Bhatti, Dirk Rodenburg +2
Electrocardiogram (ECG) has been widely used for emotion recognition. This paper presents a deep neural network based on convolutional layers and a transformer mechanism to detect…
Attentive Cross-modal Connections for Deep Multimodal Wearable-based Emotion Recognition
Anubhav Bhatti, Behnam Behinaein, Dirk Rodenburg +2
Classification of human emotions can play an essential role in the design and improvement of human-machine systems. While individual biological signals such as Electrocardiogram (E…
Detection of Maternal and Fetal Stress from the Electrocardiogram with Self-Supervised Representation Learning
Pritam Sarkar, Silvia Lobmaier, Bibiana Fabre +5
In the pregnant mother and her fetus, chronic prenatal stress results in entrainment of the fetal heartbeat by the maternal heartbeat, quantified by the fetal stress index (FSI). D…
CardioGAN: Attentive Generative Adversarial Network with Dual Discriminators for Synthesis of ECG from PPG
Pritam Sarkar, Ali Etemad
Electrocardiogram (ECG) is the electrical measurement of cardiac activity, whereas Photoplethysmogram (PPG) is the optical measurement of volumetric changes in blood circulation. W…
Unsupervised Multi-Modal Representation Learning for Affective Computing with Multi-Corpus Wearable Data
Kyle Ross, Paul Hungler, Ali Etemad
With recent developments in smart technologies, there has been a growing focus on the use of artificial intelligence and machine learning for affective computing to further enhance…
Self-supervised ECG Representation Learning for Emotion Recognition
Pritam Sarkar, Ali Etemad
We exploit a self-supervised deep multi-task learning framework for electrocardiogram (ECG) -based emotion recognition. The proposed solution consists of two stages of learning a)…
Self-supervised Learning for ECG-based Emotion Recognition
Pritam Sarkar, Ali Etemad
We present an electrocardiogram (ECG) -based emotion recognition system using self-supervised learning. Our proposed architecture consists of two main networks, a signal transforma…
Classification of Cognitive Load and Expertise for Adaptive Simulation using Deep Multitask Learning
Pritam Sarkar, Kyle Ross, Aaron J. Ruberto +3
Simulations are a pedagogical means of enabling a risk-free way for healthcare practitioners to learn, maintain, or enhance their knowledge and skills. Such simulations should prov…