337 citations · 369 across the 21 of their papers we have counts for
23 papers
Video2Reaction: Training Foundation Video Models to Predict Audience Reaction
Sidong Zhang, Trang Nguyen, Shiv Shankar +4
We introduce Video2Reaction, a multimodal dataset that maps short movie segments to the induced emotional reactions of viewers in the wild, as expressed through social media commen…
Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild
Trang Nguyen, Sidong Zhang, Shiv Shankar +4
Understanding and forecasting audience reactions to video content are crucial for improving content creation, recommendation systems, and media analysis. To enable audience reactio…
Audio-Visual Speech Separation via Bottleneck Iterative Network
Sidong Zhang, Shiv Shankar, Trang Nguyen +2
Integration of information from non-auditory cues can significantly improve the performance of speech-separation models. Often such models use deep modality-specific networks to ob…
Learning Straight Flows by Learning Curved Interpolants
Shiv Shankar, Tomas Geffner
Flow matching models typically use linear interpolants to define the forward/noise addition process. This, together with the independent coupling between noise and target distribut…
A/B testing under Interference with Partial Network Information
Shiv Shankar, Ritwik Sinha, Yash Chandak +2
A/B tests are often required to be conducted on subjects that might have social connections. For e.g., experiments on social media, or medical and social interventions to control t…
Adaptive Instrument Design for Indirect Experiments
Yash Chandak, Shiv Shankar, Vasilis Syrgkanis +1
Indirect experiments provide a valuable framework for estimating treatment effects in situations where conducting randomized control trials (RCTs) is impractical or unethical. Unli…
Optimization using Parallel Gradient Evaluations on Multiple Parameters
Yash Chandak, Shiv Shankar, Venkata Gandikota +2
We propose a first-order method for convex optimization, where instead of being restricted to the gradient from a single parameter, gradients from multiple parameters can be used d…
Off-Policy Evaluation for Action-Dependent Non-Stationary Environments
Yash Chandak, Shiv Shankar, Nathaniel D. Bastian +3
Methods for sequential decision-making are often built upon a foundational assumption that the underlying decision process is stationary. This limits the application of such method…
Implicit Training of Energy Model for Structure Prediction
Shiv Shankar, Vihari Piratla
Most deep learning research has focused on developing new model and training procedures. On the other hand the training objective has usually been restricted to combinations of sta…
Privacy Aware Experiments without Cookies
Shiv Shankar, Ritwik Sinha, Saayan Mitra +3
Consider two brands that want to jointly test alternate web experiences for their customers with an A/B test. Such collaborative tests are today enabled using \textit{third-party c…
Progressive Fusion for Multimodal Integration
Shiv Shankar, Laure Thompson, Madalina Fiterau
Integration of multimodal information from various sources has been shown to boost the performance of machine learning models and thus has received increased attention in recent ye…
Neural Dependency Coding inspired Multimodal Fusion
Shiv Shankar
Information integration from different modalities is an active area of research. Human beings and, in general, biological neural systems are quite adept at using a multitude of sig…
Adversarial Stein Training for Graph Energy Models
Shiv Shankar
Learning distributions over graph-structured data is a challenging task with many applications in biology and chemistry. In this work we use an energy-based model (EBM) based on mu…
Sibling Regression for Generalized Linear Models
Shiv Shankar, Daniel Sheldon
Field observations form the basis of many scientific studies, especially in ecological and social sciences. Despite efforts to conduct such surveys in a standardized way, observati…
High-Confidence Off-Policy (or Counterfactual) Variance Estimation
Yash Chandak, Shiv Shankar, Philip S. Thomas
Many sequential decision-making systems leverage data collected using prior policies to propose a new policy. For critical applications, it is important that high-confidence guaran…
Bosonic Random Walk Networks for Graph Learning
Shiv Shankar, Don Towsley
The development of Graph Neural Networks (GNNs) has led to great progress in machine learning on graph-structured data. These networks operate via diffusing information across the…
Three-quarter Sibling Regression for Denoising Observational Data
Shiv Shankar, Daniel Sheldon, Tao Sun +2
Many ecological studies and conservation policies are based on field observations of species, which can be affected by systematic variability introduced by the observation process.…
Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint
Vihari Piratla, Shiv Shankar
Data augmentation is a popular pre-processing trick to improve generalization accuracy. It is believed that by processing augmented inputs in tandem with the original ones, the mod…
Optimizing for the Future in Non-Stationary MDPs
Yash Chandak, Georgios Theocharous, Shiv Shankar +3
Most reinforcement learning methods are based upon the key assumption that the transition dynamics and reward functions are fixed, that is, the underlying Markov decision process i…
Differential Equation Units: Learning Functional Forms of Activation Functions from Data
MohamadAli Torkamani, Shiv Shankar, Amirmohammad Rooshenas +1
Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential e…
Learning Compact Neural Networks Using Ordinary Differential Equations as Activation Functions
MohamadAli Torkamani, Phillip Wallis, Shiv Shankar +1
Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential e…
Generalizing Across Domains via Cross-Gradient Training
Shiv Shankar, Vihari Piratla, Soumen Chakrabarti +3
We present CROSSGRAD, a method to use multi-domain training data to learn a classifier that generalizes to new domains. CROSSGRAD does not need an adaptation phase via labeled or u…
Labeled Memory Networks for Online Model Adaptation
Shiv Shankar, Sunita Sarawagi
Augmenting a neural network with memory that can grow without growing the number of trained parameters is a recent powerful concept with many exciting applications. We propose a de…