172 citations · 270 across the 34 of their papers we have counts for
42 papers
Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention
Sounak Mondal, Dimitris Samaras, Gregory Zelinsky +1
We introduce a novel learning problem: decoding gaze into natural language descriptions of human goals across diverse visual tasks. Unlike prior work, which frames gaze decoding as…
OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following
Qiaomu Miao, Haoyu Wu, Jingyi Xu +2
Understanding human gaze behavior is essential for complex scene comprehension and human-computer interaction. Traditional gaze following models are typically restricted to pure sp…
Personalized Image Descriptions from Attention Sequences
Ruoyu Xue, Hieu Le, Jingyi Xu +5
People can view the same image differently: they focus on different regions, objects, and details in varying orders and describe them in distinct linguistic styles. This leads to s…
Multi-view Gaze Target Estimation
Qiaomu Miao, Vivek Raju Golani, Jingyi Xu +3
This paper presents a method that utilizes multiple camera views for the gaze target estimation (GTE) task. The approach integrates information from different camera views to impro…
Few-shot Personalized Scanpath Prediction
Ruoyu Xue, Jingyi Xu, Sounak Mondal +4
A personalized model for scanpath prediction provides insights into the visual preferences and attention patterns of individual subjects. However, existing methods for training sca…
Diffusion-Refined VQA Annotations for Semi-Supervised Gaze Following
Qiaomu Miao, Alexandros Graikos, Jingwei Zhang +3
Training gaze following models requires a large number of images with gaze target coordinates annotated by human annotators, which is a laborious and inherently ambiguous process.…
Look Hear: Gaze Prediction for Speech-directed Human Attention
Sounak Mondal, Seoyoung Ahn, Zhibo Yang +4
For computer systems to effectively interact with humans using spoken language, they need to understand how the words being generated affect the users' moment-by-moment attention.…
Interactive Class-Agnostic Object Counting
Yifeng Huang, Viresh Ranjan, Minh Hoai
We propose a novel framework for interactive class-agnostic object counting, where a human user can interactively provide feedback to improve the accuracy of a counter. Our framewo…
Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention
Sounak Mondal, Zhibo Yang, Seoyoung Ahn +3
Predicting human gaze is important in Human-Computer Interaction (HCI). However, to practically serve HCI applications, gaze prediction models must be scalable, fast, and accurate…
Unifying Top-down and Bottom-up Scanpath Prediction Using Transformers
Zhibo Yang, Sounak Mondal, Seoyoung Ahn +4
Most models of visual attention aim at predicting either top-down or bottom-up control, as studied using different visual search and free-viewing tasks. In this paper we propose th…
Patch-level Gaze Distribution Prediction for Gaze Following
Qiaomu Miao, Minh Hoai, Dimitris Samaras
Gaze following aims to predict where a person is looking in a scene, by predicting the target location, or indicating that the target is located outside the image. Recent works det…
Self-Supervised Learning with Multi-View Rendering for 3D Point Cloud Analysis
Bach Tran, Binh-Son Hua, Anh Tuan Tran +1
Recently, great progress has been made in 3D deep learning with the emergence of deep neural networks specifically designed for 3D point clouds. These networks are often trained fr…
Few-shot Object Counting and Detection
Thanh Nguyen, Chau Pham, Khoi Nguyen +1
We tackle a new task of few-shot object counting and detection. Given a few exemplar bounding boxes of a target object class, we seek to count and detect all objects of the target…
Target-absent Human Attention
Zhibo Yang, Sounak Mondal, Seoyoung Ahn +3
The prediction of human gaze behavior is important for building human-computer interactive systems that can anticipate a user's attention. Computer vision models have been develope…
Exemplar Free Class Agnostic Counting
Viresh Ranjan, Minh Hoai
We tackle the task of Class Agnostic Counting, which aims to count objects in a novel object category at test time without any access to labeled training data for that category. Al…
Learning To Count Everything
Viresh Ranjan, Udbhav Sharma, Thu Nguyen +1
Existing works on visual counting primarily focus on one specific category at a time, such as people, animals, and cells. In this paper, we are interested in counting everything, t…
Progressive Semantic Segmentation
Chuong Huynh, Anh Tran, Khoa Luu +1
The objective of this work is to segment high-resolution images without overloading GPU memory usage or losing the fine details in the output segmentation map. The memory constrain…
Lipstick ain't enough: Beyond Color Matching for In-the-Wild Makeup Transfer
Thao Nguyen, Anh Tran, Minh Hoai
Makeup transfer is the task of applying on a source face the makeup style from a reference image. Real-life makeups are diverse and wild, which cover not only color-changing but al…
Explore Image Deblurring via Blur Kernel Space
Phong Tran, Anh Tran, Quynh Phung +1
This paper introduces a method to encode the blur operators of an arbitrary dataset of sharp-blur image pairs into a blur kernel space. Assuming the encoded kernel space is close e…
FineNet: Frame Interpolation and Enhancement for Face Video Deblurring
Phong Tran, Anh Tran, Thao Nguyen +1
The objective of this work is to deblur face videos. We propose a method that tackles this problem from two directions: (1) enhancing the blurry frames, and (2) treating the blurry…
Localization in the Crowd with Topological Constraints
Shahira Abousamra, Minh Hoai, Dimitris Samaras +1
We address the problem of crowd localization, i.e., the prediction of dots corresponding to people in a crowded scene. Due to various challenges, a localization method is prone to…
Structural and Functional Decomposition for Personality Image Captioning in a Communication Game
Thu Nguyen, Duy Phung, Minh Hoai +1
Personality image captioning (PIC) aims to describe an image with a natural language caption given a personality trait. In this work, we introduce a novel formulation for PIC based…
Distribution Matching for Crowd Counting
Boyu Wang, Huidong Liu, Dimitris Samaras +1
In crowd counting, each training image contains multiple people, where each person is annotated by a dot. Existing crowd counting methods need to use a Gaussian to smooth each anno…
Detecting Hands and Recognizing Physical Contact in the Wild
Supreeth Narasimhaswamy, Trung Nguyen, Minh Hoai
We investigate a new problem of detecting hands and recognizing their physical contact state in unconstrained conditions. This is a challenging inference task given the need to rea…