110 citations · 420 across the 55 of their papers we have counts for
63 papers
TGuide: Faster Autoregressive Video Diffusion via Historical Trajectory Guidance
Zhuo Chen, Bin Zou, Fanyue Wei +3
Can autoregressive video diffusion be accelerated further when only four denoising steps remain? Existing feature caching approaches struggle in this regime, as large feature chang…
Ego2World: Compiling Egocentric Cooking Videos into Executable Worlds for Belief-State Planning
Qinchuan Cheng, Zhantao Gong, Pengzhan Sun +2
Egocentric videos capture how people carry out everyday activities, yet testing an agent requires evaluating the consequences of actions it chooses itself. We introduce Ego2World,…
From Reasoning Failures to Composable Video Spatial Intelligence
Pengzhan Sun, Junbin Xiao, Ramanathan Rajaraman +2
Spatial reasoning benchmarks evaluate vision-language models across diverse tasks, but task-level scores do not reveal which underlying capabilities account for success or failure.…
Rethinking Probability-Based Reinforcement Learning From Posterior Concentration
Shiu-Hong Kao, Yubo Zhao, Zhenyu Tian +3
Verifier-free reinforcement learning with probability-based rewards offers a promising way to train LLMs on general reasoning tasks where external verifiers are unavailable. Yet th…
EgoVine: Post-Training with Branching Experience Compiled from Real-World Videos
Qinchuan Cheng, Bowen Lyu, Zhantao Gong +3
Real-world egocentric videos capture multi-step household procedures, but their recorded trajectories leave alternative actions and recovery from the learner's own mistakes largely…
Aligning Thoughts with Answers: Probability Rewards to Tame Thinking Drift
Pengzhan Sun, Shiu-hong Kao, Shijie Li +4
This paper studies \textbf{thinking--answer consistency} in vision-language models. We focus on Visual Intention Grounding, where a model infers a target object based on a human in…
ReSTI: A Source-Grounded Audit and Repair of STI-Bench
Pengzhan Sun, Ramanathan Rajaraman, Shiu-Hong Kao +2
Spatial--temporal benchmarks are valid only when their questions, source annotations, and answer options identify the same physical quantity. We audit STI-Bench against the officia…
HiFiHR: Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texture
Jiayin Zhu, Zhuoran Zhao, Linlin Yang +1
We present HiFiHR, a high-fidelity hand reconstruction approach that utilizes render-and-compare in the learning-based framework from a single image, capable of generating visually…
Opening the Vocabulary of Egocentric Actions
Dibyadip Chatterjee, Fadime Sener, Shugao Ma +1
Human actions in egocentric videos are often hand-object interactions composed from a verb (performed by the hand) applied to an object. Despite their extensive scaling up, egocent…
Learning to Generate Training Datasets for Robust Semantic Segmentation
Marwane Hariat, Olivier Laurent, Rémi Kazmierczak +4
Semantic segmentation methods have advanced significantly. Still, their robustness to real-world perturbations and object types not seen during training remains a challenge, partic…
Every Mistake Counts in Assembly
Guodong Ding, Fadime Sener, Shugao Ma +1
One promising use case of AI assistants is to help with complex procedures like cooking, home repair, and assembly tasks. Can we teach the assistant to interject after the user mak…
Overcoming the Trade-off Between Accuracy and Plausibility in 3D Hand Shape Reconstruction
Ziwei Yu, Chen Li, Linlin Yang +4
Direct mesh fitting for 3D hand shape reconstruction is highly accurate. However, the reconstructed meshes are prone to artifacts and do not appear as plausible hand shapes. Conver…
Enhancing Video Super-Resolution via Implicit Resampling-based Alignment
Kai Xu, Ziwei Yu, Xin Wang +2
In video super-resolution, it is common to use a frame-wise alignment to support the propagation of information over time. The role of alignment is well-studied for low-level enhan…
Contrastive Video Question Answering via Video Graph Transformer
Junbin Xiao, Pan Zhou, Angela Yao +4
We propose to perform video question answering (VideoQA) in a Contrastive manner via a Video Graph Transformer model (CoVGT). CoVGT's uniqueness and superiority are three-fold: 1)…
Bias-Compensated Integral Regression for Human Pose Estimation
Kerui Gu, Linlin Yang, Michael Bi Mi +1
In human and hand pose estimation, heatmaps are a crucial intermediate representation for a body or hand keypoint. Two popular methods to decode the heatmap into a final joint coor…
C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation
Dipika Singhania, Rahul Rahaman, Angela Yao
Temporal action segmentation tags action labels for every frame in an input untrimmed video containing multiple actions in a sequence. For the task of temporal action segmentation,…
UV-Based 3D Hand-Object Reconstruction with Grasp Optimization
Ziwei Yu, Linlin Yang, You Xie +2
We propose a novel framework for 3D hand shape reconstruction and hand-object grasp optimization from a single RGB image. The representation of hand-object contact regions is criti…
Temporal Action Segmentation: An Analysis of Modern Techniques
Guodong Ding, Fadime Sener, Angela Yao
Temporal action segmentation (TAS) in videos aims at densely identifying video frames in minutes-long videos with multiple action classes. As a long-range video understanding task,…
Perception-Distortion Balanced ADMM Optimization for Single-Image Super-Resolution
Yuehan Zhang, Bo Ji, Jia Hao +1
In image super-resolution, both pixel-wise accuracy and perceptual fidelity are desirable. However, most deep learning methods only achieve high performance in one aspect due to th…
Leveraging Action Affinity and Continuity for Semi-supervised Temporal Action Segmentation
Guodong Ding, Angela Yao
We present a semi-supervised learning approach to the temporal action segmentation task. The goal of the task is to temporally detect and segment actions in long, untrimmed procedu…
A Generalized & Robust Framework For Timestamp Supervision in Temporal Action Segmentation
Rahul Rahaman, Dipika Singhania, Alexandre Thiery +1
In temporal action segmentation, Timestamp supervision requires only a handful of labelled frames per video sequence. For unlabelled frames, previous works rely on assigning hard l…
Discrete-Constrained Regression for Local Counting Models
Haipeng Xiong, Angela Yao
Local counts, or the number of objects in a local area, is a continuous value by nature. Yet recent state-of-the-art methods show that formulating counting as a classification task…
Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities
Fadime Sener, Dibyadip Chatterjee, Daniel Shelepov +4
Assembly101 is a new procedural activity dataset featuring 4321 videos of people assembling and disassembling 101 "take-apart" toy vehicles. Participants work without fixed instruc…
TemporalUV: Capturing Loose Clothing with Temporally Coherent UV Coordinates
You Xie, Huiqi Mao, Angela Yao +1
We propose a novel approach to generate temporally coherent UV coordinates for loose clothing. Our method is not constrained by human body outlines and can capture loose garments a…