248 citations · 309 across the 28 of their papers we have counts for
29 papers
RoMo: A Large-Scale, Richly Organized Dataset and Semantic Taxonomy for Human Motion Generation
Jiahao Zhang, Joseph Liu, Young-Yoon Lee +9
Success in generative modeling across language, image, and video demonstrates that large, well-curated datasets are the key driver for building capable models. 3D Human motion, how…
MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory
Minghao Guo, Qingyue Jiao, Zeru Shi +14
Long-term agent memory is increasingly multimodal, yet existing evaluations rarely test whether agents preserve the visual evidence needed for later reasoning. In prior work, many…
JACoP: Joint Alignment for Compliant Multi-Agent Prediction
Qingze Liu, Alen Mrdovic, Danrui Li +3
Stochastic Human Trajectory Prediction (HTP) using generative modeling has emerged as a significant area of research. Although state-of-the-art models excel in optimizing the accur…
ECTraj: Enhanced Consistency Training for Multi-Agent Trajectory Prediction
Alen Mrdovic, Qingze, Liu +6
Diffusion models for multi-agent trajectory prediction are limited by iterative denoising, which causes inference latency that hinders their use in time-critical settings like auto…
ArchSeek: Retrieving Architectural Case Studies Using Vision-Language Models
Danrui Li, Yichao Shi, Yaluo Wang +2
Efficiently searching for relevant case studies is critical in architectural design, as designers rely on precedent examples to guide or inspire their ongoing projects. However, tr…
CASIM: Composite Aware Semantic Injection for Text to Motion Generation
Che-Jui Chang, Qingze Tony Liu, Honglu Zhou +2
Recent advances in generative modeling and tokenization have driven significant progress in text-to-motion generation, leading to enhanced quality and realism in generated motions.…
Cardiverse: Harnessing LLMs for Novel Card Game Prototyping
Danrui Li, Sen Zhang, Sam S. Sohn +3
The prototyping of computer games, particularly card games, requires extensive human effort in creative ideation and gameplay evaluation. Recent advances in Large Language Models (…
FCC: Fully Connected Correlation for One-Shot Segmentation
Seonghyeon Moon, Haein Kong, Muhammad Haris Khan +2
Few-shot segmentation (FSS) aims to segment the target object in a query image using only a small set of support images and masks. Therefore, having strong prior information for th…
TrajDiffuse: A Conditional Diffusion Model for Environment-Aware Trajectory Prediction
Qingze, Liu, Danrui Li +4
Accurate prediction of human or vehicle trajectories with good diversity that captures their stochastic nature is an essential task for many applications. However, many trajectory…
From Words to Worlds: Transforming One-line Prompt into Immersive Multi-modal Digital Stories with Communicative LLM Agent
Samuel S. Sohn, Danrui Li, Sen Zhang +2
Digital storytelling, essential in entertainment, education, and marketing, faces challenges in production scalability and flexibility. The StoryAgent framework, introduced in this…
Microscopic modeling of attention-based movement behaviors
Danrui Li, Mathew Schwartz, Samuel S. Sohn +3
For transportation hubs, leveraging pedestrian flows for commercial activities presents an effective strategy for funding maintenance and infrastructure improvements. However, this…
On the Equivalency, Substitutability, and Flexibility of Synthetic Data
Che-Jui Chang, Danrui Li, Seonghyeon Moon +1
We study, from an empirical standpoint, the efficacy of synthetic data in real-world scenarios. Leveraging synthetic data for training perception models has become a key strategy e…
The Importance of Multimodal Emotion Conditioning and Affect Consistency for Embodied Conversational Agents
Che-Jui Chang, Samuel S. Sohn, Sen Zhang +3
Previous studies regarding the perception of emotions for embodied virtual agents have shown the effectiveness of using virtual characters in conveying emotions through interaction…
M3Act: Learning from Synthetic Human Group Activities
Che-Jui Chang, Danrui Li, Deep Patel +7
The study of complex human interactions and group activities has become a focal point in human-centric computer vision. However, progress in related tasks is often hindered by the…
Procedure-Aware Pretraining for Instructional Video Understanding
Honglu Zhou, Roberto Martín-Martín, Mubbasir Kapadia +2
Our goal is to learn a video representation that is useful for downstream procedure understanding tasks in instructional videos. Due to the small amount of available annotations, a…
MSI: Maximize Support-Set Information for Few-Shot Segmentation
Seonghyeon Moon, Samuel S. Sohn, Honglu Zhou +4
FSS(Few-shot segmentation) aims to segment a target class using a small number of labeled images(support set). To extract information relevant to the target class, a dominant appro…
An Information-Theoretic Approach for Estimating Scenario Generalization in Crowd Motion Prediction
Gang Qiao, Kaidong Hu, Seonghyeon Moon +4
Learning-based approaches to modeling crowd motion have become increasingly successful but require training and evaluation on large datasets, coupled with complex model selection a…
HM: Hybrid Masking for Few-Shot Segmentation
Seonghyeon Moon, Samuel S. Sohn, Honglu Zhou +4
We study few-shot semantic segmentation that aims to segment a target object from a query image when provided with a few annotated support images of the target class. Several recen…
MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Prediction
Mihee Lee, Samuel S. Sohn, Seonghyeon Moon +3
Accurate long-term trajectory prediction in complex scenes, where multiple agents (e.g., pedestrians or vehicles) interact with each other and the environment while attempting to a…
D-HYPR: Harnessing Neighborhood Modeling and Asymmetry Preservation for Digraph Representation Learning
Honglu Zhou, Advith Chegu, Samuel S. Sohn +3
Digraph Representation Learning (DRL) aims to learn representations for directed homogeneous graphs (digraphs). Prior work in DRL is largely constrained (e.g., limited to directed…
COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality
Honglu Zhou, Asim Kadav, Aviv Shamsian +6
Group Activity Recognition detects the activity collectively performed by a group of actors, which requires compositional reasoning of actors and objects. We approach the task by m…
Hopper: Multi-hop Transformer for Spatiotemporal Reasoning
Honglu Zhou, Asim Kadav, Farley Lai +4
This paper considers the problem of spatiotemporal object-centric reasoning in videos. Central to our approach is the notion of object permanence, i.e., the ability to reason about…
Graph-Based Generative Representation Learning of Semantically and Behaviorally Augmented Floorplans
Vahid Azizi, Muhammad Usman, Honglu Zhou +2
Floorplans are commonly used to represent the layout of buildings. In computer aided-design (CAD) floorplans are usually represented in the form of hierarchical graph structures. R…
GitEvolve: Predicting the Evolution of GitHub Repositories
Honglu Zhou, Hareesh Ravi, Carlos M. Muniz +4
Software development is becoming increasingly open and collaborative with the advent of platforms such as GitHub. Given its crucial role, there is a need to better understand and m…