222 citations · 554 across the 23 of their papers we have counts for
23 papers
ReguSim: Evaluating LLM Agent Rule Grounding in Financial Compliance
Yiyang Luo, Yihang Jiang, Qijun Xie +4
LLM agents in financial markets may cite rules yet still submit orders that violate executable constraints or misread surveillance evidence. We introduce ReguSim, a controlled fina…
Triplet-Block Diffusion RWKV
Ke Lin, Yiyang Luo, Zhaolong Su +2
Causal Transformer language models suffer from strictly sequential decoding and a quadratic per-step attention cost. While linear-time causal models and discrete diffusion models e…
CinePreGen: Camera Controllable Video Previsualization via Engine-powered Diffusion
Yiran Chen, Anyi Rao, Xuekun Jiang +5
With advancements in video generative AI models (e.g., SORA), creators are increasingly using these techniques to enhance video previsualization. However, they face challenges with…
Cinematic Behavior Transfer via NeRF-based Differentiable Filming
Xuekun Jiang, Anyi Rao, Jingbo Wang +2
In the evolving landscape of digital media and video production, the precise manipulation and reproduction of visual elements like camera movements and character actions are highly…
SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models
Yuwei Guo, Ceyuan Yang, Anyi Rao +3
The development of text-to-video (T2V), i.e., generating videos with a given text prompt, has been significantly advanced in recent years. However, relying solely on text prompts o…
Automated Conversion of Music Videos into Lyric Videos
Jiaju Ma, Anyi Rao, Li-Yi Wei +3
Musicians and fans often produce lyric videos, a form of music videos that showcase the song's lyrics, for their favorite songs. However, making such videos can be challenging and…
Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization
Yujie Zhou, Wenwen Qiang, Anyi Rao +3
Zero-shot skeleton-based action recognition aims to recognize actions of unseen categories after training on data of seen categories. The key is to build the connection between vis…
AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning
Yuwei Guo, Ceyuan Yang, Anyi Rao +6
With the advance of text-to-image (T2I) diffusion models (e.g., Stable Diffusion) and corresponding personalization techniques such as DreamBooth and LoRA, everyone can manifest th…
HireVAE: An Online and Adaptive Factor Model Based on Hierarchical and Regime-Switch VAE
Zikai Wei, Anyi Rao, Bo Dai +1
Factor model is a fundamental investment tool in quantitative investment, which can be empowered by deep learning to become more flexible and efficient in practical complicated inv…
CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers
Dachuan Shi, Chaofan Tao, Anyi Rao +3
Recent vision-language models have achieved tremendous advances. However, their computational costs are also escalating dramatically, making model acceleration exceedingly critical…
Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences
Yujie Zhou, Haodong Duan, Anyi Rao +2
Self-supervised learning has demonstrated remarkable capability in representation learning for skeleton-based action recognition. Existing methods mainly focus on applying global d…
Adding Conditional Control to Text-to-Image Diffusion Models
Lvmin Zhang, Anyi Rao, Maneesh Agrawala
We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready…
Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Production
Anyi Rao, Xuekun Jiang, Yuwei Guo +5
Amateurs working on mini-films and short-form videos usually spend lots of time and effort on the multi-round complicated process of setting and adjusting scenes, plots, and camera…
Temporal and Contextual Transformer for Multi-Camera Editing of TV Shows
Anyi Rao, Xuekun Jiang, Sichen Wang +7
The ability to choose an appropriate camera view among multiple cameras plays a vital role in TV shows delivery. But it is hard to figure out the statistical pattern and apply inte…
A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language
Bing Su, Dazhao Du, Zhao Yang +6
Although artificial intelligence (AI) has made significant progress in understanding molecules in a wide range of fields, existing models generally acquire the single cognitive abi…
AutoGPart: Intermediate Supervision Search for Generalizable 3D Part Segmentation
Xueyi Liu, Xiaomeng Xu, Anyi Rao +2
Training a generalizable 3D part segmentation network is quite challenging but of great importance in real-world applications. To tackle this problem, some works design task-specif…
BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering
Yuanbo Xiangli, Linning Xu, Xingang Pan +5
Neural radiance fields (NeRF) has achieved outstanding performance in modeling 3D objects and controlled scenes, usually under a single scale. In this work, we focus on multi-scale…
A Unified Framework for Shot Type Classification Based on Subject Centric Lens
Anyi Rao, Jiaze Wang, Linning Xu +4
Shots are key narrative elements of various videos, e.g. movies, TV series, and user-generated videos that are thriving over the Internet. The types of shots greatly influence how…
Online Multi-modal Person Search in Videos
Jiangyue Xia, Anyi Rao, Qingqiu Huang +3
The task of searching certain people in videos has seen increasing potential in real-world applications, such as video organization and editing. Most existing approaches are devise…
MovieNet: A Holistic Dataset for Movie Understanding
Qingqiu Huang, Yu Xiong, Anyi Rao +2
Recent years have seen remarkable advances in visual understanding. However, how to understand a story-based long video with artistic styles, e.g. movie, remains challenging. In th…
A Local-to-Global Approach to Multi-modal Movie Scene Segmentation
Anyi Rao, Linning Xu, Yu Xiong +4
Scene, as the crucial unit of storytelling in movies, contains complex activities of actors and their interactions in a physical environment. Identifying the composition of scenes…
Automatic Music Accompanist
Anyi Rao, Francis Lau
Automatic musical accompaniment is where a human musician is accompanied by a computer musician. The computer musician is able to produce musical accompaniment that relates musical…
HotFlip: White-Box Adversarial Examples for Text Classification
Javid Ebrahimi, Anyi Rao, Daniel Lowd +1
We propose an efficient method to generate white-box adversarial examples to trick a character-level neural classifier. We find that only a few manipulations are needed to greatly…