153 citations · 525 across the 32 of their papers we have counts for
32 papers
Uni3D-MoE: Scalable Multimodal 3D Scene Understanding via Mixture of Experts
Yue Zhang, Yingzhao Jian, Hehe Fan +2
Recent advancements in multimodal large language models (MLLMs) have demonstrated considerable potential for comprehensive 3D scene understanding. However, existing approaches typi…
TSGS: Improving Gaussian Splatting for Transparent Surface Reconstruction via Normal and De-lighting Priors
Mingwei Li, Pu Pang, Hehe Fan +2
Reconstructing transparent surfaces is essential for tasks such as robotic manipulation in labs, yet it poses a significant challenge for 3D reconstruction techniques like 3D Gauss…
VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing
Xiangpeng Yang, Linchao Zhu, Hehe Fan +1
Recent advancements in diffusion models have significantly improved video generation and editing capabilities. However, multi-grained video editing, which encompasses class-level,…
EnergyMoGen: Compositional Human Motion Generation with Energy-Based Diffusion Model in Latent Space
Jianrong Zhang, Hehe Fan, Yi Yang
Diffusion models, particularly latent diffusion models, have demonstrated remarkable success in text-driven human motion generation. However, it remains challenging for latent diff…
CktGen: Automated Analog Circuit Design with Generative Artificial Intelligence
Yuxuan Hou, Hehe Fan, Jianrong Zhang +6
The automatic synthesis of analog circuits presents significant challenges. Most existing approaches formulate the problem as a single-objective optimization task, overlooking that…
ZeroMamba: Exploring Visual State Space Model for Zero-Shot Learning
Wenjin Hou, Dingjie Fu, Kun Li +3
Zero-shot learning (ZSL) aims to recognize unseen classes by transferring semantic knowledge from seen classes to unseen ones, guided by semantic information. To this end, existing…
Prototype Learning for Micro-gesture Classification
Guoliang Chen, Fei Wang, Kun Li +5
In this paper, we briefly introduce the solution developed by our team, HFUT-VUT, for the track of Micro-gesture Classification in the MiGA challenge at IJCAI 2024. The task of mic…
VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generation
Wenjie Zhuo, Fan Ma, Hehe Fan +1
This paper presents Invariant Score Distillation (ISD), a novel method for high-fidelity text-to-3D generation. ISD aims to tackle the over-saturation and over-smoothing problems i…
Prompt-Aware Adapter: Towards Learning Adaptive Visual Tokens for Multimodal Large Language Models
Yue Zhang, Hehe Fan, Yi Yang
To bridge the gap between vision and language modalities, Multimodal Large Language Models (MLLMs) usually learn an adapter that converts visual inputs to understandable tokens for…
TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment
Wei Li, Hehe Fan, Yongkang Wong +2
Recent advancements in image understanding have benefited from the extensive use of web image-text pairs. However, video understanding remains a challenge despite the availability…
Clustering for Protein Representation Learning
Ruijie Quan, Wenguan Wang, Fan Ma +2
Protein representation learning is a challenging task that aims to capture the structure and function of proteins from their amino acid sequences. Previous methods largely ignored…
EVA: Zero-shot Accurate Attributes and Multi-Object Video Editing
Xiangpeng Yang, Linchao Zhu, Hehe Fan +1
Current diffusion-based video editing primarily focuses on local editing (\textit{e.g.,} object/background editing) or global style editing by utilizing various dense correspondenc…
ProtChatGPT: Towards Understanding Proteins with Large Language Models
Chao Wang, Hehe Fan, Ruijie Quan +1
Protein research is crucial in various fundamental disciplines, but understanding their intricate structure-function relationships remains challenging. Recent Large Language Models…
HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting
Zhenglin Zhou, Fan Ma, Hehe Fan +2
Creating digital avatars from textual prompts has long been a desirable yet challenging task. Despite the promising results achieved with 2D diffusion priors, current methods strug…
FlowZero: Zero-Shot Text-to-Video Synthesis with LLM-Driven Dynamic Scene Syntax
Yu Lu, Linchao Zhu, Hehe Fan +1
Text-to-video (T2V) generation is a rapidly growing research area that aims to translate the scenes, objects, and actions within complex video text into a sequence of coherent visu…
Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos
Xiaoxiao Sheng, Zhiqiang Shen, Gang Xiao +3
We propose a unified point cloud video self-supervised learning framework for object-centric and scene-centric data. Previous methods commonly conduct representation learning at th…
Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videos
Zhiqiang Shen, Xiaoxiao Sheng, Hehe Fan +5
Recently, the community has made tremendous progress in developing effective methods for point cloud video understanding that learn from massive amounts of labeled data. However, a…
DPMix: Mixture of Depth and Point Cloud Video Experts for 4D Action Segmentation
Yue Zhang, Hehe Fan, Yi Yang +1
In this technical report, we present our findings from the research conducted on the Human-Object Interaction 4D (HOI4D) dataset for egocentric action segmentation task. As a relat…
Keyword-Aware Relative Spatio-Temporal Graph Networks for Video Question Answering
Yi Cheng, Hehe Fan, Dongyun Lin +3
The main challenge in video question answering (VideoQA) is to capture and understand the complex spatial and temporal relations between objects based on given questions. Existing…
A Study on Differentiable Logic and LLMs for EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition 2023
Yi Cheng, Ziwei Xu, Fen Fang +5
In this technical report, we present our findings from a study conducted on the EPIC-KITCHENS-100 Unsupervised Domain Adaptation task for Action Recognition. Our research focuses o…
Text to Point Cloud Localization with Relation-Enhanced Transformer
Guangzhi Wang, Hehe Fan, Mohan Kankanhalli
Automatically localizing a position based on a few natural language instructions is essential for future robots to communicate and collaborate with humans. To approach this goal, w…
STPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition
Ming Li, Xiangyu Xu, Hehe Fan +7
Existing methods of privacy-preserving action recognition (PPAR) mainly focus on frame-level (spatial) privacy removal through 2D CNNs. Unfortunately, they have two major drawbacks…
Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?
Yi Wang, Zhiwen Fan, Tianlong Chen +2
Vision Transformers (ViTs) have proven to be effective, in solving 2D image understanding tasks by training over large-scale image datasets; and meanwhile as a somehow separate tra…
SEFormer: Structure Embedding Transformer for 3D Object Detection
Xiaoyu Feng, Heming Du, Yueqi Duan +2
Effectively preserving and encoding structure features from objects in irregular and sparse LiDAR points is a key challenge to 3D object detection on point cloud. Recently, Transfo…