From the 1 of 6 linked papers with an AI index.
6 papers
xHC: Expanded Hyper-Connections
Xiangdong Zhang, Xiaohan Qin, Sunan Zou +10
The paper introduces xHC, a method that expands the residual stream of Transformers to many parallel streams using temporal feature augmentation and a sparse update scheme, enablin…
NITP: Next Implicit Token Prediction for LLM Pre-training
Xiangdong Zhang, Debing Zhang, Shaofeng Zhang +3
Standard next-token prediction (NTP) supervises language models solely through discrete labels in the output logit space. We argue that this sparse one-hot supervision leaves the l…
Dual-Branch Center-Surrounding Contrast: Rethinking Contrastive Learning for 3D Point Clouds
Shaofeng Zhang, Xuanqi Chen, Xiangdong Zhang +2
Most existing self-supervised learning (SSL) approaches for 3D point clouds are dominated by generative methods based on Masked Autoencoders (MAE). However, these generative method…
Towards More Diverse and Challenging Pre-training for Point Cloud Learning: Self-Supervised Cross Reconstruction with Decoupled Views
Xiangdong Zhang, Shaofeng Zhang, Junchi Yan
Point cloud learning, especially in a self-supervised way without manual labels, has gained growing attention in both vision and learning communities due to its potential utility i…
VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models
Xiangdong Zhang, Jiaqi Liao, Shaofeng Zhang +4
Recent advancements in text-to-video (T2V) diffusion models have enabled high-fidelity and realistic video synthesis. However, current T2V models often struggle to generate physica…
PCP-MAE: Learning to Predict Centers for Point Masked Autoencoders
Xiangdong Zhang, Shaofeng Zhang, Junchi Yan
Masked autoencoder has been widely explored in point cloud self-supervised learning, whereby the point cloud is generally divided into visible and masked parts. These methods typic…