4 papers
A Study of Finetuning Video Transformers for Multi-view Geometry Tasks
Huimin Wu, Kwang-Ting Cheng, Stephen Lin +1
This paper presents an investigation of vision transformer learning for multi-view geometry tasks, such as optical flow estimation, by fine-tuning video foundation models. Unlike p…
Exploring Transferability for Randomized Smoothing
Kai Qiu, Huishuai Zhang, Zhirong Wu +1
Training foundation models on extensive datasets and then finetuning them on specific tasks has emerged as the mainstream approach in artificial intelligence. However, the model ro…
NuTime: Numerically Multi-Scaled Embedding for Large-Scale Time-Series Pretraining
Chenguo Lin, Xumeng Wen, Wei Cao +4
Recent research on time-series self-supervised models shows great promise in learning semantic representations. However, it has been limited to small-scale datasets, e.g., thousand…
Associative Transformer
Yuwei Sun, Hideya Ochiai, Zhirong Wu +2
Emerging from the pairwise attention in conventional Transformers, there is a growing interest in sparse attention mechanisms that align more closely with localized, contextual lea…