23 citations · 68 across the 17 of their papers we have counts for
18 papers
Language-Image Alignment with Fixed Text Encoders
Jingfeng Yang, Ziyang Wu, Yue Zhao +1
Currently, the most dominant approach to establishing language-image alignment is to pre-train text and image encoders jointly through contrastive learning, such as CLIP and its va…
Simplifying DINO via Coding Rate Regularization
Ziyang Wu, Jingyuan Zhang, Druv Pai +5
DINO and DINOv2 are two model families being widely used to learn representations from unlabeled imagery data at large scales. Their learned representations often enable state-of-t…
Token Statistics Transformer: Linear-Time Attention via Variational Rate Reduction
Ziyang Wu, Tianjiao Ding, Yifu Lu +6
The attention operator is arguably the key distinguishing factor of transformer architectures, which have demonstrated state-of-the-art performance on a variety of tasks. However,…
Spatial-Temporal Mixture-of-Graph-Experts for Multi-Type Crime Prediction
Ziyang Wu, Fan Liu, Jindong Han +2
As various types of crime continue to threaten public safety and economic development, predicting the occurrence of multiple types of crimes becomes increasingly vital for effectiv…
A Survey of Foundation Models for Music Understanding
Wenjun Li, Ying Cai, Ziyang Wu +13
Music is essential in daily life, fulfilling emotional and entertainment needs, and connecting us personally, socially, and culturally. A better understanding of music can enhance…
Masked Completion via Structured Diffusion with White-Box Transformers
Druv Pai, Ziyang Wu, Sam Buchanan +2
Modern learning frameworks often train deep neural networks with massive amounts of unlabeled data to learn representations by solving simple pretext tasks, then use the representa…
LLoCO: Learning Long Contexts Offline
Sijun Tan, Xiuyu Li, Shishir Patil +5
Processing long contexts remains a challenge for large language models (LLMs) due to the quadratic computational and memory overhead of the self-attention mechanism and the substan…
When Do We Not Need Larger Vision Models?
Baifeng Shi, Ziyang Wu, Maolin Mao +2
Scaling up the size of vision models has been the de facto standard to obtain more powerful visual representations. In this work, we discuss the point beyond which larger vision mo…
White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
Yaodong Yu, Sam Buchanan, Druv Pai +7
In this paper, we contend that a natural objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a low-dimension…
Emergence of Segmentation with Minimalistic White-Box Transformers
Yaodong Yu, Tianzhe Chu, Shengbang Tong +4
Transformer-like models for vision tasks have recently proven effective for a wide range of downstream applications such as segmentation and detection. Previous works have shown th…
White-Box Transformers via Sparse Rate Reduction
Yaodong Yu, Sam Buchanan, Druv Pai +5
In this paper, we contend that the objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a mixture of low-dime…
Efficient Maximal Coding Rate Reduction by Variational Forms
Christina Baek, Ziyang Wu, Kwan Ho Ryan Chan +3
The principle of Maximal Coding Rate Reduction (MCR) has recently been proposed as a training objective for learning discriminative low-dimensional structures intrinsic to high…
Incremental Learning of Structured Memory via Closed-Loop Transcription
Shengbang Tong, Xili Dai, Ziyang Wu +3
This work proposes a minimal computational model for learning structured memories of multiple object classes in an incremental setting. Our approach is based on establishing a clos…
Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reduction
Xili Dai, Shengbang Tong, Mingyang Li +8
This work proposes a new computational framework for learning a structured generative model for real-world datasets. In particular, we propose to learn a closed-loop transcription…
How Low Can We Go: Trading Memory for Error in Low-Precision Training
Chengrun Yang, Ziyang Wu, Jerry Chee +2
Low-precision arithmetic trains deep learning models using less energy, less memory and less time. However, we pay a price for the savings: lower precision may yield larger round-o…
TenIPS: Inverse Propensity Sampling for Tensor Completion
Chengrun Yang, Lijun Ding, Ziyang Wu +1
Tensors are widely used to represent multiway arrays of data. The recovery of missing entries in a tensor has been extensively studied, generally under the assumption that entries…
Incremental Learning via Rate Reduction
Ziyang Wu, Christina Baek, Chong You +1
Current deep learning architectures suffer from catastrophic forgetting, a failure to retain knowledge of previously learned classes when incrementally trained on new classes. The…
Efficient AutoML Pipeline Search with Matrix and Tensor Factorization
Chengrun Yang, Jicong Fan, Ziyang Wu +1
Data scientists seeking a good supervised learning model on a new dataset have many choices to make: they must preprocess the data, select features, possibly reduce the dimension,…