6 citations · 6 across the 9 of their papers we have counts for
9 papers
Multi-Modal Representation Learning via Semi-Supervised Rate Reduction for Generalized Category Discovery
Wei He, Xianghan Meng, Zhiyuan Huang +3
Generalized Category Discovery (GCD) aims to identify both known and unknown categories, with only partial labels given for the known categories, posing a challenging open-set reco…
Open-Text Aerial Detection: A Unified Framework For Aerial Visual Grounding And Detection
Guoting Wei, Xia Yuan, Yang Zhou +6
Open-Vocabulary Aerial Detection (OVAD) and Remote Sensing Visual Grounding (RSVG) have emerged as two key paradigms for aerial scene understanding. However, each paradigm suffers…
Delving into Muon and Beyond: Deep Analysis and Extensions
Xianbiao Qi, Marco Chen, Jiaquan Ye +2
The Muon optimizer has recently attracted considerable attention for its strong empirical performance and use of orthogonalized updates on matrix-shaped parameters, yet its underly…
SimpleGPT: Improving GPT via A Simple Normalization Strategy
Marco Chen, Xianbiao Qi, Yelin He +2
In this work, we revisit Transformer optimization through the lens of second-order geometry and establish a direct connection between architectural design, activation scale, the He…
HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs
Dongquan Yang, Yifan Yang, Xiaotian Yu +2
Processing long-context inputs with large language models presents a significant challenge due to the enormous memory requirements of the Key-Value (KV) cache during inference. Exi…
DNT: a Deeply Normalized Transformer that can be trained by Momentum SGD
Xianbiao Qi, Marco Chen, Wenjie Xiao +4
Transformers have become the de facto backbone of modern deep learning, yet their training typically demands an advanced optimizer with adaptive learning rate like AdamW, rather th…
Taming Transformer Without Using Learning Rate Warmup
Xianbiao Qi, Yelin He, Jiaquan Ye +5
Scaling Transformer to a large scale without using some technical tricks such as learning rate warump and using an obviously lower learning rate is an extremely challenging task, a…
Exploring a Principled Framework for Deep Subspace Clustering
Xianghan Meng, Zhiyuan Huang, Wei He +3
Subspace clustering is a classical unsupervised learning task, built on a basic assumption that high-dimensional data can be approximated by a union of subspaces (UoS). Nevertheles…
Neural Normalized Cut: A Differential and Generalizable Approach for Spectral Clustering
Wei He, Shangzhi Zhang, Chun-Guang Li +3
Spectral clustering, as a popular tool for data clustering, requires an eigen-decomposition step on a given affinity to obtain the spectral embedding. Nevertheless, such a step suf…