15 citations · 80 across the 42 of their papers we have counts for
20 papers · 1 filter
The Structural Origin of Attention Sink: Variance Discrepancy, Super Neurons, and Dimension Disparity
Siquan Li, Kaiqi Jiang, Jiacheng Sun +1
Despite the prevalence of the attention sink phenomenon in Large Language Models (LLMs), where initial tokens disproportionately monopolize attention scores, its structural origins…
Understanding and Guiding Layer Placement in Parameter-Efficient Fine-Tuning of Large Language Models
Yichen Xu, Yuyang Liang, Shan Dai +3
As large language models (LLMs) continue to grow, the cost of full-parameter fine-tuning has made parameter-efficient fine-tuning (PEFT) the default strategy for downstream adaptat…
Reinforcing Diffusion Models by Direct Group Preference Optimization
Yihong Luo, Tianyang Hu, Jing Tang
While reinforcement learning methods such as Group Relative Preference Optimization (GRPO) have significantly enhanced Large Language Models, adapting them to diffusion models rema…
Noise Consistency Training: A Native Approach for One-Step Generator in Learning Additional Controls
Yihong Luo, Shuchen Xue, Tianyang Hu +1
The pursuit of efficient and controllable high-quality content generation remains a central challenge in artificial intelligence-generated content (AIGC). While one-step generators…
Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
Shuchen Xue, Tianyu Xie, Tianyang Hu +5
Efficiently scaling Large Language Models (LLMs) necessitates exploring alternatives to dominant autoregressive (AR) methods, with Masked Diffusion Models (MDMs) emerging as candid…
Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling
Tianyu Xie, Shuchen Xue, Zijin Feng +4
Discrete diffusion models have recently shown great promise for modeling complex discrete data, with masked diffusion models (MDMs) offering a compelling trade-off between quality…
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
Brian K Chen, Tianyang Hu, Hui Jin +2
In-Context Learning (ICL) has been a powerful emergent property of large language models that has attracted increasing attention in recent years. In contrast to regular gradient-ba…
Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion
Xuantong Liu, Tianyang Hu, Wenjia Wang +2
As a dominant force in text-to-image generation tasks, Diffusion Probabilistic Models (DPMs) face a critical challenge in controllability, struggling to adhere strictly to complex,…
The Surprising Effectiveness of Skip-Tuning in Diffusion Sampling
Jiajun Ma, Shuchen Xue, Tianyang Hu +5
With the incorporation of the UNet architecture, diffusion probabilistic models have become a dominant force in image generation tasks. One key design in UNet is the skip connectio…
AlgoFormer: An Efficient Transformer Framework with Algorithmic Structures
Yihang Gao, Chuanyang Zheng, Enze Xie +6
Besides natural language processing, transformers exhibit extraordinary performance in solving broader applications, including scientific computing and computer vision. Previous wo…
Elucidating The Design Space of Classifier-Guided Diffusion Generation
Jiajun Ma, Tianyang Hu, Wenjia Wang +1
Guidance in conditional diffusion generation is of great importance for sample quality and controllability. However, existing guidance schemes are to be desired. On one hand, mains…
Training Energy-Based Models with Diffusion Contrastive Divergences
Weijian Luo, Hao Jiang, Tianyang Hu +3
Energy-Based Models (EBMs) have been widely used for generative modeling. Contrastive Divergence (CD), a prevailing training objective for EBMs, requires sampling from the EBM with…
Complexity Matters: Rethinking the Latent Space for Generative Modeling
Tianyang Hu, Fei Chen, Haonan Wang +4
In generative modeling, numerous successful approaches leverage a low-dimensional latent space, e.g., Stable Diffusion models the latent space induced by an encoder and generates i…
Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalization
Yimeng Chen, Tianyang Hu, Fengwei Zhou +2
The proliferation of pretrained models, as a result of advancements in pretraining techniques, has led to the emergence of a vast zoo of publicly available models. Effectively util…
Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models
Weijian Luo, Tianyang Hu, Shifeng Zhang +3
Due to the ease of training, ability to scale, and high sample quality, diffusion models (DMs) have become the preferred option for generative modeling, with numerous pre-trained m…
Inducing Neural Collapse in Deep Long-tailed Learning
Xuantong Liu, Jianfeng Zhang, Tianyang Hu +3
Although deep neural networks achieve tremendous success on various classification tasks, the generalization ability drops sheer when training datasets exhibit long-tailed distribu…
Deciphering the Projection Head: Representation Evaluation Self-supervised Learning
Jiajun Ma, Tianyang Hu, Wenjia Wang
Self-supervised learning (SSL) aims to learn intrinsic features without labels. Despite the diverse architectures of SSL methods, the projection head always plays an important role…
ZooD: Exploiting Model Zoo for Out-of-Distribution Generalization
Qishi Dong, Awais Muhammad, Fengwei Zhou +5
Recent advances on large-scale pre-training have shown great potentials of leveraging a large set of Pre-Trained Models (PTMs) for improving Out-of-Distribution (OoD) generalizatio…
Continual Learning by Modeling Intra-Class Variation
Longhui Yu, Tianyang Hu, Lanqing Hong +3
It has been observed that neural networks perform poorly when the data or tasks are presented sequentially. Unlike humans, neural networks suffer greatly from catastrophic forgetti…
Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding
Tianyang Hu, Zhili Liu, Fengwei Zhou +2
Contrastive learning, especially self-supervised contrastive learning (SSCL), has achieved great success in extracting powerful features from unlabeled data. In this work, we contr…