2 citations · 4 across the 6 of their papers we have counts for
6 papers
Fine-Tuning Low-Bit Models with Gradient in Quantized Code Space
Shiguang Wu, Zhouchen Lin, Quanming Yao
Fine-tuning Low-bit models aims to adapt a quantized model while keeping the final deployed checkpoint in the same low-bit form. This setting is practically important as it reduces…
Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers
Yihong Chen, Zhouchen Lin, Quanming Yao
Attention sinks and massive activations are recurring and closely related phenomena in Transformer models. Existing explanations have largely focused on the forward pass, yet in pr…
Learning to Learn with Contrastive Meta-Objective
Shiguang Wu, Yaqing Wang, Yatao Bian +1
Meta-learning enables learning systems to adapt quickly to new tasks, similar to humans. Different meta-learning approaches all work under/with the mini-batch episodic training fra…
UniMoT: Unified Molecule-Text Language Model with Discrete Token Representation
Shuhan Guo, Yatao Bian, Ruibing Wang +3
The remarkable success of Large Language Models (LLMs) across diverse tasks has driven the research community to extend their capabilities to molecular applications. However, most…
HIGHT: Hierarchical Graph Tokenization for Molecule-Language Alignment
Yongqiang Chen, Quanming Yao, Juzheng Zhang +2
Recently, there has been a surge of interest in extending the success of large language models (LLMs) from texts to molecules. Most existing approaches adopt a graph neural network…
Graph Unitary Message Passing
Haiquan Qiu, Quanming Yao
Unitarity is a useful principle for stabilizing deep neural networks, but in graph neural networks (GNNs) instability is induced not only by learnable parameters but also by the gr…