4 papers
Rigidity-Aware Geometric Pretraining for Protein Design and Conformational Ensembles
Zhanghan Ni, Yanjing Li, Zeju Qiu +4
Generative models have recently advanced protein design by learning the statistical regularities of natural structures. However, current approaches face three ke…
POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
Zeju Qiu, Lixin Liu, Adrian Weller +2
Efficient and stable training of large language models (LLMs) remains a core challenge in modern machine learning systems. To address this challenge, Reparameterized Orthogonal Equ…
Orthogonal Finetuning Made Scalable
Zeju Qiu, Weiyang Liu, Adrian Weller +1
Orthogonal finetuning (OFT) offers highly parameter-efficient adaptation while preventing catastrophic forgetting, but its high runtime and memory demands limit practical deploymen…
Reparameterized LLM Training via Orthogonal Equivalence Transformation
Zeju Qiu, Simon Buchholz, Tim Z. Xiao +3
While large language models (LLMs) are driving the rapid advancement of artificial intelligence, effectively and reliably training these large models remains one of the field's mos…