8 citations · 13 across the 3 of their papers we have counts for
3 papers
cs.CL2024
StableMask: Refining Causal Masking in Decoder-only Transformer
Qingyu Yin, Xuzheng He, Xiang Zhuang +4
The decoder-only Transformer architecture with causal masking and relative position encoding (RPE) has become the de facto choice in language modeling. Despite its exceptional perf…
cs.LG2023★ 5 cited
Learning Invariant Molecular Representation in Latent Discrete Space
Xiang Zhuang, Qiang Zhang, Keyan Ding +5
Molecular representation learning lays the foundation for drug discovery. However, existing methods suffer from poor out-of-distribution (OOD) generalization, particularly when dat…
q-bio.BM2023★ 8 cited
InstructProtein: Aligning Human and Protein Language via Knowledge Instruction
Zeyuan Wang, Qiang Zhang, Keyan Ding +4
Large Language Models (LLMs) have revolutionized the field of natural language processing, but they fall short in comprehending biological sequences such as proteins. To address th…