34 citations · 57 across the 8 of their papers we have counts for
8 papers
When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation
Yinuo Jiang, Yongjie Ye, Zhou Tao +4
On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals. Recent selective OPD methods improve…
Advancing biomolecular understanding and design following human instructions
Xiang Zhuang, Keyan Ding, Tianwen Lyu +9
Understanding and designing biomolecules, such as proteins and small molecules, is central to advancing drug discovery, synthetic biology and enzyme engineering. Recent breakthroug…
StableMask: Refining Causal Masking in Decoder-only Transformer
Qingyu Yin, Xuzheng He, Xiang Zhuang +4
The decoder-only Transformer architecture with causal masking and relative position encoding (RPE) has become the de facto choice in language modeling. Despite its exceptional perf…
Scientific Large Language Models: A Survey on Biological & Chemical Domains
Qiang Zhang, Keyang Ding, Tianwen Lyv +22
Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension, representing a significant stride toward artificial general intelli…
Learning Invariant Molecular Representation in Latent Discrete Space
Xiang Zhuang, Qiang Zhang, Keyan Ding +5
Molecular representation learning lays the foundation for drug discovery. However, existing methods suffer from poor out-of-distribution (OOD) generalization, particularly when dat…
InstructProtein: Aligning Human and Protein Language via Knowledge Instruction
Zeyuan Wang, Qiang Zhang, Keyan Ding +4
Large Language Models (LLMs) have revolutionized the field of natural language processing, but they fall short in comprehending biological sequences such as proteins. To address th…
Molecular Contrastive Learning with Chemical Element Knowledge Graph
Yin Fang, Qiang Zhang, Haihong Yang +7
Molecular representation learning contributes to multiple downstream tasks such as molecular property prediction and drug design. To properly represent molecules, graph contrastive…
Benchmarking Knowledge-driven Zero-shot Learning
Yuxia Geng, Jiaoyan Chen, Xiang Zhuang +5
External knowledge (a.k.a. side information) plays a critical role in zero-shot learning (ZSL) which aims to predict with unseen classes that have never appeared in training data.…