13 papers
Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
Tianqi Du, Qi Zhang, Yifei Wang +1
Recently, world models have emerged as a promising paradigm for building intelligent agents by learning predictive models that estimate future environment states conditioned on obs…
Advancing Ligand-based Virtual Screening and Molecular Generation with Pretrained Molecular Embedding Distance
Shiyun Wa, Yifei Wang, Simone Sciabola +1
Molecular similarity plays a central role in ligand-based drug discovery, such as virtual screening, analog searching, and goal-directed molecular generation. However, traditional…
On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedy
Jingyi Cui, Qi Zhang, Yifei Wang +1
Sparse autoencoders (SAEs) have recently emerged as a powerful tool for interpreting the features learned by large language models (LLMs). By reconstructing features with sparsely…
SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training
Qi Zhang, Yifei Wang, Xiaohan Wang +4
In recent years, pre-trained large language models have achieved remarkable success across diverse tasks. Besides the pivotal role of self-supervised pre-training, their effectiven…
Autoregressive Models Rival Diffusion Models at ANY-ORDER Generation
Tianqi Du, Lizhe Fang, Weijie Yang +4
Diffusion language models enable any-order generation and bidirectional conditioning, offering appealing flexibility for tasks such as infilling, rewriting, and self-correction. Ho…
Monet: Reasoning in Latent Visual Space Beyond Images and Language
Qixun Wang, Yang Shi, Yifei Wang +5
"Thinking with images" has emerged as an effective paradigm for advancing visual reasoning, extending beyond text-only chains of thought by injecting visual evidence into intermedi…