182 citations · 232 across the 9 of their papers we have counts for
5 papers · 1 filter
ModuLoRA: Finetuning 2-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers
Junjie Yin, Jiahao Dong, Yingheng Wang +2
We propose a memory-efficient finetuning algorithm for large language models (LLMs) that supports finetuning LLMs with 65B parameters in 2/3/4-bit precision on as little as one 24G…
MHub: Unlocking the Potential of Machine Learning for Materials Discovery
Yuanqi Du, Yingheng Wang, Yining Huang +6
We introduce MHub, a toolkit for advancing machine learning in materials discovery. Machine learning has achieved remarkable progress in modeling molecular structures, especial…
InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models
Yingheng Wang, Yair Schiff, Aaron Gokaslan +4
While diffusion models excel at generating high-quality samples, their latent variables typically lack semantic meaning and are not suitable for representation learning. Here, we p…
Time Series Contrastive Learning with Information-Aware Augmentations
Dongsheng Luo, Wei Cheng, Yingheng Wang +8
Various contrastive learning approaches have been proposed in recent years and achieve significant empirical success. While effective and prevalent, contrastive learning has been l…
Xtal2DoS: Attention-based Crystal to Sequence Learning for Density of States Prediction
Junwen Bai, Yuanqi Du, Yingheng Wang +3
Modern machine learning techniques have been extensively applied to materials science, especially for property prediction tasks. A majority of these methods address scalar property…