182 citations · 232 across the 9 of their papers we have counts for
6 papers · 1 filter
ModuLoRA: Finetuning 2-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers
Junjie Yin, Jiahao Dong, Yingheng Wang +2
We propose a memory-efficient finetuning algorithm for large language models (LLMs) that supports finetuning LLMs with 65B parameters in 2/3/4-bit precision on as little as one 24G…
InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models
Yingheng Wang, Yair Schiff, Aaron Gokaslan +4
While diffusion models excel at generating high-quality samples, their latent variables typically lack semantic meaning and are not suitable for representation learning. Here, we p…
Time Series Contrastive Learning with Information-Aware Augmentations
Dongsheng Luo, Wei Cheng, Yingheng Wang +8
Various contrastive learning approaches have been proposed in recent years and achieve significant empirical success. While effective and prevalent, contrastive learning has been l…
Xtal2DoS: Attention-based Crystal to Sequence Learning for Density of States Prediction
Junwen Bai, Yuanqi Du, Yingheng Wang +3
Modern machine learning techniques have been extensively applied to materials science, especially for property prediction tasks. A majority of these methods address scalar property…
Going Deeper into Permutation-Sensitive Graph Neural Networks
Zhongyu Huang, Yingheng Wang, Chaozhuo Li +1
The invariance to permutations of the adjacency matrix, i.e., graph isomorphism, is an overarching requirement for Graph Neural Networks (GNNs). Conventionally, this prerequisite c…
Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction
Yingheng Wang, Yaosen Min, Xin Chen +1
Drug-drug interaction(DDI) prediction is an important task in the medical health machine learning community. This study presents a new method, multi-view graph contrastive represen…