37 citations · 97 across the 26 of their papers we have counts for
28 papers
SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs
Ruyue Liu, Rong Yin, Xiangzhen Bo +5
Large scale pretrained models have revolutionized Natural Language Processing (NLP) and Computer Vision (CV), showcasing remarkable cross domain generalization abilities. However,…
Multi-Modal Molecular Representation Learning via Structure Awareness
Rong Yin, Ruyue Liu, Xiaoshuai Hao +4
Accurate extraction of molecular representations is a critical step in the drug discovery process. In recent years, significant progress has been made in molecular representation l…
AS-GCL: Asymmetric Spectral Augmentation on Graph Contrastive Learning
Ruyue Liu, Rong Yin, Yong Liu +4
Graph Contrastive Learning (GCL) has emerged as the foremost approach for self-supervised learning on graph-structured data. GCL reduces reliance on labeled data by learning robust…
Communication-Efficient Personalized Federal Graph Learning via Low-Rank Decomposition
Ruyue Liu, Rong Yin, Xiangzhen Bo +5
Federated graph learning (FGL) has gained significant attention for enabling heterogeneous clients to process their private graph data locally while interacting with a centralized…
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
Xunyu Zhu, Jian Li, Can Ma +1
Large Language Models (LLMs) have demonstrated exceptional proficiency in mathematical reasoning tasks due to their extensive parameter counts and training on vast datasets. Despit…
FedNS: A Fast Sketching Newton-Type Algorithm for Federated Learning
Jian Li, Yong Liu, Wei Wang +2
Recent Newton-type federated learning algorithms have demonstrated linear convergence with respect to the communication rounds. However, communicating Hessian matrices is often unf…
Distilling Mathematical Reasoning Capabilities into Small Language Models
Xunyu Zhu, Jian Li, Yong Liu +2
This work addresses the challenge of democratizing advanced Large Language Models (LLMs) by compressing their mathematical reasoning capabilities into sub-billion parameter Small L…
ASWT-SGNN: Adaptive Spectral Wavelet Transform-based Self-Supervised Graph Neural Network
Ruyue Liu, Rong Yin, Yong Liu +1
Graph Comparative Learning (GCL) is a self-supervised method that combines the advantages of Graph Convolutional Networks (GCNs) and comparative learning, making it promising for l…
A Survey on Model Compression for Large Language Models
Xunyu Zhu, Jian Li, Yong Liu +2
Large Language Models (LLMs) have transformed natural language processing tasks successfully. Yet, their large size and high computational needs pose challenges for practical use,…
Robust Neural Architecture Search
Xunyu Zhu, Jian Li, Yong Liu +1
Neural Architectures Search (NAS) becomes more and more popular over these years. However, NAS-generated models tends to suffer greater vulnerability to various malicious attacks.…
Operation-level Progressive Differentiable Architecture Search
Xunyu Zhu, Jian Li, Yong Liu +1
Differentiable Neural Architecture Search (DARTS) is becoming more and more popular among Neural Architecture Search (NAS) methods because of its high search efficiency and low com…
Improving Differentiable Architecture Search via Self-Distillation
Xunyu Zhu, Jian Li, Yong Liu +1
Differentiable Architecture Search (DARTS) is a simple yet efficient Neural Architecture Search (NAS) method. During the search stage, DARTS trains a supernet by jointly optimizing…
Towards Practical Differential Privacy in Data Analysis: Understanding the Effect of Epsilon on Utility in Private ERM
Yuzhe Li, Yong Liu, Bo Li +2
In this paper, we focus our attention on private Empirical Risk Minimization (ERM), which is one of the most commonly used data analysis method. We take the first step towards solv…
Sharper Utility Bounds for Differentially Private Models
Yilin Kang, Yong Liu, Jian Li +1
In this paper, by introducing Generalized Bernstein condition, we propose the first high probability excess population risk bound for dif…
Stability and Generalization of Differentially Private Minimax Problems
Yilin Kang, Yong Liu, Jian Li +1
In the field of machine learning, many problems can be formulated as the minimax problem, including reinforcement learning, generative adversarial networks, to just name a few. So…
Towards Sharper Utility Bounds for Differentially Private Pairwise Learning
Yilin Kang, Yong Liu, Jian Li +1
Pairwise learning focuses on learning tasks with pairwise loss functions, depends on pairs of training instances, and naturally fits for modeling relationships between pairs of sam…
Neural Architecture Optimization with Graph VAE
Jian Li, Yong Liu, Jiankun Liu +1
Due to their high computational efficiency on a continuous space, gradient optimization methods have shown great potential in the neural architecture search (NAS) domain. The mappi…
Theoretical Analysis of Divide-and-Conquer ERM: Beyond Square Loss and RKHS
Yong Liu, Lizhong Ding, Weiping Wang
Theoretical analysis of the divide-and-conquer based distributed learning with least square loss in the reproducing kernel Hilbert space (RKHS) have recently been explored within t…
Nearly Optimal Clustering Risk Bounds for Kernel K-Means
Yong Liu, Lizhong Ding, Weiping Wang
In this paper, we study the statistical properties of kernel -means and obtain a nearly optimal excess clustering risk bound, substantially improving the state-of-art bounds in…
Convolutional Spectral Kernel Learning
Jian Li, Yong Liu, Weiping Wang
Recently, non-stationary spectral kernels have drawn much attention, owing to its powerful feature representation ability in revealing long-range correlations and input-dependent c…
Input Perturbation: A New Paradigm between Central and Local Differential Privacy
Yilin Kang, Yong Liu, Ben Niu +3
Traditionally, there are two models on differential privacy: the central model and the local model. The central model focuses on the machine learning model and the local model focu…
Data Heterogeneity Differential Privacy: From Theory to Algorithm
Yilin Kang, Jian Li, Yong Liu +1
Traditionally, the random noise is equally injected when training with different data instances in the field of differential privacy (DP). In this paper, we first give sharper exce…
Weighted Distributed Differential Privacy ERM: Convex and Non-convex
Yilin Kang, Yong Liu, Weiping Wang
Distributed machine learning is an approach allowing different parties to learn a model over all data sets without disclosing their own data. In this paper, we propose a weighted d…
Automated Spectral Kernel Learning
Jian Li, Yong Liu, Weiping Wang
The generalization performance of kernel methods is largely determined by the kernel, but common kernels are stationary thus input-independent and output-independent, that limits t…