37 citations · 67 across the 16 of their papers we have counts for
17 papers
Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillation
Xunyu Zhu, Jian Li, Can Ma +1
Large Language Models (LLMs) demonstrate exceptional reasoning capabilities, often achieving state-of-the-art performance in various tasks. However, their substantial computational…
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
Xunyu Zhu, Jian Li, Can Ma +1
Large Language Models (LLMs) have demonstrated exceptional proficiency in mathematical reasoning tasks due to their extensive parameter counts and training on vast datasets. Despit…
FedNS: A Fast Sketching Newton-Type Algorithm for Federated Learning
Jian Li, Yong Liu, Wei Wang +2
Recent Newton-type federated learning algorithms have demonstrated linear convergence with respect to the communication rounds. However, communicating Hessian matrices is often unf…
Distilling Mathematical Reasoning Capabilities into Small Language Models
Xunyu Zhu, Jian Li, Yong Liu +2
This work addresses the challenge of democratizing advanced Large Language Models (LLMs) by compressing their mathematical reasoning capabilities into sub-billion parameter Small L…
A Survey on Model Compression for Large Language Models
Xunyu Zhu, Jian Li, Yong Liu +2
Large Language Models (LLMs) have transformed natural language processing tasks successfully. Yet, their large size and high computational needs pose challenges for practical use,…
Robust Neural Architecture Search
Xunyu Zhu, Jian Li, Yong Liu +1
Neural Architectures Search (NAS) becomes more and more popular over these years. However, NAS-generated models tends to suffer greater vulnerability to various malicious attacks.…
Improving Differentiable Architecture Search via Self-Distillation
Xunyu Zhu, Jian Li, Yong Liu +1
Differentiable Architecture Search (DARTS) is a simple yet efficient Neural Architecture Search (NAS) method. During the search stage, DARTS trains a supernet by jointly optimizing…
Ridgeless Regression with Random Features
Jian Li, Yong Liu, Yingying Zhang
Recent theoretical studies illustrated that kernel ridgeless regression can guarantee good generalization ability without an explicit regularization. In this paper, we investigate…
Sharper Utility Bounds for Differentially Private Models
Yilin Kang, Yong Liu, Jian Li +1
In this paper, by introducing Generalized Bernstein condition, we propose the first high probability excess population risk bound for dif…
Stability and Generalization of Differentially Private Minimax Problems
Yilin Kang, Yong Liu, Jian Li +1
In the field of machine learning, many problems can be formulated as the minimax problem, including reinforcement learning, generative adversarial networks, to just name a few. So…
Neural Architecture Optimization with Graph VAE
Jian Li, Yong Liu, Jiankun Liu +1
Due to their high computational efficiency on a continuous space, gradient optimization methods have shown great potential in the neural architecture search (NAS) domain. The mappi…
Convolutional Spectral Kernel Learning
Jian Li, Yong Liu, Weiping Wang
Recently, non-stationary spectral kernels have drawn much attention, owing to its powerful feature representation ability in revealing long-range correlations and input-dependent c…
Data Heterogeneity Differential Privacy: From Theory to Algorithm
Yilin Kang, Jian Li, Yong Liu +1
Traditionally, the random noise is equally injected when training with different data instances in the field of differential privacy (DP). In this paper, we first give sharper exce…
Automated Spectral Kernel Learning
Jian Li, Yong Liu, Weiping Wang
The generalization performance of kernel methods is largely determined by the kernel, but common kernels are stationary thus input-independent and output-independent, that limits t…
Towards Sharp Analysis for Distributed Learning with Random Features
Jian Li, Yong Liu, Weiping Wang
In recent studies, the generalization properties for distributed learning and random features assumed the existence of the target concept over the hypothesis space. However, this s…
Efficient Cross-Validation for Semi-Supervised Learning
Yong Liu, Jian Li, Guangjun Wu +2
Manifold regularization, such as laplacian regularized least squares (LapRLS) and laplacian support vector machine (LapSVM), has been widely used in semi-supervised learning, and i…
Max-Diversity Distributed Learning: Theory and Algorithms
Yong Liu, Jian Li, Weiping Wang
We study the risk performance of distributed learning for the regularization empirical risk minimization with fast convergence rate, substantially improving the error analysis of t…