257 citations · 299 across the 19 of their papers we have counts for
16 papers · 1 filter
How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training
Yixin Ou, Yunzhi Yao, Ningyu Zhang +5
Despite exceptional capabilities in knowledge-intensive tasks, Large Language Models (LLMs) face a critical gap in understanding how they internalize new knowledge, particularly ho…
ProofAug: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis
Haoxiong Liu, Jiacheng Sun, Zhenguo Li +1
The synergy between deep learning models and traditional automation tools, such as built-in tactics of the proof assistant and off-the-shelf automated theorem provers, plays a cruc…
How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs
Guhao Feng, Kai Yang, Yuntian Gu +6
Despite the remarkable success of Transformer-based large language models (LLMs) across various domains, understanding and enhancing their mathematical capabilities remains a signi…
Elucidating The Design Space of Classifier-Guided Diffusion Generation
Jiajun Ma, Tianyang Hu, Wenjia Wang +1
Guidance in conditional diffusion generation is of great importance for sample quality and controllability. However, existing guidance schemes are to be desired. On one hand, mains…
SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models
Shuchen Xue, Mingyang Yi, Weijian Luo +4
Diffusion Probabilistic Models (DPMs) have achieved considerable success in generation tasks. As sampling from DPMs is equivalent to solving diffusion SDE or ODE which is time-cons…
Training Energy-Based Models with Diffusion Contrastive Divergences
Weijian Luo, Hao Jiang, Tianyang Hu +3
Energy-Based Models (EBMs) have been widely used for generative modeling. Contrastive Divergence (CD), a prevailing training objective for EBMs, requires sampling from the EBM with…
Complexity Matters: Rethinking the Latent Space for Generative Modeling
Tianyang Hu, Fei Chen, Haonan Wang +4
In generative modeling, numerous successful approaches leverage a low-dimensional latent space, e.g., Stable Diffusion models the latent space induced by an encoder and generates i…
On the Generalization of Diffusion Model
Mingyang Yi, Jiacheng Sun, Zhenguo Li
The diffusion probabilistic generative models are widely used to generate high-quality data. Though they can synthetic data that does not exist in the training set, the rationale b…
Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models
Weijian Luo, Tianyang Hu, Shifeng Zhang +3
Due to the ease of training, ability to scale, and high sample quality, diffusion models (DMs) have become the preferred option for generative modeling, with numerous pre-trained m…
Breaking Correlation Shift via Conditional Invariant Regularizer
Mingyang Yi, Ruoyu Wang, Jiachen Sun +2
Recently, generalization on out-of-distribution (OOD) data with correlation shift has attracted great attentions. The correlation shift is caused by the spurious attributes that co…
Layer-Parallel Training of Residual Networks with Auxiliary-Variable Networks
Qi Sun, Hexin Dong, Zewei Chen +3
Gradient-based methods for the distributed training of residual networks (ResNets) typically require a forward pass of the input data, followed by back-propagating the error gradie…
Towards Understanding the Generative Capability of Adversarially Robust Classifiers
Yao Zhu, Jiacheng Ma, Jiacheng Sun +3
Recently, some works found an interesting phenomenon that adversarially robust classifiers can generate good images comparable to generative models. We investigate this phenomenon…
Improved OOD Generalization via Adversarial Training and Pre-training
Mingyang Yi, Lu Hou, Jiacheng Sun +4
Recently, learning a model that generalizes well on out-of-distribution (OOD) data has attracted great attention in the machine learning community. In this paper, after defining OO…
Batch Group Normalization
Xiao-Yun Zhou, Jiacheng Sun, Nanyang Ye +6
Deep Convolutional Neural Networks (DCNNs) are hard and time-consuming to train. Normalization is one of the effective solutions. Among previous normalization methods, Batch Normal…
A Practical Layer-Parallel Training Algorithm for Residual Networks
Qi Sun, Hexin Dong, Zewei Chen +5
Gradient-based algorithms for training ResNets typically require a forward pass of the input data, followed by back-propagating the objective gradient to update parameters, which a…
New Interpretations of Normalization Methods in Deep Learning
Jiacheng Sun, Xiangyong Cao, Hanwen Liang +3
In recent years, a variety of normalization methods have been proposed to help train neural networks, such as batch normalization (BN), layer normalization (LN), weight normalizati…