111 citations · 753 across the 34 of their papers we have counts for
34 papers
Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks
Mingze Wang, Chao Ma
The training process of ReLU neural networks often exhibits complicated nonlinear phenomena. The nonlinearity of models and non-convexity of loss pose significant challenges for th…
Understanding Causality with Large Language Models: Feasibility and Opportunities
Cheng Zhang, Stefan Bauer, Paul Bennett +8
We assess the ability of large language models (LLMs) to answer causal questions by analyzing their strengths and weaknesses against three types of causal question. We believe that…
Causal Reasoning in the Presence of Latent Confounders via Neural ADMG Learning
Matthew Ashman, Chao Ma, Agrin Hilmkil +2
Latent confounding has been a long-standing obstacle for causal reasoning from observational data. One popular approach is to model the data using acyclic directed mixed graphs (AD…
Causal-Discovery Performance of ChatGPT in the context of Neuropathic Pain Diagnosis
Ruibo Tu, Chao Ma, Cheng Zhang
ChatGPT has demonstrated exceptional proficiency in natural language conversation, e.g., it can answer a wide range of questions while no previous large language models can. Thus,…
Revealing Spatial-temporal Taxi Demand Patterns after Vaccination in COVID-19 Pandemic
Zihao Li, Cheng Zhang, Xiaoqiang Kong +2
The COVID-19 pandemic has had an unprecedented impact on our daily lives. With the increase in vaccination rate, normalcy gradually returns, so is the taxi demand. However, the cha…
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks
Daniel Kunin, Atsushi Yamamura, Chao Ma +1
In this work, we explore the maximum-margin bias of quasi-homogeneous neural networks trained with gradient flow on an exponential loss and past a point of separability. We introdu…
Correcting Convexity Bias in Function and Functional Estimate
Chao Ma, Lexing Ying
A general framework with a series of different methods is proposed to improve the estimate of convex function (or functional) values when only noisy observations of the true input…
Generalization Error Bounds for Deep Neural Networks Trained by SGD
Mingze Wang, Chao Ma
Generalization error bounds for deep neural networks trained by stochastic gradient descent (SGD) are derived by combining a dynamical control of an appropriate parameter norm and…
Early Stage Convergence and Global Convergence of Training Mildly Parameterized Neural Networks
Mingze Wang, Chao Ma
The convergence of GD and SGD when training mildly parameterized neural networks starting from random initialization is studied. For a broad range of models and loss functions, inc…
Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes
Chao Ma, Daniel Kunin, Lei Wu +1
A quadratic approximation of neural network loss landscapes has been extensively used to study the optimization process of these networks. Though, it usually holds in a very small…
Provably convergent quasistatic dynamics for mean-field two-player zero-sum games
Chao Ma, Lexing Ying
In this paper, we study the problem of finding mixed Nash equilibrium for mean-field two-player zero-sum games. Solving this problem requires optimizing over two probability distri…
Identifiable Generative Models for Missing Not at Random Data Imputation
Chao Ma, Cheng Zhang
Real-world datasets often have missing values associated with complex generative processes, where the cause of the missingness may not be fully observed. This is known as missing n…
A Riemannian Mean Field Formulation for Two-layer Neural Networks with Batch Normalization
Chao Ma, Lexing Ying
The training dynamics of two-layer neural networks with batch normalization (BN) is studied. It is written as the training dynamics of a neural network without BN on a Riemannian m…
On Linear Stability of SGD and Input-Smoothness of Neural Networks
Chao Ma, Lexing Ying
The multiplicative structure of parameters and input data in the first layer of neural networks is explored to build connection between the landscape of the loss function with resp…
Achieving Adversarial Robustness Requires An Active Teacher
Chao Ma, Lexing Ying
A new understanding of adversarial examples and adversarial robustness is proposed by decoupling the data generator and the label generator (which we call the teacher). In our fram…
Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning
Pan Zhou, Jiashi Feng, Chao Ma +3
It is not clear yet why ADAM-alike adaptive gradient algorithms suffer from worse generalization performance than SGD despite their faster training speed. This work aims to provide…
Complexity Measures for Neural Networks with General Activation Functions Using Path-based Norms
Zhong Li, Chao Ma, Lei Wu
A simple approach is proposed to obtain complexity controls for neural networks with general activation functions. The approach is motivated by approximating the general activation…
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't
Weinan E, Chao Ma, Stephan Wojtowytsch +1
The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-ba…
A Qualitative Study of the Dynamic Behavior for Adaptive Gradient Algorithms
Chao Ma, Lei Wu, Weinan E
The dynamic behavior of RMSprop and Adam algorithms is studied through a combination of careful numerical experiments and theoretical explanations. Three types of qualitative featu…
The Slow Deterioration of the Generalization Error of the Random Feature Model
Chao Ma, Lei Wu, Weinan E
The random feature model exhibits a kind of resonance behavior when the number of parameters is close to the training sample size. This behavior is characterized by the appearance…
The Quenching-Activation Behavior of the Gradient Descent Dynamics for Two-layer Neural Network Models
Chao Ma, Lei Wu, Weinan E
A numerical and phenomenological study of the gradient descent (GD) algorithm for training two-layer neural network models is carried out for different parameter regimes when the t…
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth
Yiping Lu, Chao Ma, Yulong Lu +2
Training deep neural networks with stochastic gradient descent (SGD) can often achieve zero training loss on real-world tasks although the optimization landscape is known to be hig…
Machine Learning from a Continuous Viewpoint
Weinan E, Chao Ma, Lei Wu
We present a continuous formulation of machine learning, as a problem in the calculus of variations and differential-integral equations, in the spirit of classical numerical analys…
The Generalization Error of the Minimum-norm Solutions for Over-parameterized Neural Networks
Weinan E, Chao Ma, Lei Wu
We study the generalization properties of minimum-norm solutions for three over-parametrized machine learning models including the random feature model, the two-layer neural networ…