10 citations · 26 across the 13 of their papers we have counts for
8 papers · 1 filter
Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learning
Jikai Jin, Vasilis Syrgkanis, Sham Kakade +1
Faithful evaluation of language model capabilities is crucial for deriving actionable insights that can inform model development. However, rigorous causal evaluations in this domai…
Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking
Kaifeng Lyu, Jikai Jin, Zhiyuan Li +3
Recent work by Power et al. (2022) highlighted a surprising "grokking" phenomenon in learning arithmetic tasks: a neural net first "memorizes" the training set, resulting in perfec…
Learning Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity
Jikai Jin, Vasilis Syrgkanis
We study causal representation learning, the task of recovering high-level latent variables and their causal relationships in the form of a causal graph from low-level observed dat…
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing
Jikai Jin, Zhiyuan Li, Kaifeng Lyu +2
It is believed that Gradient Descent (GD) induces an implicit bias towards good generalization in training machine learning models. This paper provides a fine-grained analysis of t…
Minimax Optimal Kernel Operator Learning via Multilevel Training
Jikai Jin, Yiping Lu, Jose Blanchet +1
Learning mappings between infinite-dimensional function spaces has achieved empirical success in many disciplines of machine learning, including generative modeling, functional dat…
Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power
Binghui Li, Jikai Jin, Han Zhong +2
It is well-known that modern neural networks are vulnerable to adversarial examples. To mitigate this problem, a series of robust learning algorithms have been proposed. However, a…
Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis
Jikai Jin, Bohang Zhang, Haiyang Wang +1
Distributionally robust optimization (DRO) is a widely-used approach to learn models that are robust against distribution shift. Compared with the standard optimization setting, th…
Improved Analysis of Clipping Algorithms for Non-convex Optimization
Bohang Zhang, Jikai Jin, Cong Fang +1
Gradient clipping is commonly used in training deep neural networks partly due to its practicability in relieving the exploding gradient problem. Recently, \citet{zhang2019gradient…