4 papers
Theoretical Analysis of Sparse Optimization with Reparameterization, Weight Decay, and Adaptive Learning Rate
Huangyu Xu, Jingqin Yang, Qianqian Xu +1
Sparse optimization is a fundamental challenge in various practical applications. A popular approach to sparse optimization is regularization. However, it may encounter op…
Generalization Bounds of Stochastic Gradient Descent in Homogeneous Neural Networks
Wenquan Ma, Yang Sui, Jiaye Teng +3
Algorithmic stability is among the most potent techniques in generalization analysis. However, its derivation usually requires a stepsize under non-convex…
Information Flow in Self-Supervised Learning
Zhiquan Tan, Jingqin Yang, Weiran Huang +2
In this paper, we conduct a comprehensive analysis of two dual-branch (Siamese architecture) self-supervised learning approaches, namely Barlow Twins and spectral contrastive learn…
Cumulative Reasoning with Large Language Models
Yifan Zhang, Jingqin Yang, Yang Yuan +1
Recent advancements in large language models (LLMs) have shown remarkable progress, yet their ability to solve complex problems remains limited. In this work, we introduce Cumulati…