17 papers
Bayesian Optimality of In-Context Learning with Selective State Spaces
Di Zhang, Jiaqi Xing
We propose Bayesian optimal sequential prediction as a new principle for understanding in-context learning (ICL). Unlike interpretations framing Transformers as performing implicit…
Bigger Is Safer: Provable Robustness in In-Context Learning Scales with Capacity
Di Zhang, Ningxu Zhang, Zimeng Liu
In-context learning (ICL) allows large language models to adapt to new tasks from a few examples without updating their parameters. Existing theories explain ICL by assuming the te…
AgentDevel: Reframing Self-Evolving LLM Agents as Release Engineering
Di Zhang
Recent progress in large language model (LLM) agents has largely focused on embedding self-improvement mechanisms inside the agent or searching over many concurrent variants. While…
Multi-AI Agent Framework Reveals the "Oxide Gatekeeper" in Aluminum Nanoparticle Oxidation
Yiming Lu, Tingyu Lu, Di Zhang +2
Aluminum nanoparticles (ANPs) are among the most energy-dense solid fuels, yet the atomic mechanisms governing their transition from passivated particles to explosive reactants rem…
Exact Flow Linear Attention: Exact Solution from Continuous-Time Dynamics
Jingdi Lei, Di Zhang, Soujanya Poria
In this paper, we introduce Exact Flow Linear Attention~(EFLA), an exact-flow formulation of delta-rule linear attention. We show that the delta-rule update can be interpreted as a…
Fast Bayesian Updates via Harmonic Representations
Di Zhang
Bayesian inference, while foundational to probabilistic reasoning, is often hampered by the computational intractability of posterior distributions, particularly through the challe…
Learning Geometry: A Framework for Building Adaptive Manifold Models through Metric Optimization
Di Zhang
This paper proposes a novel paradigm for machine learning that moves beyond traditional parameter optimization. Unlike conventional approaches that search for optimal parameters wi…
The Neural Differential Manifold: An Architecture with Explicit Geometric Structure
Di Zhang
This paper introduces the Neural Differential Manifold (NDM), a novel neural network architecture that explicitly incorporates geometric structure into its fundamental design. Depa…
Communication-Efficient and Memory-Aware Parallel Bootstrapping using MPI
Di Zhang
Bootstrapping is a powerful statistical resampling technique for estimating the sampling distribution of an estimator. However, its computational cost becomes prohibitive for large…
On the Optimal Representation Efficiency of Barlow Twins: An Information-Geometric Interpretation
Di Zhang
Self-supervised learning (SSL) has achieved remarkable success by learning meaningful representations without labeled data. However, a unified theoretical framework for understandi…
A Frequency-Domain Analysis of the Multi-Armed Bandit Problem: A New Perspective on the Exploration-Exploitation Trade-off
Di Zhang
The stochastic multi-armed bandit (MAB) problem is one of the most fundamental models in sequential decision-making, with the core challenge being the trade-off between exploration…
PolyKAN: A Polyhedral Analysis Framework for Provable and Approximately Optimal KAN Compression
Di Zhang
Kolmogorov-Arnold Networks (KANs) have emerged as a promising alternative to traditional Multi-Layer Perceptrons (MLPs), offering enhanced interpretability and a solid mathematical…
Source PAC Coding for Low-latency Secret Key Generation in Short Blocklength Regime
Lulu Song, Di Zhang, Tingting Zhang
Source polar coding is a potential solution for short blocklength-based low-latency key generation with limited sources, which is a critical aspect of six generation (6G) Internet…
Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
Di Zhang, Xun Wu, Shaohan Huang +6
Recent advances in reinforcement learning (RL) have substantially improved the training of large-scale language models, leading to significant gains in generation quality and reaso…
AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
Can Jin, Yang Zhou, Qixin Zhang +8
Test-time scaling strategies for Large Language Models predominantly rely on either reinforcement learning with sparse outcome rewards or search-based methods guided by static Proc…
Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adapter
Weizhi Zhong, Huan Yang, Zheng Liu +5
Personalized text-to-image generation aims to synthesize images of user-provided concepts in diverse contexts. Despite recent progress in multi-concept personalization, most are li…
The Stochastic Conjugate Subgradient Algorithm For Kernel Support Vector Machines
Di Zhang, Suvrajeet Sen
Stochastic First-Order (SFO) methods have been a cornerstone in addressing a broad spectrum of modern machine learning (ML) challenges. However, their efficacy is increasingly ques…