1 citations · 2 across the 10 of their papers we have counts for
11 papers
Scaling Domain Data Repetition in LLM Pretraining
Jingwei Li, Xinran Gu, Rui Dai +5
As large language models scale, their training-token budgets must also increase to maintain an appropriate tokens-per-parameter ratio (\(\mathrm{TPP}\)). However, high-quality doma…
Online Scheduling with a Stochastic Signal
Romain Cosson, Jingwei Li, Alexander Lindermayr +1
Nonclairvoyant scheduling is a fundamental online model in which processing times are initially unknown to the scheduler. Unfortunately, for important objectives such as total comp…
Capacity-Aware Mixture Law Enables Efficient LLM Data Optimization
Jingwei Li, Xinran Gu, Jingzhao Zhang
A data mixture refers to how different data sources are combined to train large language models, and selecting an effective mixture is crucial for optimal downstream performance. E…
Hallucination is a Consequence of Space-Optimality: A Rate-Distortion Theorem for Membership Testing
Anxin Guo, Jingwei Li
Large language models often hallucinate with high confidence on "random facts" that lack inferable patterns. We formalize the memorization of such facts as a membership testing pro…
Finite Sample Analyses for Continuous-time Linear Systems: System Identification and Online Control
Hongyi Zhou, Jingwei Li, Jingzhao Zhang
Real world evolves in continuous time but computations are done from finite samples. Therefore, we study algorithms using finite observations in continuous-time linear dynamical sy…
Understanding Nonlinear Implicit Bias via Region Counts in Input Space
Jingwei Li, Jing Xu, Zifan Wang +2
One explanation for the strong generalization ability of neural networks is implicit bias. Yet, the definition and mechanism of implicit bias in non-linear contexts remains little…
Towards Black-Box Membership Inference Attack for Diffusion Models
Jingwei Li, Jing Dong, Tianxing He +1
Given the rising popularity of AI-generated art and the associated copyright concerns, identifying whether an artwork was used to train a diffusion model is an important research t…
To Store or Not to Store: a graph theoretical approach for Dataset Versioning
Anxin Guo, Jingwei Li, Pattara Sukprasert +3
In this work, we study the cost efficient data versioning problem, where the goal is to optimize the storage and reconstruction (retrieval) costs of data versions, given a graph of…
Iteratively Learn Diverse Strategies with State Distance Information
Wei Fu, Weihua Du, Jingwei Li +3
In complex reinforcement learning (RL) problems, policies with similar rewards may have substantially different behaviors. It remains a fundamental challenge to optimize rewards wh…
Online Control with Adversarial Disturbance for Continuous-time Linear Systems
Jingwei Li, Jing Dong, Can Chang +2
We study online control for continuous-time linear systems with finite sampling rates, where the objective is to design an online procedure that learns under non-stochastic noise a…
Online Policy Optimization for Robust MDP
Jing Dong, Jingwei Li, Baoxiang Wang +1
Reinforcement learning (RL) has exceeded human performance in many synthetic settings such as video games and Go. However, real-world deployment of end-to-end RL models is less com…