236 citations · 353 across the 21 of their papers we have counts for
14 papers · 1 filter
LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence
Xingxuan Zhang, Gang Ren, Han Yu +35
We argue that progress toward general intelligence requires complementary foundation models grounded in language, the physical world, and structured data. This report presents Limi…
Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study
Xingxuan Zhang, Haoran Wang, Jiansheng Li +6
Large language models (LLMs) like GPT-4 and LLaMA-3 utilize the powerful in-context learning (ICL) capability of Transformer architecture to learn on the fly from limited examples.…
Sample Weight Averaging for Stable Prediction
Han Yu, Yue He, Renzhe Xu +4
The challenge of Out-of-Distribution (OOD) generalization poses a foundational concern for the application of machine learning algorithms to risk-sensitive areas. Inspired by tradi…
Error Slice Discovery via Manifold Compactness
Han Yu, Hao Zou, Jiashuo Liu +4
Despite the great performance of deep learning models in many areas, they still make mistakes and underperform on certain subsets of data, i.e. error slices. Given a trained model,…
Competing for Shareable Arms in Multi-Player Multi-Armed Bandits
Renzhe Xu, Haotian Wang, Xingxuan Zhang +2
Competitions for shareable and limited resources have long been studied with strategic agents. In reality, agents often have to learn and maximize the rewards of the resources at t…
Rethinking the Evaluation Protocol of Domain Generalization
Han Yu, Xingxuan Zhang, Renzhe Xu +3
Domain generalization aims to solve the challenge of Out-of-Distribution (OOD) generalization by leveraging common knowledge learned from multiple training domains to generalize to…
Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization
Xingxuan Zhang, Renzhe Xu, Han Yu +2
Recently, flat minima are proven to be effective for improving generalization and sharpness-aware minimization (SAM) achieves state-of-the-art performance. Yet the current definiti…
Model Agnostic Sample Reweighting for Out-of-Distribution Learning
Xiao Zhou, Yong Lin, Renjie Pi +4
Distributionally robust optimization (DRO) and invariant risk minimization (IRM) are two popular methods proposed to improve out-of-distribution (OOD) generalization performance of…
Stable Learning via Sparse Variable Independence
Han Yu, Peng Cui, Yue He +4
The problem of covariate-shift generalization has attracted intensive research attention. Previous stable learning algorithms employ sample reweighting schemes to decorrelate the c…
Product Ranking for Revenue Maximization with Multiple Purchases
Renzhe Xu, Xingxuan Zhang, Bo Li +3
Product ranking is the core problem for revenue-maximizing online retailers. To design proper product ranking algorithms, various consumer choice models are proposed to characteriz…
A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization
Renzhe Xu, Xingxuan Zhang, Zheyan Shen +2
Covariate-shift generalization, a typical case in out-of-distribution (OOD) generalization, requires a good performance on the unknown test distribution, which varies from the acce…
Towards Out-Of-Distribution Generalization: A Survey
Jiashuo Liu, Zheyan Shen, Yue He +4
Traditional machine learning paradigms are based on the assumption that both training and test data follow the same statistical pattern, which is mathematically referred to as Inde…
Deep Stable Learning for Out-Of-Distribution Generalization
Xingxuan Zhang, Peng Cui, Renzhe Xu +3
Approaches based on deep neural networks have achieved striking performance when testing data and training data share similar distribution, but can significantly fail otherwise. Th…
Algorithmic Decision Making with Conditional Fairness
Renzhe Xu, Peng Cui, Kun Kuang +4
Nowadays fairness issues have raised great concerns in decision-making systems. Various fairness notions have been proposed to measure the degree to which an algorithm is unfair. I…