8 papers
LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models
Yuanrui Wang, Xingxuan Zhang, Han Yu +7
Tabular foundation models (TFMs) increasingly rival tree ensembles, but their performance is often compute-inefficient: with standard affine scalar tokenization, each feature injec…
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
DeepSeek-AI, Anyi Xu, Bangcai Lin +315
We present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models -- DeepSeek-V4-Pro with 1.6T parameters (49B activated) and DeepSe…
Generating Risky Samples with Conformity Constraints via Diffusion Models
Han Yu, Hao Zou, Xingxuan Zhang +4
Although neural networks achieve promising performance in many tasks, they may still fail when encountering some examples and bring about risks to applications. To discover risky s…
Error Slice Discovery via Manifold Compactness
Han Yu, Hao Zou, Jiashuo Liu +4
Despite the great performance of deep learning models in many areas, they still make mistakes and underperform on certain subsets of data, i.e. error slices. Given a trained model,…
LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence
Xingxuan Zhang, Gang Ren, Han Yu +35
We argue that progress toward general intelligence requires complementary foundation models grounded in language, the physical world, and structured data. This report presents Limi…
ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
Han Yu, Kehan Li, Dongbai Li +3
Recently, there has been gradually more attention paid to Out-of-Distribution (OOD) performance prediction, whose goal is to predict the performance of trained models on unlabeled…