1.9k citations · 2.7k across the 13 of their papers we have counts for
13 papers
What Should Data Science Education Do with Large Language Models?
Xinming Tu, James Zou, Weijie J. Su +1
The rapid advances of large language models (LLMs), such as ChatGPT, are revolutionizing data science and statistics. These state-of-the-art tools can streamline complex processes.…
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory
Xizhou Zhu, Yuntao Chen, Hao Tian +10
The captivating realm of Minecraft has attracted substantial research interest in recent years, serving as a rich platform for developing intelligent agents capable of functioning…
Reward Collapse in Aligning Large Language Models
Ziang Song, Tianle Cai, Jason D. Lee +1
The extraordinary capabilities of large language models (LLMs) such as ChatGPT and GPT-4 are in part unleashed by aligning them with reward models that are trained on human prefere…
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information
Weijie Su, Xizhou Zhu, Chenxin Tao +7
To effectively exploit the potential of large-scale models, various pre-training strategies supported by massive data from different sources are proposed, including supervised pre-…
A Law of Data Separation in Deep Learning
Hangfeng He, Weijie J. Su
While deep learning has enabled significant advances in many areas of science, its black-box nature hinders architecture design for future artificial intelligence applications and…
FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Data
Zhun Deng, Jiayao Zhang, Linjun Zhang +4
Algorithmic fairness plays an important role in machine learning and imposing fairness constraints during learning is a common approach. However, many datasets are imbalanced in ce…
Siamese Image Modeling for Self-Supervised Vision Representation Learning
Chenxin Tao, Xizhou Zhu, Weijie Su +6
Self-supervised learning (SSL) has delivered superior performance on a variety of downstream vision tasks. Two main-stream SSL frameworks have been proposed, i.e., Instance Discrim…
ROCK: Causal Inference Principles for Reasoning about Commonsense Causality
Jiayao Zhang, Hongming Zhang, Weijie J. Su +1
Commonsense causality reasoning (CCR) aims at identifying plausible causes and effects in natural language descriptions that are deemed reasonable by an average person. Although be…
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations
Jiayao Zhang, Hua Wang, Weijie J. Su
Understanding the training dynamics of deep learning models is perhaps a necessary step toward demystifying the effectiveness of these models. In particular, how do data from diffe…
You Are the Best Reviewer of Your Own Papers: An Owner-Assisted Scoring Mechanism
Weijie J. Su
I consider a setting where reviewers offer very noisy scores for several items for the selection of high-quality ones (e.g., peer review of large conference proceedings), whereas t…
An Unconstrained Layer-Peeled Perspective on Neural Collapse
Wenlong Ji, Yiping Lu, Yiliang Zhang +2
Neural collapse is a highly symmetric geometric pattern of neural networks that emerges during the terminal phase of training, with profound implications on the generalization perf…
Deformable DETR: Deformable Transformers for End-to-End Object Detection
Xizhou Zhu, Weijie Su, Lewei Lu +3
DETR has been recently proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance. However, it suffers from slow conv…
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Weijie Su, Xizhou Zhu, Yue Cao +4
We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short). VL-BERT adopts the simple yet powerful Trans…