22 citations · 100 across the 15 of their papers we have counts for
15 papers
XTab: Cross-table Pretraining for Tabular Transformers
Bingzhao Zhu, Xingjian Shi, Nick Erickson +3
The success of self-supervised learning in computer vision and natural language processing has motivated pretraining methods on tabular data. However, most existing tabular self-su…
Automated Few-shot Classification with Instruction-Finetuned Language Models
Rami Aly, Xingjian Shi, Kaixiang Lin +2
A particularly successful class of approaches for few-shot learning combines language models with prompts -- hand-crafted task descriptions that complement data samples. However, d…
Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation
Yuxin Ren, Zihan Zhong, Xingjian Shi +3
It has been commonly observed that a teacher model with superior performance does not necessarily result in a stronger student, highlighting a discrepancy between current teacher t…
LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation
Jiaxin Cheng, Xiao Liang, Xingjian Shi +3
Layout-to-image generation refers to the task of synthesizing photo-realistic images based on semantic layouts. In this paper, we propose LayoutDiffuse that adapts a foundational d…
Towards Geospatial Foundation Models via Continual Pretraining
Matias Mendieta, Boran Han, Xingjian Shi +2
Geospatial technologies are becoming increasingly essential in our world for a wide range of applications, including agriculture, urban planning, and disaster response. To help imp…
Parameter-Efficient Fine-Tuning Design Spaces
Jiaao Chen, Aston Zhang, Xingjian Shi +3
Parameter-efficient fine-tuning aims to achieve performance comparable to fine-tuning, using fewer trainable parameters. Several strategies (e.g., Adapters, prefix tuning, BitFit,…
SPT: Semi-Parametric Prompt Tuning for Multitask Prompted Learning
M Saiful Bari, Aston Zhang, Shuai Zheng +4
Pre-trained large language models can efficiently interpolate human-written prompts in a natural way. Multitask prompted learning can help generalization through a diverse set of t…
Learning Multimodal Data Augmentation in Feature Space
Zichang Liu, Zhiqiang Tang, Xingjian Shi +4
The ability to jointly learn from multiple modalities, such as text, audio, and visual data, is a defining feature of intelligent systems. While there have been promising advances…
Benchmarking Robustness of Multimodal Image-Text Models under Distribution Shift
Jielin Qiu, Yi Zhu, Xingjian Shi +5
Multimodal image-text models have shown remarkable performance in the past few years. However, evaluating robustness against distribution shifts is crucial before adopting them in…
Visual Prompt Tuning for Test-time Domain Adaptation
Yunhe Gao, Xingjian Shi, Yi Zhu +5
Models should be able to adapt to unseen data during test-time to avoid performance drops caused by inevitable distribution shifts in real-world deployment scenarios. In this work,…
A Transformer-Based Substitute Recommendation Model Incorporating Weakly Supervised Customer Behavior Data
Wenting Ye, Hongfei Yang, Shuai Zhao +3
The substitute-based recommendation is widely used in E-commerce to provide better alternatives to customers. However, existing research typically uses the customer behavior signal…
Removing Batch Normalization Boosts Adversarial Training
Haotao Wang, Aston Zhang, Shuai Zheng +3
Adversarial training (AT) defends deep neural networks against adversarial attacks. One challenge that limits its practical application is the performance degradation on clean samp…
Fix Bugs with Transformer through a Neural-Symbolic Edit Grammar
Yaojie Hu, Xingjian Shi, Qiang Zhou +1
We introduce NSEdit (neural-symbolic edit), a novel Transformer-based code repair method. Given only the source code that contains bugs, NSEdit predicts an editing sequence that ca…
Benchmarking Multimodal AutoML for Tabular Data with Text Fields
Xingjian Shi, Jonas Mueller, Nick Erickson +2
We consider the use of automated supervised learning systems for data tables that not only contain numeric/categorical columns, but one or more text fields as well. Here we assembl…
Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing
Haoyu He, Xingjian Shi, Jonas Mueller +3
We aim to identify how different components in the KD pipeline affect the resulting performance and how much the optimal KD pipeline varies across different datasets/tasks, such as…