TabLLM: Few-shot Classification of Tabular Data with Large Language Models

Abstract

We study the application of large language models to zero-shot and few-shot classification of tabular data. We prompt the large language model with a serialization of the tabular data to a natural-language string, together with a short description of the classification problem. In the few-shot setting, we fine-tune the large language model using some labeled examples. We evaluate several serialization methods including templates, table-to-text models, and large language models. Despite its simplicity, we find that this technique outperforms prior deep-learning-based tabular classification methods on several benchmark datasets. In most cases, even zero-shot classification obtains non-trivial performance, illustrating the method's ability to exploit prior knowledge encoded in large language models. Unlike many deep learning methods for tabular datasets, this approach is also competitive with strong traditional baselines like gradient-boosted trees, especially in the very-few-shot setting.

Cited by in corpus (7)

• Leveraging Large Language Models for Sequential Recommendation
• Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative Study
• Emergency Department Decision Support using Clinical Pseudo-notes
• Enhancing Trust and Safety in Digital Payments: An LLM-Powered Approach
• Initial Exploration of Zero-Shot Privacy Utility Tradeoffs in Tabular Data Using GPT-4
• MuG: A Multimodal Classification Benchmark on Game Data with Tabular, Textual, and Visual Fields
• Vectorizing string entries for data processing on tables: when are larger language models better?