BloombergGPT: A Large Language Model for Finance

Abstract

The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering. Large Language Models (LLMs) have been shown to be effective on a variety of tasks; however, no LLM specialized for the financial domain has been reported in literature. In this work, we present BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data. We construct a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet, augmented with 345 billion tokens from general purpose datasets. We validate BloombergGPT on standard LLM benchmarks, open financial benchmarks, and a suite of internal benchmarks that most accurately reflect our intended usage. Our mixed dataset training leads to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks. Additionally, we explain our modeling choices, training process, and evaluation methodology. We release Training Chronicles (Appendix C) detailing our experience in training BloombergGPT.

arXiv commentUpdated to include Training Chronicles (Appendix C)

Cited by in corpus (17)

• A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
• Recommender Systems in the Era of Large Language Models (LLMs)
• Transforming Sentiment Analysis in the Financial Domain with ChatGPT
• Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study
• LLM for SoC Security: A Paradigm Shift
• A Scoping Review of ChatGPT Research in Accounting and Finance
• Designing Heterogeneous LLM Agents for Financial Sentiment Analysis
• Large Process Models: A Vision for Business Process Management in the Age of Generative AI
• Fine-tuning and Utilization Methods of Domain-specific LLMs
• Towards Automated Regulatory Compliance Verification in Financial Auditing with Large Language Models
• Several categories of Large Language Models (LLMs): A Short Survey
• Geo-FuB: A Method for Constructing an Operator-Function Knowledge Base for Geospatial Code Generation Tasks Using Large Language Models
• CANAL -- Cyber Activity News Alerting Language Model: Empirical Approach vs. Expensive LLM
• Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models
• FANAL -- Financial Activity News Alerting Language Modeling Framework
• Enhancing Financial Domain Adaptation of Language Models via Model Augmentation
• CompanyKG: A Large-Scale Heterogeneous Graph for Company Similarity Quantification