Improving the Capabilities of Large Language Model Based Marketing Analytics Copilots With Semantic Search And Fine-Tuning

Abstract

Artificial intelligence (AI) is widely deployed to solve problems related to marketing attribution and budget optimization. However, AI models can be quite complex, and it can be difficult to understand model workings and insights without extensive implementation teams. In principle, recently developed large language models (LLMs), like GPT-4, can be deployed to provide marketing insights, reducing the time and effort required to make critical decisions. In practice, there are substantial challenges that need to be overcome to reliably use such models. We focus on domain-specific question-answering, SQL generation needed for data retrieval, and tabular analysis and show how a combination of semantic search, prompt engineering, and fine-tuning can be applied to dramatically improve the ability of LLMs to execute these tasks accurately. We compare both proprietary models, like GPT-4, and open-source models, like Llama-2-70b, as well as various embedding methods. These models are tested on sample use cases specific to marketing mix modeling and attribution.

16 pages, 5 figures, presented at the 2nd International Conference on NLP & AI (NLPAI 2024)

References in corpus (11)

• Training language models to follow instructions with human feedback
• How transferable are features in deep neural networks?
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
• LaMDA: Language Models for Dialog Applications
• The Impact of AI on Developer Productivity: Evidence from GitHub Copilot
• Improving alignment of dialogue agents via targeted human judgements
• BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
• Compositional Semantic Parsing with Large Language Models
• TableGPT: Towards Unifying Tables, Nature Language and Commands into One GPT
• Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generation