Efficient Text Encoders for Labor Market Analysis

Abstract

Labor market analysis relies on extracting insights from job advertisements, which provide valuable yet unstructured information on job titles and corresponding skill requirements. While state-of-the-art methods for skill extraction achieve strong performance, they depend on large language models (LLMs), which are computationally expensive and slow. In this paper, we propose \textbf{ConTeXT-match}, a novel contrastive learning approach with token-level attention that is well-suited for the extreme multi-label classification task of skill classification. \textbf{ConTeXT-match} significantly improves skill extraction efficiency and performance, achieving state-of-the-art results with a lightweight bi-encoder model. To support robust evaluation, we introduce \textbf{Skill-XL}, a new benchmark with exhaustive, sentence-level skill annotations that explicitly address the redundancy in the large label space. Finally, we present \textbf{JobBERT V2}, an improved job title normalization model that leverages extracted skills to produce high-quality job title representations. Experiments demonstrate that our models are efficient, accurate, and scalable, making them ideal for large-scale, real-time labor market analysis.

This work has been submitted to the IEEE for possible publication

References in corpus (8)

• Representation Learning with Contrastive Predictive Coding
• DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents
• Diversity Subsampling: Custom Subsamples from Large Data Sets
• Large Language Models as Batteries-Included Zero-Shot ESCO Skills Matchers
• Design of Negative Sampling Strategies for Distantly Supervised Skill Extraction
• In-Context Learning for Extreme Multi-Label Classification
• VacancySBERT: the approach for representation of titles and skills for semantic similarity search in the recruitment domain
• Skill-LLM: Repurposing General-Purpose LLMs for Skill Extraction