Power-LLaVA: Large Language and Vision Assistant for Power Transmission Line Inspection

Abstract

The inspection of power transmission line has achieved notable achievements in the past few years, primarily due to the integration of deep learning technology. However, current inspection approaches continue to encounter difficulties in generalization and intelligence, which restricts their further applicability. In this paper, we introduce Power-LLaVA, the first large language and vision assistant designed to offer professional and reliable inspection services for power transmission line by engaging in dialogues with humans. Moreover, we also construct a large-scale and high-quality dataset specialized for the inspection task. By employing a two-stage training strategy on the constructed dataset, Power-LLaVA demonstrates exceptional performance at a comparatively low training cost. Extensive experiments further prove the great capabilities of Power-LLaVA within the realm of power transmission line inspection. Code shall be released.

References in corpus (13)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Training language models to follow instructions with human feedback
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• LLaMA: Open and Efficient Foundation Language Models
• YOLOX: Exceeding YOLO Series in 2021
• Flamingo: a Visual Language Model for Few-Shot Learning
• MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
• Instruction Tuning with GPT-4
• Language Is Not All You Need: Aligning Perception with Language Models
• InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition
• BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine
• SVIT: Scaling up Visual Instruction Tuning
• DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding