Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US

Abstract

The United States spends more than $1B each year on initiatives such as the American Community Survey (ACS), a labor-intensive door-to-door study that measures statistics relating to race, gender, education, occupation, unemployment, and other demographic factors. Although a comprehensive source of data, the lag between demographic changes and their appearance in the ACS can exceed half a decade. As digital imagery becomes ubiquitous and machine vision techniques improve, automated data analysis may provide a cheaper and faster alternative. Here, we present a method that determines socioeconomic trends from 50 million images of street scenes, gathered in 200 American cities by Google Street View cars. Using deep learning-based computer vision techniques, we determined the make, model, and year of all motor vehicles encountered in particular neighborhoods. Data from this census of motor vehicles, which enumerated 22M automobiles in total (8% of all automobiles in the US), was used to accurately estimate income, race, education, and voting patterns, with single-precinct resolution. (The average US precinct contains approximately 1000 people.) The resulting associations are surprisingly simple and powerful. For instance, if the number of sedans encountered during a 15-minute drive through a city is higher than the number of pickup trucks, the city is likely to vote for a Democrat during the next Presidential election (88% chance); otherwise, it is likely to vote Republican (82%). Our results suggest that automated systems for monitoring demographic trends may effectively complement labor-intensive approaches, with the potential to detect trends with fine spatial resolution, in close to real time.

41 pages including supplementary material. Under review at PNAS

References in corpus (3)

• Frustratingly Easy Domain Adaptation
• StreetStyle: Exploring world-wide clothing styles from millions of photos
• Fine-Grained Car Detection for Visual Census Estimation

Cited by in corpus (38)

• Take a Look Around: Using Street View and Satellite Images to Estimate House Prices
• Classification of Urban Morphology with Deep Learning: Application on Urban Vitality
• BoxCars: Improving Fine-Grained Recognition of Vehicles using 3-D Bounding Boxes in Traffic Surveillance
• Computational Socioeconomics
• Learning Patterns of Tourist Movement and Photography from Geotagged Photos at Archaeological Heritage Sites in Cuzco, Peru
• Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey
• Quantifying memories: mapping urban perception
• Automatic Large Scale Detection of Red Palm Weevil Infestation using Aerial and Street View Images
• Generalizable prediction of academic performance from short texts on social media
• Bit-Flip Attack: Crushing Neural Network with Progressive Bit Search
• Detecting disparities in police deployments using dashcam data
• Self-supervised learning unveils change in urban housing from street-level images
• Deep learning for nano-photonic materials -- The solution to everything!?
• Deep hybrid model with satellite imagery: how to combine demand modeling and computer vision for behavior analysis?
• Automated Detecting and Placing Road Objects from Street-level Images
• Learning Neighborhood Representation from Multi-Modal Multi-Graph: Image, Text, Mobility Graph and Beyond
• Streetscape Analysis with Generative AI (SAGAI): Vision-Language Assessment and Mapping of Urban Scenes
• SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning
• Self-reinforcing Unsupervised Matching
• Mapping road safety features from streetview imagery: A deep learning approach
• Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem
• A pedestrian level strategy to minimize outdoor sunlight exposure in hot summer
• AutoLTS: Automating Cycling Stress Assessment via Contrastive Learning and Spatial Post-processing
• Urban2Vec: Incorporating Street View Imagery and POIs for Multi-Modal Urban Neighborhood Embedding
• Learning Street View Representations with Spatiotemporal Contrast
• Learning to Factorize and Relight a City
• Google Street View image of a house predicts car accident risk of its resident
• Socioeconomic correlations of urban patterns inferred from aerial images: interpreting activation maps of Convolutional Neural Networks
• Artifacts of Idiosyncracy in Global Street View Data
• Granularity at Scale: Estimating Neighborhood Socioeconomic Indicators from High-Resolution Orthographic Imagery and Hybrid Learning
• Crime and social environments: Differences between misdemeanors and felonies
• Surveilling Surveillance: Estimating the Prevalence of Surveillance Cameras with Street View Data
• Explainable, automated urban interventions to improve pedestrian and vehicle safety
• Learning a Dynamic Map of Visual Appearance
• Risk Prediction on Traffic Accidents using a Compact Neural Model for Multimodal Information Fusion over Urban Big Data
• Longevity Associated Geometry Identified in Satellite Images: Sidewalks, Driveways and Hiking Trails
• Clustering Bioactive Molecules in 3D Chemical Space with Unsupervised Deep Learning
• Oscillatory evolution of collective behavior in evolutionary games played with reinforcement learning