MosAIc: Finding Artistic Connections across Culture with Conditional Image Retrieval
arXiv:2007.07177
Abstract
We introduce MosAIc, an interactive web app that allows users to find pairs of semantically related artworks that span different cultures, media, and millennia. To create this application, we introduce Conditional Image Retrieval (CIR) which combines visual similarity search with user supplied filters or "conditions". This technique allows one to find pairs of similar images that span distinct subsets of the image corpus. We provide a generic way to adapt existing image retrieval data-structures to this new domain and provide theoretical bounds on our approach's efficiency. To quantify the performance of CIR systems, we introduce new datasets for evaluating CIR methods and show that CIR performs non-parametric style transfer. Finally, we demonstrate that our CIR data-structures can identify "blind spots" in Generative Adversarial Networks (GAN) where they fail to properly model the true data distribution.
References in corpus (18)
- Scikit-learn: Machine Learning in Python
- The NumPy array: a structure for efficient numerical computation
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
- DenseNet: Implementing Efficient ConvNet Descriptor Pyramids
- Survey of Nearest Neighbor Techniques
- Aggregated Residual Transformations for Deep Neural Networks
- What makes ImageNet good for transfer learning?
- EFANNA : An Extremely Fast Approximate Nearest Neighbor Search Algorithm Based on kNN Graph
- Unsupervised Alignment of Embeddings with Wasserstein Procrustes
- Visual Attribute Transfer through Deep Image Analogy
- Random Projection Trees Revisited
- Large-Scale Intelligent Microservices
- MMLSpark: Unifying Machine Learning Ecosystems at Massive Scales