Sketch2code: Generating a website from a paper mockup
arXiv:1905.13750
Abstract
An early stage of developing user-facing applications is creating a wireframe to layout the interface. Once a wireframe has been created it is given to a developer to implement in code. Developing boiler plate user interface code is time consuming work but still requires an experienced developer. In this dissertation we present two approaches which automates this process, one using classical computer vision techniques, and another using a novel application of deep semantic segmentation networks. We release a dataset of websites which can be used to train and evaluate these approaches. Further, we have designed a novel evaluation framework which allows empirical evaluation by creating synthetic sketches. Our evaluation illustrates that our deep learning approach outperforms our classical computer vision approach and we conclude that deep learning is the most promising direction for future research.
References in corpus (22)
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- An overview of gradient descent optimization algorithms
- Going Deeper with Convolutions
- A guide to convolution arithmetic for deep learning
- A Review on Deep Learning Techniques Applied to Semantic Segmentation
- Richer Convolutional Features for Edge Detection
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
- Rich feature hierarchies for accurate object detection and semantic segmentation
- R2CNN: Rotational Region CNN for Orientation Robust Scene Text Detection
- Pyramid Scene Parsing Network
- Does median filtering truly preserve edges better than linear filtering?
- Show and Tell: A Neural Image Caption Generator
- Deep Visual-Semantic Alignments for Generating Image Descriptions
- Image Super-Resolution Using Deep Convolutional Networks
- Comparing the Performance of L*A*B* and HSV Color Spaces with Respect to Color Image Segmentation
- Deep Direct Regression for Multi-Oriented Scene Text Detection
- Comparative analysis of common edge detection techniques in context of object extraction
- Exploiting Local Structures with the Kronecker Layer in Convolutional Networks
- pix2code: Generating Code from a Graphical User Interface Screenshot
- Efficient Large Scale Video Classification
- HTML5 WebSocket protocol and its application to distributed computing