Auto-Icon+: An Automated End-to-End Code Generation Tool for Icon Designs in UI Development
arXiv:2204.08676 · doi:10.1145/3531065
Abstract
Approximately 50% of development resources are devoted to UI development tasks [9]. Occupying a large proportion of development resources, developing icons can be a time-consuming task, because developers need to consider not only effective implementation methods but also easy-to-understand descriptions. In this paper, we present Auto-Icon+, an approach for automatically generating readable and efficient code for icons from design artifacts. According to our interviews to understand the gap between designers (icons are assembled from multiple components) and developers (icons as single images), we apply a heuristic clustering algorithm to compose the components into an icon image. We then propose an approach based on a deep learning model and computer vision methods to convert the composed icon image to fonts with descriptive labels, thereby reducing the laborious manual effort for developers and facilitating UI development. We quantitatively evaluate the quality of our method in the real world UI development environment and demonstrate that our method offers developers accurate, efficient, readable, and usable code for icon designs, in terms of saving 65.2% implementing time.
Accepted to ACM Transactions on Interactive Intelligent Systems (TIIS)
References in corpus (8)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?
- Owl Eyes: Spotting UI Display Issues via Visual Understanding
- Screen2Vec: Semantic Embedding of GUI Screens and GUI Components
- Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots
- From Lost to Found: Discover Missing UI Design Semantics through Recovering Missing Tags
- Guided Bug Crush: Assist Manual GUI Testing of Android Apps via Hint Moves
- GIFdroid: Automated Replay of Visual Bug Reports for Android Apps