Design of Semantic-Based Colorization of Graphical User Interface Through Conditional Generative Adversarial Nets

ABSTRACT

There has recently been a significant movement toward aiding graphic design tasks based on artificial intelligence or machine learning. In addition, colorization plays an important role within the topic of GUI design. Previous studies regarding automatic colorization have focused on a consideration of the realistic aspects of an image without consideration of the design semantics or usability, which are critical aspects for a practical GUI design. We, therefore, propose an end-to-end network for a generative combination of color sets for a GUI design based on the design semantics, while utilizing thousands of actual GUI design datasets acquired from LG Electronics to train the network. By utilizing the GUI design dataset, our network effectively generates color sets for a GUI design by considering various design aspects, such as the usability factors. In detail, we concatenate the textual design concept, characteristics of the application, and usage frequency for the elements of the design semantics. We then construct a conditional generative adversarial net processing of the design semantics as a condition to generate suitable color sets and construct the GUI design based on these sets. The experiments indicate that our proposed method effectively generates color sets for a GUI design based on the design semantics. In addition, our proposed method shows a better score than other methods on a user test conducted to verify the practicality, perception, recognition, diversity, and esthetic features. Moreover, experimental results prove that users can effectively grasp the intended design concept of our generated GUI design with higher top-1, top-2, and top-3 levels of accuracy.

Additional information

Notes on contributors

Younghoon Lee

Younghoon Lee is a Professor with the Department of Industrial and Systems Engineering, Seoul National University of Science and Technology, Seoul, Korea. His research interests include user interface, machine learning, and data mining applications.

Sungzoon Cho

Sungzoon Cho is a Professor with the Department of Industrial Engineering and Institute for Industrial Systems Innovation, Seoul National University, Seoul, Korea. His research interests are neural network, pattern recognition, data mining, and their applications in different areas such as response modeling and keystroke-based authentication.