Image-based novel fault detection with deep learning classifiers using hierarchical labels

Abstract

One important characteristic of modern fault classification systems is the ability to flag the system when faced with previously unseen fault types. This work considers the unknown fault detection capabilities of deep neural network-based fault classifiers. Specifically, we propose a methodology on how, when available, labels regarding the fault taxonomy can be used to increase unknown fault detection performance without sacrificing model performance. To achieve this, we propose to utilize soft label techniques to improve the state-of-the-art deep novel fault detection techniques during the training process and novel hierarchically consistent detection statistics for online novel fault detection. Finally, we demonstrated increased detection performance on novel fault detection in inspection images from the hot steel rolling process, with results well replicated across multiple scenarios and baseline detection methods.

Keywords:

• Novel fault detection
• hierarchical structure
• deep learning
• fault classification

Data and code availability statement

Due to the proprietary nature of the collected dataset from the industry, supporting data cannot be made openly available. However, the code is made available on the Github repo at https://github.com/hyan46/Deep-Novel-Fault-Detection-using-Hierarchical-Labels

Additional information

Funding

The authors acknowledge the generous support from the funding agency of NSF CMMI 1922739 and DOE DE EE0009354.

Notes on contributors

Nurettin Dorukhan Sergin

Nurettin Dorukhan Sergin is a Research Scientist at Meta Platforms Inc., advancing the state-of-the-art machine learning techniques for neural interfaces to be used in the context of augmented reality devices. He obtained his PhD in Industrial Engineering from Arizona State University. During his doctoral studies, Dr. Sergin focused his research on the viability of advanced machine learning techniques as it is applied to statistical process monitoring of high-dimensional large-scale process data streams, such as images or timeseries sampled at high-frequency.

Jiayu Huang

Jiayu Huang is a PhD student in the Industrial Engineer program at Arizona State University. Her current research is focused on domain knowledge incorporation in deep learning, multi-modal generative model for event prediction, automation of segmentation and anomaly detection on image data.

Tzyy-Shuh Chang

Tzyy-Shuh Chang received BS, MS, and PhD degrees in mechanical engineering from National Taiwan University (1987), the Ohio State University (1991), and the University of Michigan (1995), respectively. He co-founded OG Technologies, Inc., a Michigan corporation, and led the company to be the globally leading supplier and brand of advanced surface inspection equipment, an R&D 100 awardee, for the steel industry. Since its inception, OG Technologies has established its business by way of advanced research and development and cooperative activities with the academia and metal industry, focusing on bringing the state-of-the-art technologies in sensing and data analytics to the millennium-old industry.

Hao Yan

Hao Yan is an assistant professor in the School of Computing and Augmented Intelligence at Arizona State University. Previously, Hao Yan received a BS degree in Physics from the Peking University, Beijing, China, in 2011. He also received the M.S. degree in Statistics, the MS degree in Computational Science and Engineering, and the PhD degree in Industrial Engineering from Georgia Institute of Technology, Atlanta, in 2015, 2016, 2017, respectively. His research interests focus on developing scalable statistical learning algorithms for large-scale high-dimensional data with complex heterogeneous structures to extract useful information for the purpose of system performance assessment, anomaly detection, intelligent sampling, and decision making.