Classification of defects in additively manufactured nickel alloys using supervised machine learning

Abstract

The presence of undesirable microstructural features in additively manufactured components, such as cracks, pores and lack of fusion defects presents a challenge for engineers, particularly if these components are applied in structure-critical applications. Such features might need to be manually classified, counted and their size distributions measured during metallographic evaluation, which is a time-consuming task. In this study, the performance of two supervised machine learning methods (kth-nearest neighbours and decision trees) to automatically classify typical defects found during metallographic examination of additively manufactured nickel alloys is briefly outlined and discussed.

KEYWORDS:

• Additive manufacturing; nickel alloys; defects
• machine learning

Additive manufacturing is a near-net shape production technology that utilises a high-energy heat source to selectively melt or fuse together metallic powder to produce a three-dimensional part [1]. Notwithstanding the potential benefits of additive manufacturing technologies for near-net shape production, the presence of internal and external defects in additively manufactured components presents a problem for engineers, particularly if these components are applied in structure-critical applications and in situations where a component is subject to a fluctuating load [2,3]. Common types of internal defects found within additively manufactured parts include; lack of fusion defects, gas porosity, solidification cracks, impurities, solid-state cracks and void formation due to key-hole collapse in certain high energy density processes [4]. Due to their size, such defects (anomalous features or undesirable microstructural features) are detected using X-ray computational tomography [5] or, perhaps more commonly, during metallographic inspection using light microscopy. These features might need to be manually classified, counted and their size distributions measured, particularly in the early stages of process development where parameters such as the heat source power, velocity, hatch spacing and layer height are systematically varied to establish an optimised processing window [6]. Clearly, this can be a somewhat laborious and time-consuming task so the aim of this study is to evaluate whether machine learning can be used to automate this process.

Machine learning is a branch of Artificial Intelligence (AI) originating from pattern identification and cognitive acquisition concepts, which uses computational algorithms that effectively adapt an extensive dataset and produce a usable model to generate predictions based on historical evidence [7,8]. Machine learning is becoming an increasingly powerful tool in the field of materials science and engineering, and has been applied in a range of applications from the identification of breakthrough, high-T_c superconductors [9] to the prediction of unexplored hybrid organic–inorganic perovskites for photovoltaics [10]. With regard to additive manufacturing, Snell et al. [11] explored the suitability of unsupervised learning methods (k-means clustering) for rapidly classifying three common types of pores in titanium and nickel alloys observed via x-ray computational tomography and light microscopy, whilst Scime and Beuth [12] used a computer vision algorithm to classify powder-bed anomalies, such as re-coater streaking and incomplete spreading, during a laser powder bed fusion process. Additionally, Nalajam et al. [13] used three machine learning methods (k-means clustering, support vector machines and random forests) for porosity detection in aluminium alloy thin-walled structures produced by wire-arc additive manufacture.

In this study, supervised machine learning and more specifically, classification algorithms (kth-nearest neighbours and decision trees) are employed to automatically classify different types of defect that are sometimes observed in additively manufactured nickel alloys. A discussion about the basic principles of these algorithms is beyond the scope of this paper and the reader is directed to Lantz [14] for an informative introduction to the topic. Classification algorithms are best used when clean, labelled datasets are available for the machine to learn from. For classification problems, the labelled dataset is typically split into a larger training set and a smaller test set. The larger training set is used to ‘teach’ the algorithm, which is then applied to the training set as a predictive model, from which correct and incorrect predictions of defect type can easily be compared.

A labelled dataset of commonly observed defect types in additively manufactured nickel alloys was created via metallographic examination of the following three nickel alloys processed by selective laser melting: CM247LC, LR8 and Alloy 713C. These specimens were fabricated over the course of a range of research programmes at The University of Sheffield [15,16] and example micrographs showing the typical defects observed in these alloys are given in Figure 1. Data pertaining to the size and morphology of these defects were obtained from the polished cross-sections using a thresholding and particle analysis tool in the image analysis software, ImageJ [17]. For the purpose of this study, the commonly observed defects were labelled as: ‘Pore’, ‘Crack’, ‘Lack of Fusion (LoF) defect’ and ‘Pore with Crack (PwC)’. In total, 590 defects were labelled and quantitative data describing their size and shape were collated in a comma-separated values (CSV) file. Size and shape descriptors include Feret diameter, circularity, roundness and aspect ratio, and the full list of measured defect characteristics, along with some typical values for these geometrical descriptors, are given in Table 1.

Figure 1. (a) Example micrograph showing typical defects sometimes found in additively manufactured nickel alloys, along with corresponding binary images of (b) cracks, (c) a pore, (d) a connected pore and crack and (e) a lack of fusion defect.

Table 1. Structure of the labelled dataset of defect types observed in additively manufactured Nickel alloys.

Defect Type	Area (μm^2)	Major Axis (μm)	Minor Axis (μm)	Angle	Circularity	Feret Diameter (μm)	Aspect Ratio	Roundness	Solidity
Crack	337	40.1	10.3	95.7	0.21	57.8	4.56	0.30	0.49
Pore	757	30.4	25.6	93.3	0.82	32.5	1.22	0.84	0.92
PwC	1752	62.0	32.9	89.9	0.27	91.9	2.16	0.55	0.60
LoF	25621	230.9	95.1	89.5	0.32	301.8	2.57	0.45	0.65

Notes: The dataset contains 590 instances and 9 attributes. The mean values for measured defect characteristics (e.g. Area and Circularity) are also included (PwC = Pore with Crack; LoF = Lack of Fusion Defect).

Figure 2 is a three-dimensional scatter plot of the 590 defects listed in the labelled dataset in Feret diameter-, circularity- and roundness-space. Pores are characterised by high indices of roundness and circularity and, due to having small Feret diameters, they form a cluster in the top corner of the scatter plot. Cracks are slightly larger than pores and they form a cluster in the left-hand corner of Figure 2 as they are more irregular in shape, although this cluster is considerably more diffuse and is likely to also include ‘Pore with Crack’-type defects. Lack of fusion defects are considerably larger than the other types of defect and, although they are also irregular in shape, they do not fall into a well-defined group in Feret diameter-, circularity and roundness-space.

Figure 2. A normalised 3D scatter plot of the roundness, Feret diameter and circularity of the 590 defects listed in the labelled dataset of defects commonly found in additively manufactured nickel alloys. All feature data were re-scaled to a standard range of 0–1 using Equation (1).

Normalisation of the dataset was performed to ensure that no single size or shape descriptor had a larger influence on the defect classification outcome due to its magnitude. All feature data (x) were re-scaled to a standard range of 0–1 (X_n) using the following minimum-maximum normalisation approach: (1) $X_n=\frac{x-x_{min}}{x_{max}-x_{min}}$(1) Following this, the normalised dataset was split into a training set (80%) and a test set (20%) using a stratified sampling approach to provide a consistent proportion of each defect type in both sets.

The kth-nearest neighbour (kNN) algorithm used in this study was the ‘Class’ package, which can be embedded into the statistical computing environment, R. The ‘Class’ package uses Euclidean distances to measure the shortest path from a defect in a multi-dimensional plane to the nearest cluster via a straight line and studies have shown that this measurement approach provides the highest classification accuracy if the data are not particularly noisy [18]. With regard to decision trees, various algorithms exist – including classification and regression trees, Iterative Dichotomiser 3, C4.5 and C5.0. Using the ‘divide and conquer strategy’, decision trees find the optimal features within the entire dataset to split recursively into more homogeneous subsets until they are all sufficiently homogeneous. Various splitting criterions exist, such as entropy and information gain, Gini index, Chi-squared statistic and gain ratio. For the classification of defects, the C5.0 algorithm was implemented due to its reported strong performance on ‘out-of-the-box’ problems [14]. The ‘Class’ kNN and the C5.0 decision tree algorithms were selected as these are ‘off the shelf’ packages that can readily be implemented in R.

Confusion matrices illustrating the performance of the kNN and C5.0 decision tree algorithms for classifying different types of defect are shown in Figure 3 (a) and (b). For kNN models, the number of nearest neighbours (the k-value) is an important tuning parameter as it determines the bias-variance trade-off within the model, which is crucial for representing its accuracy and ability to generalise towards new unseen data. A range of k-values were therefore tested to monitor its influence on model performance and it was found that k values of 5 and 7 yielded the highest model accuracy by classifying 89.8% of the defects correctly. The C5.0 decision tree performed marginally better than the kNN algorithm, with an overall accuracy of 92% after 5 trials. With an increasing number of trials, model accuracy fluctuated between 90% and 92% and, considering a minimum trial value of 5 provided the maximum classification accuracy while keeping the mean tree size to a minimum, it was deemed to be the optimal trial value for this investigation. The accuracy of the algorithms varied between the different defect categories however, with both performing well when classifying pores and cracks, but accuracy was significantly reduced for the other defect types (in particular, the ‘Pore with Crack’-type defects). The user and producer accuracies of the kNN and C5.0 decision tree algorithms for each defect category are listed in Figure 3(c).

Figure 3. Confusion matrices illustrating the performance of the (a) kNN and (b) C5.0 decision tree algorithms for classifying different types of defects in the test dataset. The user and producer accuracies of the kNN and C5.0 decision tree algorithms for each defect type are provided in (c). The total number of defects in the test dataset is 118.

It should be noted that only a limited amount of information about classification performance on a broader scale can be drawn from confusion matrices. Additional performance measures such as sensitivity, precision, recall and the kappa (κ) statistic are available to engineers working with machine learning models [14]. For instance, the (κ) statistic provides a metric of how well the predicted and actual examples agree with each other; values of κ < 0.2 suggest poor agreement, whilst values in excess of 0.8 suggest good or very good agreement [14]. Values of κ = 0.81 and 0.88 (2 d.p.) were calculated for the kNN and decision tree models, respectively.

Overall, the nearest neighbour and decision tree algorithms performed well in classifying defect types in the test dataset. Both algorithms were particularly effective in correctly predicting whether a defect was a crack or a pore, but were less effective at correctly classifying lack of fusion features and situations where a pore and crack were connected. The latter is probably because defects labelled as ‘Pore with Crack’ fall somewhere between the clusters of pores and clusters of cracks in Figure 2. Furthermore, there is a natural bias in the defect type dataset (e.g. cracks and pores are more common that lack of fusion defects) and this may influence the prediction of the machine learning algorithms. Nevertheless, the ‘Class’ kNN and C5.0 decision tree algorithms are straightforward to implement in the statistical computer environment, R, and both undergraduate and postgraduate taught aerospace engineering students at The University of Sheffield have successfully deployed both approaches for classifying defects found via metallography without any prior experience of machine learning. The labelled dataset of 590 defects and the R scripts for importing the dataset and applying the ‘Class’ kNN and C5.0 decision tree algorithms can be made available on request.

Acknowledgements

The authors wish to thank Dr Gavin Baxter for useful comments during the preparation of this article.

Disclosure statement

No potential conflict of interest was reported by the author(s).