Semi-supervised machine learning of optical in-situ monitoring data for anomaly detection in laser powder bed fusion

ABSTRACT

Laser powder bed fusion (L-PBF) is one of the most widely used metal additive manufacturing technology for fabrication of functional and structural components. However, inconsistency in quality and reliability of L-PBF products is still a significant barrier preventing it from wider adoption. Machine learning (ML) of monitoring data offers a unique solution to effectively identify possible defects and predict the quality of L-PBF products. In this work, we introduce a semi-supervised ML approach to detect anomalies that occurred in L-PBF products. We train the ML model to classify surface appearances in the reference monitoring data. We then correlate the classified appearances to post-process characteristics, e.g. surface roughness, morphology, or tensile strength. We demonstrate that the established correlation enables the determination of key appearances indicative of the quality of the printed samples including anomaly-free, lack-of-fusion and overheated. We further validate our ML approach by performing prediction on test samples having various geometries.

KEYWORDS:

• Machine learning
• additive manufacturing
• laser powder bed fusion
• quality control

1. Introduction

3D printing or additive manufacturing (AM) has become the key enabling factor in advanced manufacturing sector of Industry 4.0 technologies thanks to its numerous advantages such as minimal geometrical constraints in design, flexibility in customisation, and most of all, reduction of material wastage which is a crucial pillar in circular economy to sustainably develop society (Joshi and Sheikh 2015; Tofail et al. 2018; Liu et al. 2021). Laser powder bed fusion (L-PBF) is a metal AM technology that has tremendous potential in manufacturing functional and structural components in most industrial sectors (Ladani and Sadeghilaridjani 2021; Wang et al. 2022).

Lack of standardised methods to ensure consistency and reliability is widely recognised as the critical barrier that slowdowns the acceptance of L-PBF products into highly regulated industries (King et al. 2015). Generally, L-PBF processes involve a large (up to 50) number of process parameters (Oliveira, LaLonde, and Ma 2020), e.g. laser power, scanning speed, scanning strategy, layer thickness and gas flow rate. Internal defects of the products inevitably emerge with inadequate parameters. Even with optimal sets of process parameters, defects may occur due to geometrical complexity of printed parts, e.g. overhang structures, or thin walls. Once a defect is initiated from a layer, it may propagate to subsequent ones, either causing visible damages to the entire printed part or generating internal structural weaknesses only detectable by costly post-process detection methods. The lack of well-established quality control technologies is unacceptable in automobile, aerospace or biomedical, which are among industrial sectors known for having the most elaborate quality control protocols but having a leading role in driving development of the AM sector.

The discovery of structural weakness during quality check on the manufactured part is costly for the resources, money and time invested. Therefore, in-situ monitoring of the printing process to detect potential defects and make predictions of the product quality leads to a more efficient manufacturing process of L-PBF technology (Everton et al. 2016; McCann et al. 2021). Several in-process monitoring systems have been developed using various types of sensors, including optical sensors (zur Jacobsmühlen et al. 2015; Abdelrahman et al. 2017; zur Jacobsmühlen et al. 2017; Grasso et al. 2017; Lu et al. 2019; Zhang, Liu, and Shin 2019; Lu et al. 2020), thermal sensors (Lane et al. 2016; Raplee et al. 2017; Khanzadeh et al. 2018), acoustic sensors (Kouprianoff et al. 2017; Shevchik et al. 2018; Ye et al. 2018; Drissi-Daoudi et al. 2022) or X-ray (Leung et al. 2018) to acquire visual information, e.g. surface pattern, powder bed homogeneity, thermal information, e.g. temperature intensity, melt pool geometry, heat affected zone along the scan path or acoustic information of L-PBF processes. The acquired information has been routinely used to validate empirical or physics-based models targeting to detect defects or predict the overall quality of printed parts. The modelling approach relies on various simplifying assumptions to reduce the computational cost caused by entangled physical phenomena, large number of involving parameters and complex boundary conditions, thereby leading to simplified results incapable of indicating the actual quality of printed parts. Moreover, it is not trivial to correlate the acquired monitoring data with the hidden defective patterns as there is a huge volume of continuously generated data. Therefore, machine learning (ML) has been increasingly adopted in recent years to extract useful information from monitoring data for quality control of L-PBF products (Wang, Tan, et al. 2020; Sing et al. 2021).

Machine learning (ML) algorithms can be classified as supervised (with labelled training data) or unsupervised (with unlabelled training data) (Alloghani et al. 2020). Unsupervised ML algorithms, e.g. support vector regression (SVR) (Song et al. 2016; Grasso et al. 2017), support vector machine (SVM) (Aoyagi et al. 2019), self-organising maps (SOM) clustering (Khanzadeh et al. 2019) and k-means clustering (Scime and Beuth 2018a), have been extensively used for defect detection, defect classification and process parameter optimisation in L-PBF systems. Various supervised ML models have also been employed to predict printability and repeatability of L-PBF products (Scime and Beuth 2018b; Chen et al. 2020; Huang and Li 2021). The applications of supervised and unsupervised ML algorithms in various aspects of additive manufacturing are comprehensively reviewed by Wang, Sun, et al. (2020); Goh, Sing, and Yeong (2021) and Sing et al. (2021). The reported studies utilising supervised or unsupervised approaches have demonstrated significant advancements in quality control of L-PBF processes. Nevertheless, there are still certain limitations in these approaches. On the one hand, there is a lack of transparency in predictive outputs of unsupervised ML models. This requires human intervention to validate prediction results. On the other hand, supervised ML models are costly and potentially inefficient as they require human experts to manually label features in the training datasets. Semi-supervised learning is an approach that falls between unsupervised and supervised learning. It combines a small amount of labelled data with a large amount of unlabelled training data. Semi-supervised learning has not been widely utilised for quality control of L-PBF processes (Okaro et al. 2019; Pandiyan et al. 2021) despite their flexibility in implementation.

In this work, we introduce a semi-supervised ML approach with minimal effort in labelling training datasets to detect anomalies occurring during the printing process and predict the quality of L-PBF printed parts. We first train the ML model using layer-wise images of reference samples printed with varying process parameters to classify characteristic appearances of the reference samples at the pixel level by performing semantic segmentation. We then correlate the classified appearances with post-process characteristics, e.g. surface roughness, morphology, or tensile strength, to determine key appearances indicative of the quality of the printed samples including anomaly-free, lack-of-fusion and overheated. Finally, we demonstrate the capability of our ML model in defect detection and quality prediction of test samples having various geometries, e.g. cubes, bevel gears, overhanging and unsupported structures. The ML model offers a quantitative tool to evaluate the quality of test samples. Our approach using semi-supervised ML approach can be expanded to control the quality of products printed by other 3D printing processes and materials thanks to its simplicity in labelling training data.

2. Methodology

2.1. Experimental materials and designs

We carry out all printing experiments on a metal 3D printer (SLM 500-HL, SLM Solutions Group AG). The printer has a building chamber of $500\times 280\times 365\mathrm{m}{\mathrm{m}}^3$ and two fibre lasers having a maximum power of $400\mathrm{W}$. The printed material is stainless steel 316L (SS316L) powders. In our experiment, we capture images of the powder bed using a monitoring system attached to the printing chamber as shown in Figure 1. With this setup, layer-wise images of printed samples are captured through a viewport on top of the printing chamber by a digital single lens reflex camera (D850, Nikon) having pixel resolution of $8256\times 5504$ pixels (Lu et al. 2019). For each layer, two images are captured: one after laser scanning, and the other one after powder recoating.

Figure 1. Schematic figure of experiment setup.

We perform an extensive experimental study as follows: in each print job, we print 10 reference samples using 10 distinct parameter settings. Here, we note that the printing parameter setting including the laser energy and scanning velocity can be set separately for each sample. The reference samples are designed as circular cylinders having a diameter of $8\mathrm{m}\mathrm{m}$ and height of $8\mathrm{m}\mathrm{m}$ We then repeat the print job seven times keeping the same arrangement for printing parameter settings to obtain sufficient data for ML training. After excluding bad images resulted from technical difficulties, we obtain a total of $1758$ layer-wise images of reference samples from the repeated print jobs. In Figure 2, we show the arrangement of these reference samples within the region of interest of the experiment setup.

Figure 2. (a) Representative raw image of reference samples. (b) Processed image. (c) Corresponding ground truth pixel-labelled image.

In Table 1, we show the printing parameter settings and the corresponding labels used in our machine learning (ML) analysis. We vary the printing parameters by adjusting the laser power $p$ and the scanning velocity $v$. The laser energy density $e$ of each printing parameter is defined as (1) $e={\displaystyle \frac{p}{v\times h\times t},}$(1) where h is the hatch spacing and t is the layer thickness. The hatch spacing and layer thickness are kept constant at $0.14\mathrm{m}\mathrm{m}$ and $0.05\mathrm{m}\mathrm{m}$, respectively. The laser power $p$ and scanning velocity $v$ are varied in multiples of the baseline parameters recommended by the manufacturer: the baseline laser power is $P=175\mathrm{W}$ and the baseline scanning velocity is $V=550\mathrm{m}\mathrm{m}/\mathrm{s}$. We define the baseline energy density as $E=P/\left(Vht\right)=45\mathrm{J}/\mathrm{m}{\mathrm{m}}^3$. In our experiment, we vary the energy density between $0.5E$ and $4.0E$ to induce the formation of various surface appearances, and subsequently identify the correlation between surface appearances with post-process characteristics, e.g. surface roughness, morphology and tensile strength.

Table 1. Printing parameter settings and corresponding appearance labels for printed reference samples.

Printing parameter setting	Energy density $\left(\mathrm{J}/\mathrm{m}{\mathrm{m}}^3\right)$	Laser power (W)	Scanning velocity (mm/s)	Appearance label for reference sample
${\mathcal{S}}_1$	$0.5E$	$0.5P$	$V$	${\mathcal{F}}_1$
${\mathcal{S}}_2$	$0.5E$	$P$	$2V$	${\mathcal{F}}_2$
*${\mathcal{S}}_3$	$\mathit{E}$	$\mathit{P}$	$\mathit{V}$	*${\mathcal{F}}_3$
${\mathcal{S}}_4$	$E$	$2P$	$2V$	${\mathcal{F}}_4$
${\mathcal{S}}_5$	$1.5E$	$P$	$0.67V$	${\mathcal{F}}_5$
${\mathcal{S}}_6$	$1.5E$	$1.5P$	$V$	${\mathcal{F}}_6$
${\mathcal{S}}_7$	$2E$	$P$	$0.5V$	${\mathcal{F}}_7$
${\mathcal{S}}_8$	$2E$	$2P$	$V$	${\mathcal{F}}_8$
${\mathcal{S}}_9$	$3E$	$2P$	$0.67V$	${\mathcal{F}}_9$
${\mathcal{S}}_{10}$	$4E$	$2P$	$0.5V$	${\mathcal{F}}_{10}$
Unfused powder				${\mathcal{F}}_0$

*Baseline settings: $E=45\mathrm{J}/\mathrm{m}{\mathrm{m}}^3;P=175\mathrm{W};V=550\mathrm{m}\mathrm{m}/\mathrm{s}$.

2.2. Image processing

To reduce artefacts potentially causing bias in the ML model, we process raw layer-wise images using Matlab (R2021a, MathWorks) environment with the following steps (Lu et al. 2019). First, we define a region of interest (ROI) covering an area of $63\times 40\mathrm{m}{\mathrm{m}}^2$ around the 10 cylindrical reference samples and extract images within the ROI before downsizing them from $3248\times 4872$ pixels to $550\times 350$ pixels to reduce the computational cost for ML model training. Each pixel of the cropped images represents a physical area of $110\times 110\mu {\mathrm{m}}^2$ (Figure 2(a)).

Second, we perform background subtraction on the cropped images to enhance features of interest and remove any non-uniform illumination in the background. Here, we generate a background image for calibration by taking a uniformly coated powder bed before printing and then applying Gaussian blur to this image to remove small features from it. The background image is then subtracted from all layer-wise images using an absolute difference operation. To avoid over saturation in the resulting images, the brightness of the background image is slightly reduced. The resulting images are enhanced in contrast and freed of intensity variation caused by non-uniform illumination. Contrast-limited adaptive histogram equalisation is then applied to further enhance the contrast of the resulting images (Figure 2(b)).

Finally, we create an image of ground truth pixel labels based on the designed areas and corresponding printing parameter settings of the reference samples. The pixel-labelled image consists of 10 classes of surface appearance (${\mathcal{F}}_1$ to ${\mathcal{F}}_{10}$) of reference samples correspondingly printed by 10 printing parameter settings (${\mathcal{S}}_1$ to ${\mathcal{S}}_{10}$) and one class of the area covered by unfused powder $\left({\mathcal{F}}_0\right)$. Here, since there is no variation in the geometries of the reference samples and printed areas are kept fixed across all layers, we expect that for the reference samples, each unique parameter setting, e.g. ${\mathcal{S}}_i$ for $i$ from $1$ to $10$, directly results in a distinct printed surface appearance, labelled as ${\mathcal{F}}_i$ (Table 1).

Figure 2(c) displays the ground truth pixel-labelled image used in our analysis. As the reference samples are all vertical circular cylinders, we only need one pixel-labelled image for the entire stack of images. This simplifies the labelling step in training supervised ML models.

We note that in a typical printing process, a part printed with the same parameter setting as a reference sample may have printed areas with significantly different appearances compared to those of the reference sample. This may result from the variation of either the printed part’s geometry, e.g. overhang or thin-walled structures, or other printing conditions, e.g. preheated temperature of the base plate or gas flow rate. As a result, a typical printing process using any of the preset printing parameter settings may result in a printed surface with appearance identified by the ML model as one consisting of multiple appearance classes.

2.3. Machine learning models

We implement ML models in Matlab environment (R2021a, MathWorks) to perform semantic segmentation on the training dataset obtained from the reference samples. Semantic segmentation allows identification and classification of various classes at pixel level by learning the context of the input images. Taking advantage of the tremendous growth of deep learning models in recent years, we compare three commonly used classification networks for semantic image segmentation, i.e. Vgg16 as the backbone of SegNet, ResNet50 as the backbone of DeepLab v3+ and Xception as the backbone of DeepLab v3+. We then utilise the network with the highest performance to perform testing on datasets obtained from samples different from the reference ones.

The size of input layers of the ML networks is modified to [350,550,1] to match the size of processed images. The number of class labels is adjusted to $11$, which is the number of appearance classes in our training data. The class weights for the final pixel classification layer of each network are then rebalanced by the median frequency of each class label to avoid biasing the ML model toward the unfused powder class, which dominates the training dataset owning to its largest area in each input image.

The dataset of reference samples in our study comprises $1730$ images obtained from multiple print jobs; half of the images are captured after laser scanning and the other half are captured after powder recoating. To train our ML models, we only use the images captured after laser scanning to obtain information on surface morphology of the reference samples. We use $90\text{\%}$ of the images captured after laser scanning for training the ML models and the remaining $10\text{\%}$ for performance testing.

Data augmentation is applied on the training dataset to enhance the robustness of the ML model. The training images are augmented with random reflection, random translation between $-40$ pixels and $40$ pixels, random rotation with the angle between $-{90}^{\circ }$ and ${90}^{\circ }$, and random scaling with scale factor varying from $0.5$ to $2$. To optimise memory usage, augmented images are randomly created in Matlab during the training process without saving them to memory.

2.4. Microstructure characterisation and tensile testing

Microstructural characteristics of the reference samples are examined by a scanning electron microscope (JCM-6000, Jeol Ltd.). Surface roughness of the reference samples is measured using a surface profiler (SJ400, Mitutoyo). Subsize dogbone tensile coupons are printed with the same parameter settings as reference samples in the XYZ orientation according to the coordinate system specified by ISO/ASTM 52921:2016 standard. The dimensions of tensile samples are specified by ASTM E8/E8M-15a standard. Tensile tests are carried out in displacement control using a universal testing machine (10 kN, Instron 5569, Instron).

3. Results and discussion

3.1. Performance comparison between machine learning networks

Performances of three semantic segmentation network structures, i.e. Vgg16 as the backbone of SegNet, ResNet50 as the backbone of DeepLab v3+, and Xception as the backbone of DeepLab v3+, are tested on 86 images, or equivalently $10\text{\%}$ of the dataset captured after laser scanning of the reference samples. We evaluate the performance of the ML network structures using three performance scores typically used for semantic segmentation: global accuracy, average intersection over union $\left(MeanIoU\right)$ and average contour matching score for boundary F1 $\left(MeanBF\right)$ (Csurka et al. 2013). First, the global accuracy estimates the percentage of correctly classified pixels of all classes in all images of a dataset and is defined as (2) $\mathrm{G}\mathrm{l}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{l}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{l}\mathrm{y}\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d}\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{t}}$(2)

Second, the $IoU$ metric describes the intersection of different classes and is defined by the ratio of correctly classified pixels to the union of ground truth and predicted pixels in that class: (3) $IoU={\displaystyle \frac{\left|GT\cap PR\right|}{\left|GT\cup PR\right|},}$(3) where $GT$ and $PR$, respectively, denote the ground truth and the predicted results. The $MeanIoU$ is the average $IoU$ score of all classes in all images of the dataset.

Finally, the $BF$ score measures how the boundary of each class matches the ground truth boundary. It is defined as the F1-measure of the precision (P) and recall (R) values calculated within a distance error tolerance, i.e. $0.75\text{\%}$ of the length of the image diagonal (Csurka et al. 2013): (4) $BF={\displaystyle \frac{2\times P\times R}{\left(P+R\right)}.}$(4)

The $MeanBF$ is the average $BF$ score of all classes in all images of the dataset.

In Figure 3, we show comparisons between the overall performances of the ML network structures on the testing dataset of reference samples. The DeepLab v3+ based network structures, with both backbones ResNet50 and Xception, outperform SegNet in all performance metrics. Here, we note that ResNet50 as the backbone of the network performs slightly better than Xception. In Figure 4, we show representative predictions and the corresponding errors of the tested ML network structures. We observe that most errors of the SegNet based network structure occur at the boundaries of the samples, consistent with its low $MeanIoU$ and $MeanBF$ scores shown in Figure 3. The outstanding performance of DeepLab v3 + network structure is due to its Atrous Convolution, which enhances the receptive field of convolutional layers (Tianhua, Siqun, and Junchuan 2018) and allows the model to capture crucial information surrounding each pixel, e.g. at the boundaries separating laser-scanned areas from the unfused powder areas. The resulting $MeanIoU$ and $MeanBF$ scores of DeepLab v3 + based network structure are therefore noticeably higher than those by SegNet based network structure even though the global accuracies of all three network structures are comparably high $\left(⪆0.95\right)$.

Figure 3. Performance scores of three ML network structures measured on the testing datasets of reference samples.

Figure 4. (a) Input image. (b–d) Representative predictions of three ML network structures on the testing dataset of reference samples. (e) Corresponding ground truth pixel-labelled image. (f–h) Comparison of prediction results with expected ground truth. The green and magenta highlight errors outside and inside of the laser-scanned areas, respectively.

As the backbone of the DeepLab v3 + network, ResNet50 produces slightly better results than Xception even though the structure of Xception reportedly performed better in semantic segmentation tasks compared to VGG-Net or ResNet backbones (Zhang et al. 2020). This may be linked to Xception’s focus on training efficiency enhancement and the resulting cost of lowering the number of parameters for capturing image features compared to ResNet. As a result, we use DeepLab v3 + network structure with ResNet50 as the backbone for our ML model in subsequent analysis.

3.2. Microstructures and tensile properties

To identify the relations between appearances of printed surfaces and quality, we characterise surface morphology and tensile performance of samples printed with the same parameter settings as those of reference samples. In Figures 5 and 6, we show representative snapshots of the surface morphology and the corresponding surface roughness of several reference samples. We observe that the surface morphology of samples printed with low energy density, e.g. $0.5E$, is characterised by discontinuous melting tracks and lack of fusion of metal powders while those with moderate energy density, e.g. $E,1.5E$ and $2E$, have smoother surface texture. For samples printed with the same energy density $E$, the one printed by parameter setting ${\mathcal{S}}_4$ has rougher surface and more discontinuous melting tracks than the one printed by parameter setting ${\mathcal{S}}_3.$ This may result from instability of melt pool due to higher printing speed of parameter setting ${\mathcal{S}}_4$. However, there is no significant variation in surface morphology and roughness caused by different printing speeds between samples printed with the same energy density in the range between $1.5E$ and $2E$ (Figure 6). Further increasing energy density to $3E$ and $4E$ causes overheating and micro cracks (Figure 5(g,h)), indicating significant residue stress in samples printed with parameter setting ${\mathcal{S}}_9$ and ${\mathcal{S}}_{10}$. This observation strongly suggests that samples printed with parameter settings ${\mathcal{S}}_1$, ${\mathcal{S}}_2$ and ${\mathcal{S}}_4$ tend to suffer from lack-of-fusion, while those printed with ${\mathcal{S}}_9$ and ${\mathcal{S}}_{10}$ are likely to be overheated during printing.

Figure 5. Surface morphology of reference samples printed with varying printing parameter settings.

Figure 6. Surface roughness of reference samples printed with varying printing parameter settings.

In Figure 7(a), we present the yield strength and the ultimate strength of tensile coupons printed with the same parameter settings as reference samples. First, we note that the sample printed with the baseline parameter setting ${\mathcal{S}}_3$ has high yield strength and tensile strength, consistent with the manufacturer’s recommendation. Other samples having comparable tensile performances are those printed by ${\mathcal{S}}_5$, ${\mathcal{S}}_6,{\mathcal{S}}_7,{\mathcal{S}}_8$. This group of samples has yield strengths higher than $205\mathrm{M}\mathrm{P}\mathrm{a}$ and tensile strengths higher than $515\mathrm{M}\mathrm{P}\mathrm{a}$, which respectively are minimum requirements specified by ASTM F3184-16 standard. The samples printed by ${\mathcal{S}}_1$ and ${\mathcal{S}}_2$ have poor tensile performance due to lack of fusion of metal powders, consistent with their observed surface morphology (Figure 5). Those printed with the highest energy densities, i.e. ${\mathcal{S}}_9$ and ${\mathcal{S}}_{10}$, have slightly lower tensile strengths than those of highest performance, e.g. ${\mathcal{S}}_5$, ${\mathcal{S}}_6$, ${\mathcal{S}}_7$, due to the residue stress.

Figure 7. Tensile performance of coupons printed with the same parameter settings as reference samples.

In Figure 7(b), we show elongation at break of tensile coupons printed with the same parameter settings as reference samples. The presence of residue stress significantly reduces the ductility of samples printed with parameter settings ${\mathcal{S}}_9$ and ${\mathcal{S}}_{10}$. However, their ductility is still higher than that of samples printed with energy density of $0.5E$, a result consistent with the discontinuous melting tracks and incomplete powder melting shown in Figure 5. For samples printed with parameter setting ${\mathcal{S}}_4$, the reduced yield strength, ultimate strength and ductility may result from instability of melt pool caused by high printing speed.

The above results demonstrate that (1) different printing parameter settings could result in similar surface appearances and mechanical performances and (2) there is a strong correlation between possible defects and surface appearances observed during printing. In particular, surface appearances of samples printed by parameter settings ${\mathcal{S}}_1$, ${\mathcal{S}}_2$, ${\mathcal{S}}_4$ are associated with lack of fusion and poor tensile strength, while those by ${\mathcal{S}}_9$ and ${\mathcal{S}}_{10}$ are associated with overheated printing and reduced ductility due to residue stress. Samples printed by parameter settings ${\mathcal{S}}_3$, ${\mathcal{S}}_5$, ${\mathcal{S}}_6,{\mathcal{S}}_7$, ${\mathcal{S}}_8$ have smooth surface texture and tensile strength meeting minimum requirements specified by ASTM F3184-16.

As a result, in subsequent predictive analysis using ML, we adopt the following labels to summarise the relations between surface appearances and possible defects: areas predicted as ${\mathcal{F}}_1,{\mathcal{F}}_2,{\mathcal{F}}_4$ are labelled lack-of-fusion; areas predicted as ${\mathcal{F}}_9$, ${\mathcal{F}}_{10}$ are labelled overheated, and areas predicted as ${\mathcal{F}}_3,{\mathcal{F}}_5,{\mathcal{F}}_6,{\mathcal{F}}_7$, ${\mathcal{F}}_8$ are labelled anomaly-free.

3.3. Evaluation of trained machine learning network

We evaluate the ML model’s robustness using separate datasets obtained from samples of wider variety of shapes, i.e. cubes and spiral bevel gear. In Figures 8 and 9, we show representative predictions of the ML model on cubes printed with varying printing parameter settings and representative predictions at several built heights of a spiral bevel gear printed with the parameter setting ${\mathcal{S}}_8$. We note that our ML model differentiates unfused powder areas from laser-scanned areas with remarkably high accuracy, despite geometrical differences between reference samples and the tested samples.

Figure 8. Representative predictions of the ML model on cubes printed with varying printing parameter settings.

Figure 9. Representative predictions at different built heights of a spiral bevel gear.

We recall that our ML model is incorporated with data augmentation (see Section 2.3), which helps to prevent overfitting ML models. To confirm the effectiveness of data augmentation in preventing overfitting, we train the ML model without data augmentation and subsequently observe that it performs well on the testing dataset obtained from the reference samples but poorly on those obtained from the printing of the cubes and the spiral bevel gear. In particular, the ML model without data augmentation is not capable of distinguishing unfused powder areas from laser-scanned areas for printed samples having different shapes from that of reference samples.

To further quantify our ML model’s ability in differentiating the unfused powder and the laser-scanned areas, we show in Figure 10 the global accuracy, $MeanIoU$ and $MeanBF$ scores for unfused powder areas and laser-scanned areas of both the cubes and the bevel gear. Here we note that the areas scanned with laser include those indicated by either lack-of-fusion, anomaly-free, or overheated. As a result, all classes indicating lack-of-fusion, anomaly-free and overheated are merged into one indicating laser-scanned areas to calculate the scores for the areas scanned with laser and the areas of unfused powder.

Figure 10. Performance scores, i.e. global accuracy, MeanIoU and MeanBF, of the selected ML model for cubes and bevel gear.

We observe that the $MeanIoU$ and $MeanBF$ scores are lower for the bevel gear compared to those for the cubes as the ML model does not perform well for thin features such as teeth on top of the spiral bevel gear (Figure 9(d)). Nevertheless, the $MeanIoU$ and $MeanBF$ scores for both cases are higher than $0.8$, suggesting that our ML model is well capable of differentiating the laser-scanned areas and unfused areas.

To evaluate the predictive capability of the ML model, we show in Figure 11(a) the distribution of class labels identified by the ML model over the entire image stack obtained from cubes printed with varying printing parameter settings, and in Figure 11(b) the corresponding surface roughness of the printed cubes. For those printed with the parameter settings ${\mathcal{S}}_1$, ${\mathcal{S}}_2$ and ${\mathcal{S}}_4$, high percentages of the laser-scanned areas are predicted as lack-of-fusion, consistent with their correspondingly high surface roughness values. This is also consistent with the known correlation between low laser energy density and lack-of-fusion between printed layers. Likewise, for those printed with the parameter settings ${\mathcal{S}}_3$, ${\mathcal{S}}_7$ and ${\mathcal{S}}_8$, more than $95\text{\%}$ of the laser-scanned areas are predicted as anomaly-free, consistent with their low surface roughness, i.e. $\mathrm{R}\mathrm{a}<12$. We note that predictions made for cubes printed by a single printing parameter setting are not uniform throughout the entire image stack, e.g. a noticeable portion of the laser-scanned areas of cubes printed with ${\mathcal{S}}_4$ is predicted as anomaly-free. These inhomogeneous predictions may originate either from variation in surface appearance caused by uncontrolled fluctuation of printing conditions, e.g. unstable air flow, or change in thermal distribution of the test samples. Nonetheless, further research incorporating thermal history and information from previously printed layers using Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber 1997) module may greatly enhance capability of this ML model toward prediction of defects based on surface.

Figure 11. (a) Distribution of class labels identified by the ML model for cubes; each column corresponds to the entire image stack of a distinct printing parameter setting. (b) Surface roughness of cubes printed with varying printing parameter settings.

3.4. Anomaly detection

To demonstrate the anomaly detection capability of our ML model, we perform predictions on images captured after laser scanning and after powder recoating of a sample printed using parameter setting ${\mathcal{S}}_8$ with overhanging structures (Figure 12(a)), which typically cause super-elevation, i.e. an anomaly potentially endangering structural integrity of L-PBF products. We note that super-elevation typically occurs due to overheating of sintered powder, causing elevation of the overheated areas from the surrounding. The elevated sintered areas typically are not covered by powder in the subsequent powder recoating step. As a result, an area suffered from super-elevation is detected if it is identified as laser-scanned in the image captured after powder recoating.

Figure 12. (a) Geometry of sample with overhanging structures. (b) Images and corresponding analysis for build height z = 7.5 mm. From left to right: image captured after laser scanning; image after powder coating; anomaly predictions after laser scanning; anomaly prediction after powder coating. (c) Images and anomaly predictions for build height z = 10 mm. Arrow indicates powder recoating direction.

In Figure 12(b), we show images captured after laser scanning and after powder coating for a layer at build height $z=7.5\mathrm{m}\mathrm{m}$ and corresponding anomalies predictions by our ML model. For structures with low overhanging angles (less than ${55}^{\circ }$), most laser-scanned areas are predicted as anomaly-free by the ML model. As the overhanging angle increases, more and more laser-scanned areas are predicted as overheated. This observation is consistent with the thermal characteristics of overhanging structures in L-PBF: structures with high overhanging angles have more unfused powder underneath, hence are prone to local overheating and subsequently cause elevation of the built part due to low thermal conductivity of unfused powder. In the subsequent stage, i.e. after powder recoating, the areas predicted as laser-scanned are identified as those super-elevation.

We highlight that the areas identified as overheated after laser scanning are not always the same as those identified as super-elevation in the subsequent after powder recoating stage, as demonstrated in Figure 12(c) for the overhanging structures with angles ${45}^{\circ }$ and ${50}^{\circ }$ at build height $z=10\mathrm{m}\mathrm{m}$. This implies that overheating does not always cause super-elevation. Nevertheless, close proximity of overheated and super-elevation areas observed in our experiment strongly suggest that surface appearance can be used as an early warning tool for possible defects such as super-elevation.

To quantitatively evaluate overheating and super-elevation in overhanging structures, we calculate the normalised overheated area $A_o$ and super-elevation area $A_e$ in each layer using the predictions of ML model and the designed laser-scanned areas of overhanging structures. In Figure 13, we show the dependences of $A_o$ and $A_e$ on build height $z$ for overhanging structures of angles from ${15}^{\circ }$ to ${60}^{\circ }$. We observe a strong correlation between $A_o$ and $A_e$ for large overhanging angles, ${55}^{\circ }$ and ${60}^{\circ }$. For the structure having overhanging angle of ${50}^{\circ }$, super-elevation is absent, while overheating effect is detectable at top layers, i.e. for build height larger than $8.5\mathrm{m}\mathrm{m}$, indicating that this is the maximum overhanging angle possible for printing without super-elevation using setting ${\mathcal{S}}_8$.

Figure 13. (a) Normalised overheated area A_o vs. build height z. (b) Normalised super-elevation area A_e vs z for overhanging structures of different angles. Both A_o and A_e are detected by the ML model on sample with over hanging structures.

We exhibit the anomaly detection capability of our ML model by showing an actual sample printed with overhanging structures and its corresponding 3D model overlayed with super-elevation locations (Figure 14). It is evidenced that severe elevation and overheating occur in the printed sample for overhanging angles ${55}^{\circ }$ and ${60}^{\circ }$ similar to the super-elevation regions indicated in the 3D model. This strongly demonstrates the practical use of our ML model to monitor and detect possible defects that occurred in L-PBF process.

Figure 14. (a) Actual printed overhanging sample, the arrows indicate super-elevation for the two largest overhanging angles. (b) 3D model reconstructed from CAD file overlapping with super-elevation regions detected by the ML model.

Finally, we provide a separate demonstration of our ML model’s capability in detecting anomalies during the printing of bridge structures without supports. In our sample design, the bridge span varies from $1\mathrm{m}\mathrm{m}$ to $5\mathrm{m}\mathrm{m}$ (Figure 15(a)). We perform predictions on images captured both after laser scanning and after powder recoating. At the bottom surfaces of the unsupported bridges, i.e. $z=7.6\mathrm{m}\mathrm{m}$, the ML model detects large overheating areas for bridge span $3\mathrm{m}\mathrm{m}$, $4\mathrm{m}\mathrm{m}$ and $5\mathrm{m}\mathrm{m}$ in the image captured after laser scanning (Figure 15(b)). However, these overheated areas do not cause extensive elevation, as observed from the subsequent image captured after powder recoating. At larger build height, e.g. $z=9\mathrm{m}\mathrm{m}$, prediction on image captured after laser scanning shows insignificant overheating, except a few locations where super-elevation occurs at the sample’s edge, as there are sufficient printed layers below the bridges (Figure 15(c)).

Figure 15. (a) Geometry of sample with unsupported bridges. (b) Images and anomaly predictions for build height z = 7.6 mm. From left to right: image capture after laser scanning; image after powder spreading; anomaly prediction after laser scanning; anomaly prediction after powder coating. (c) Images and anomaly predictions for z = 9 mm. Arrow indicates powder recoating direction.

In Figure 16, we further quantify overheating and super-elevation for build height from $6\mathrm{m}\mathrm{m}$ to $10\mathrm{m}\mathrm{m}$ by showing the normalised overheated area $A_o$ and super-elevation area $A_e$ for each layer, calculated using the bridge span areas, versus build height. In Figure 16(a), significant increase in overheated areas for $3\mathrm{m}\mathrm{m}$, $4\mathrm{m}\mathrm{m}$ and $5\mathrm{m}\mathrm{m}$ bridge spans when the build height reaches the bridge height $z_0=7.5\mathrm{m}\mathrm{m}$, consistent with the defective regions of the actual printed part (Figure 17). We however note that overheating only persists from $z=7.5\mathrm{m}\mathrm{m}$ to $8.1\mathrm{m}\mathrm{m}$, beyond which $A_o$ reduces to a small level $(<20\text{\%})$. It is also worth noting that super-elevation for unsupported structures remains low, as shown in Figure 16(b).

Figure 16. (a) Normalised overheated area A_o vs. build height z. (b) Normalised super-elevation area A_e vs z for bridges of different spans. Both A_o and A_e are detected by the ML model.

Figure 17. Actual printed sample with unsupported bridges.

4. Conclusion

In this work, we introduce and implement a semi-supervised machine learning (ML) approach for anomaly detection in L-PBF processes. First, we obtain a reference set of layer-wise images acquired from monitoring reference samples printed with various pre-defined parameter settings. We train our ML model to classify surface appearances in the reference images. By examining the correlation between surface appearances and post-process characteristics such as surface roughness, morphology and tensile strength, we establish key surface appearances indicative of quality of the printed samples, including anomaly-free, lack-of-fusion and overheated. We then experimentally confirm that our ML model’s capability in classifying these quality-indicative surface appearances can be used for either defect detection or quality prediction of samples having various geometries, e.g. cubes, bevel gears, overhanging and unsupported structures.

The semi-supervised ML approach to quality monitoring of L-PBF processes can be expanded to other materials and 3D printing processes thanks to its simplicity in labelling training data. Further work using more extensive training datasets or incorporating thermal history by adopting Long Short-Term Memory (LSTM) module may be beneficial to both enhancing quality prediction capabilities of the ML model and confirming the occurrence of defects.

Acknowledgement

The computational work for this article was performed using resources of the National Supercomputing Centre, Singapore (https://www.nscc.sg).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research is supported by Republic of Singapore’s National Research Foundation Singapore under its Innovation Cluster Programme (NAMIC), and VinUniversity, Vietnam.