Regression-based detection of missing boundaries in multiphase polycrystalline microstructures

ABSTRACT

An efficient alternative approach for detecting missing boundaries in micrographs is presented in the current work. By treating the missing boundaries as a class of object, a suitable detection algorithm is extended to realise discontinuities in interfaces separating phases and grains. The metrics, including precision and recall, estimated during the development of the model indicate noteworthy performance. Moreover, a direct comparison with actual situations attests to the accuracy of the current approach in detecting missing boundaries across different multiphase polycrystalline micrographs.

KEYWORDS:

• Missing boundaries
• microscopy errors
• boundaries discontinuities
• polycrystalline system
• machine learning

The properties of a material are broadly a reflection of its structure on the microscopic scale, referred to as its microstructure. A convincing understanding on the behaviour of a material, under a given condition, can be gained by investigating this microstructure. The microstructures of materials are often complex and include more than one phase and/or numerous grains [1, 2]. The accuracy of the analysis on the behaviour of such materials relies hugely on the depiction of the corresponding multiphase polycrystalline microstructure. However, micrographs rendered by both optical and electron microscopy generally contain certain irregularities, thereby introducing a deviation from the actual microstructure, which ultimately, might lead to misinterpretation [3–5]. Defects in micrographs can stem either from improper sample preparation or ill-defined imaging configurations. One of the most common features in observations o f micrographs is that of missing boundaries [6, 7]. In other words, when a multiphase polycrystalline microstructure is captured, not all the interfaces separating the phases and grains are accurately represented in the corresponding micrograph. This lack of complete depiction of the interfaces introduces missing boundaries in the micrograph. Owing to the significant influence of grain (and phase) boundaries on the behaviour of materials, and their pivotal role in any microstructural transformation, added attention has been given to these features [8–10].

The process of eliminating missing boundaries in micrographs invariably begins with identifying this spatially distributed defect. Detecting the missing boundaries manually is an arduous task, particularly when the microstructure comprises several grains and phases. Therefore, computer vision techniques are generally adopted to identify the missing boundaries in the micrographs. Within the framework of reconstructing grains in three-dimensional polycrystalline microstructures, an approach has recently been proposed for detecting and rectifying missing boundaries [11, 12]. This computationally rigorous technique treats the boundaries as the foreground and separates it from the less-relevant features of the micrograph including the bulk grains and phases [13]. Edge-detection algorithm aides in separating the boundaries from the background [14]. The micrograph now exclusively representing the boundaries is superimposed on the similarly treated neighbouring sliced images, which are generated as a part of the investigation, to identify the missing boundaries. Though this technique is exhaustive, it is also computationally demanding and is rather bound within the associated framework. In view of the characteristic steps involved with the existing approach, the proposed technique seemingly offers an efficient alternate for rapid detection of missing boundaries in multiphase polycrystalline systems.

A missing boundary in a micrograph can be viewed as a definite localised discontinuity in an otherwise continuous network. Such consideration allows the missing boundaries to be treated as objects in a given image. Correspondingly, upon handling the missing boundaries as objects, in the present study, an efficient object-detecting algorithm is extended to identify these defects in the micrograph. The approach adopted for developing the current missing-boundaries detection model is schematically illustrated in Figure 1. Considering that the detection model is essentially trained to identify and locate missing boundaries, its development begins with acquisition of the relevant data. As opposed to conventional data, which primarily comprise of numerical or categorical information, micrographs of multiphase polycrystalline systems serve as the building blocks for the present model.

Figure 1. A schematic representation of the steps involved in extending object-detection algorithm to identify missing boundaries in micrographs. The processing of the micrographs indicates the random removal of the sections of the boundary network to introduce the missing boundaries.

The micrographs employed to train and validate the model are numerically generated by suitably discretising a two-dimensional domain. Resemblance of the numerically generated micrographs to the experimentally observed microstructures is ensured by involving a proven scheme called a Voronoi tessellation [15]. Stated otherwise, a tessellation scheme, which has already been sufficiently shown to render polycrystalline microstructures with features mirroring the experimentally-captured micrographs, is adopted to generate multiphase polycrystalline images [16–18]. Twenty micrographs, each encompassing approximately 1000 grains of average size ranging from 20 to 30 µm is generated using the tessellation. One of three distinct phases are randomly, yet proportionately, assigned to each of the grains, through an appropriate colour scheme as seen in Figure 1, in order to develop a three-phase polycrystalline micrographs. Such triplex micrographs characteristically include boundaries separating grains of identical chemical composition and ones associated with distinct phases. In other words, the micrographs considered in the present work accommodate interfaces between grains and phases.

The micrographs, before integrating with the model, are sliced into 200 workable portions with each comprising 100 phase-associated grains. These 200 sliced micrographs constitute the principal dataset illustrated in Figure 1. Since microstructures are numerically generated, the boundaries separating the phases or grains are hardly missing. Therefore, an additional processing step is introduced, as shown in Figure 1, prior to the training and validation of the model. In this preparatory step, an image-processing tool is extended [19], to randomly remove interfaces between the phases and grains in each of the sliced micrographs. The resulting processed dataset of micrographs with missing boundaries is subsequently employed to develop the detection model. A major portion of the processed dataset (80%) is employed to train the model, with the validation being performed on the remaining portion (20%).

Object detection, in the framework of computer vision and machine learning, is essentially a two-step process [20]. It begins with identifying regions of interest in a given image, followed by labelling them using a suitable classifier. These characteristic steps are iterated adequately to convincingly detect the objects. Irrespective of its accuracy, this detection technique, primarily owing to its underlying approach, is rather sluggish and consumes relatively excess computational resources. The algorithm employed in the present work, on the other hand, facilitates rapid detection through regression [21]. Missing boundaries, in the current framework, is detected by employing a well-known technique called YOLOv5 [22, 23]. Correspondingly, the detection of missing boundaries proceeds by discretising the micrographs into identical M × M grids. Assuming to be the spatial centre of a missing boundary, each of these cells predict a bounding box supposedly encompassing the object. The prediction associates each of the grids with parameters of the corresponding bounding boxes which include its centre and dimension. Moreover, the confidence of finding the missing boundaries in the predicted bounding boxes are estimated in relation to the ground truth, and subsequently augmented to the existing parameters of the cells. Depending on the estimated confidence, the bounding boxes accurately encapsulating the missing boundaries are sustained at the expense of others.

The training of the current regression-based detection approach involves introducing the ground truth by manually labelling of the objects. Put differently, the model to detect missing boundaries is trained by manually inserting bounding boxes around the corresponding defects in the micrographs. The ground truth defined in the form of the manually introduced labels, besides training, aides in refining the model during its validation. The performance of the model during testing and validation is ascertained through appropriate metrics and are graphically represented in Figure 2. The losses which indicate the deviation of the prediction from the ground truth are illustrated in Figure 2a. While the box loss quantifies the disparity between the parameters of the manual labels and the predicted bounding boxes, the difference in the confidences indicating the presence of the missing boundaries is expressed as the object loss. For a batch of four micrographs, Figure 2a shows that both box and object losses become increasingly negligible, as the number of epochs reach 1000. The minimal losses affirm the accuracy of the model in detecting missing boundaries. Other hyperparameters, besides epoch, are tuned to enhance the performance of the technique which includes learning rate, weight decay and momentum assuming values of 0.01, 0.0005 and 0.937, respectively.

Figure 2. (a) Change in the box and object loss with number of epochs during training and validation of the model. (b) Precision-Recall curve of the model in detecting the missing boundaries.

The accuracy of the model in detecting the missing boundaries, in addition to the losses, is gleaned from two other parameters referred to as precision and recall. Precision is the ratio of the true positive, indicating accurate detection of missing boundaries, and the sum of true and false positives, which reflect the incorrect perception of regular interfaces as missing by the technique. On the other hand, the ratio of true positive and sum of true positive and false negative, representing the undetected missing interfaces, is termed as recall. In other words, while the precision measures the performance of the model in the light of its inaccurate detection, whereas the failure to identify the missing boundaries is included in the calculation of recall. Both precision and recalled are estimated as the trained approach is validated with suitable dataset. When expressed in the form of precision-recall plot, the area under the curve reflects the average precision of the model (AP). Considering that present approach detects only one class of object, AP is also equivalent to the mean Average Precision (mAP). The precision-recall plot for the current missing-boundaries detecting model is shown in Figure 2b. An ideal model which detects all the objects in a given image, would yield an absolute mean Average Precision of mAP = 1.0. In Figure 2b, the accurate performance of the model in detecting the missing boundaries is indicated by the marginal deviation of its mean average precision, mAP = 0.985, from the absolute value. Moreover, it is vital to note that the validation dataset involved in the calculation and plotting of the precision-recall curve includes experimental micrographs. Accordingly, Figure 2b unravels a convincing performance of the present technique over the physical microstructures.

After sufficient training and validation, the model is allowed to detect missing grain boundaries in hitherto unknown micrographs. The ability of the present approach to realise the missing boundaries is illustrated in Figure 3 by placing the manually labelled micrographs alongside the respective machine-detected images. Evidently, almost all missing boundaries in the micrographs have been accurately detected by the current model. Irrespective of its nature, either separating phases or grains, the approach as shown in Figure 3 identifies and locates the missing sections of the interface.

Figure 3. The ground truth of missing boundaries in multiphase polycrystalline micrographs is compared with the detection of the present model.

In order to ensure that detection of the current model is not primarily dependent on the colour scheme associated with the micrographs, grey-scale representation of these images are generated and the approach is allowed to explore for missing boundaries. The ability of the model to detect missing boundaries in grey-scale multiphase polycrystalline microstructures is shown in Figure 4. The accurate performance of the approach, made evident in comparison with manually labelling, remains unaltered despite the change in the colour scheme. The sustained accuracy exhibited by the model, irrespective of the general appearance of the micrographs, indicate that the detection is primarily based on the relevant features associated with the boundaries. Therefore, this approach can directly be extended to realise the missing boundaries in the experimental micrographs. This claim is substantiated by analysing experimental microstructures reported in the literature. Attempts will be made in near future to augment this approach with image processing techniques to rectify missing boundaries, besides detecting them. Moreover, the efficacy of the current approach in detecting long-range missing boundaries will be examined in subsequent investigations.

Figure 4. Comparison of ground truth and current predictions in grey scale depiction of the three-phase polycrystalline micrographs.

 Notes on contributors

The corresponding author, Prince Gideon Kubendran Amos, is an assistant professor in the department of Metallurgical and Materials Engineering at the National Institute of Technology Tiruchirappalli. He is also the Principal Investigator of the Theoretical Metallurgical Group, in which the other author Manoj Prabakar is a research fellow.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

PGK Amos thanks the financial support of the Science & Engineering Research Board (SERB) under the project SRG/2021/000092.