Vibration and infrared thermography based multiple fault diagnosis of bearing using deep learning

ABSTRACT

       The occurrence of multiple faults is a practical problem in the bearings of rotating machines, and early diagnosis of such issues in an intelligent manner is vital in the era of industry 4.0. The present work investigated various combinations of bearing faults, including dual and multiple fault conditions. Two prevalent fault diagnosis methods were employed: vibration monitoring using time-frequency scalograms extracted through Continuous Wavelet Transform (CWT) and a non-invasive Infrared Thermography (IRT). A 2-D Convolutional Neural Network (CNN) was used to classify various combinations of fault conditions through automated feature extraction. The proposed methodology was validated at two constant speed conditions of 19 Hz and 29 Hz and continuously accelerated and decelerated speed conditions in the range of 19 Hz - 29 Hz. Adequate accuracy was achieved in both dual and multiple fault conditions in the case of vibration-based fault diagnosis, with a range of 99.39 % to 99.97 %. Meanwhile, in the case of proposed IRT-based fault diagnosis, 100 % classification accuracy was achieved for dual and multiple faults in all conditions. These results signify the robustness and reliability of the proposed methodology for dual and multiple fault diagnosis in bearings at constant and varying speed conditions.

KEYWORDS:

• Fault detection
• infrared thermography
• continuous wavelet transform
• fault classification

1. Introduction

Rotating machines are integrated part of every production and power generation system. Fault detection and isolation of rotating machine components are crucial as these faults could lead to a complete or partial shutdown of whole system. Also, these conditions could lead to accidents in the industries [1–3]. Bearing is the most important component of these rotating machines, and correct and timely diagnosis of faults in the bearing is important for their safe operation [4]. The faults can be found in any component of bearing, such as rolling element, inner race, outer race, or cage. However, the faults in actual industrial conditions occur in various combinations and may produce catastrophic conditions for rotating machines and may fail eventually. There are various sensors available for diagnosing these faults based on signals such as vibration, current, noise, temperature, etc. Among these, vibration is well established and older technique [5,6]. Researchers have used vibration monitoring assisted with various efficient methods such as Variational Mode Decomposition (VMD), and Maximum Correlation Kurtosis Deconvolution (MCKD) for the fault diagnosis of rolling element bearing [7]. The advantages of the vibration monitoring are well defined, but still, it has various limitations such as contact with machine structure, frequent calibration, signal noise, damage to the sensor after long-term contact, etc. Apart from vibration-based fault diagnosis, another important method is infrared thermography which uses information based on temperature change and can be quickly and effectively used for fault diagnosis in rotating machine components in a non-invasive manner with strong immunity against the speed fluctuations [8]. A thermal image is also known as a thermogram, which is simply a map of thermal energy. As an inherent characteristic, heat is radiated by each object, and its amount depends on the surface temperature [9]. This phenomenon encourages the use of IRT as a prominent condition monitoring tool for faults in the components of rotating machines, such as bearing faults, misalignment [10,11] and faults in various electrical rotating machinery [12–14].

In the practical working environment of rotating machines, the faults occur in various combinations. However, in the case of single fault conditions, commensurate modality sensors perform well, but in an actual working environment where faults occur in combination, a single modality sensor may be found less reliable. Also, in actual industrial scenarios, there are frequent fault conditions that, in combination with other kinds of faults, generate harsh conditions for a particular type of sensor capable enough to capture fault in normal conditions. One such situation is the lack of lubrication that leads to most of the faults in bearings and also causes a source of localised heat generation in the bearings. This limits the applicability of contact-type sensors such as vibration sensors and needs to be used in a complementary manner with other sensors having an effective non-contact information capturing capability regarding fault conditions [15,16]. So, in that complex environment where real-time decisions are to be made for effective and efficient maintenance of machines, the inspection with commensurate sensors is less reliable, and there is an urgent need for the involvement of different modality sensors [17]. Further, researchers have done fault detection considering a single fault which is rarely the situation in practical working conditions, but less work is reported to diagnose multiple fault conditions in an unstable speed environment for bearings. These conditions demand extra effort as it is challenging to diagnose the characteristic frequencies and requires rigorous signal processing [18].

There are various approaches available for fault diagnosis, including statistical methods, physics-based methods, and data-driven methods such as machine learning and deep learning. Statistical methods found their applications in fault diagnosis and other fundamental problems related to industrial and medical applications [19–21]. These methods can be assisted with various efficient optimisation algorithms for accurate and efficient results [22,23]. The physics-based methods are based on the modelling of machine components and subsequent analysis in different fault conditions [24]. However, in complex conditions, this method invites labour-intensive and time-consuming situations. In recent times, data-driven approaches such as machine learning and deep learning got much attention from the researchers and engineers, and advancement in computational power became a boon for these methods [25,26]. Machine learning-based methods got popular in recent decades and found effective solutions for fault diagnosis in rotating machines and other engineering problems [27,28]. However, methodologies based on machine learning involve mandatory feature extraction and hence require more effort. As opposed to this, deep learning became the front player due to its automated feature extraction capability, and growth in computational power makes it more prevalent among fault diagnosis researchers [29]. The most widely used deep learning algorithms involve Convolutional Autoencoder (CAE), Deep Belief Network (DBN), Deep Boltzmann Machine (DBM), CNN, and Recurrent Neural Network (RNN) [30,31]. Among these techniques, CNN finds its place very strongly for the multi-sensor information and classification of faults as it has a lesser dependency on preprocessing [32]. Multiple layers with activation functions leverage the CNN in extracting the complex features from data with single sensor data and multisensory data [33]. A multi-sensor vibration-based classification was done using a Le-Net 5 CNN using the information from images converted by 1-D vibration signals for fault detection in a wind power test rig [34]. In another work, various faults related to unbalance, rub, bearings, and feature extraction were done using CNN and Bag of Visual-Word (BoVW), aided by an algorithm called Scale-Invariant Feature Transform (SIFT). The faults were successfully classified with comprehensive accuracy using a Support Vector Machine (SVM) classifier [35]. The usability of IRT-based methodology against vibration monitoring proved for the gear fault detection using CNN [36].

In conclusion, a single or commensurate modality capturing sensor is limited by various conditions generated due to severe fault conditions in the components of rotating machines, and heat generation due to multiple faults is the most obvious and challenging one among these. So, there is a need to use non-invasive type sensors complemented with commercial contact type robust vibration transducers. This leads to a multilateral, reliable, and high-level fault recognition, ensuring timely, and effective decision-making regarding fault identification and classification.

In the present work, a vibration and infrared thermography-based approach is presented for a reliable and robust fault diagnosis in ball bearings based on deep learning. The important contributions of the present research are:

1. Various combinations of faults in ball bearing were experimentally analysed, including dual and multiple fault conditions at constant, accelerated, and decelerated speed conditions.

2. A 2-D CNN was designed to develop an automated feature extraction-based fault diagnosis of bearing faults in an intelligent manner and eliminate the efforts required for manual feature extraction.

3. Vibration monitoring through CWT-based time-frequency scalograms was used for the analysis and classification of various combinations of faults.

4. Further, a non-invasive infrared thermography was proposed that works on the heatmap or temperature distribution at the bearing surface. Moreover, the performance was comparatively analysed against vibration monitoring in single, dual, and multiple fault conditions.

The remaining part of the paper consists of the following sections. In Section 2, an explanation of experimentation and data acquisition is given. Section 3 describes the proposed deep learning-based fault diagnosis methodology. Section 4 elaborates the obtained results and the conclusions in Section 5.

2. Experimentation and data acquisition

The experimentation was carried out on a robust bearing test rig having a DC motor of 220 V and 2 HP as a prime mover. The rig has strong support to facilitate various experimental simulation conditions like speed variation and different loading conditions, etc. Figure 1 shows the experimental setup used for the acquisition of vibration and thermography data. A uniaxial accelerometer (B&K, type 4370) with a sensitivity of 10 pC/ms⁻² and a thermal imaging camera (FLIR-P640) having a frame rate of 7 with a resolution of 640 × 480 were used for the data acquisition. A sampling frequency of 6.4 kHz was utilised for acquiring the vibration data. For IRT, the most crucial parameter is emissivity, which depends on inspected material’s surface properties, and its typical value ranges from 0 to 1. In the present case, it is taken as 0.64 according to the bearing housing material [37]. Another critical parameter is the apparent temperature that can be measured with standard methods; however, it has lesser significance for materials having higher emissivity. The distance between the camera and the target was 80 cm to focus on the bearing housing for accurate data collection.

Figure 1. Experimental setup for data acquisition.

Experiments were carried out at two constant speeds, i.e. 19 Hz and 29 Hz, and with continuously varying speeds as ramp up and ramp down of 10 Hz in five minutes, as given in Table 1. Also, the rig was run for 45 minutes before capturing the data for thermal stability and reaching steady-state conditions, as followed in [38]. There was a rest of 60 minutes for the rig to reach room temperature for each fault condition.

Table 1. Dual and multiple fault conditions in bearing.

Dual Fault Conditions		Multiple Fault Conditions		Speed conditions
Label	Constituent faults	Label	Constituent faults	1140 rpm (19 Hz) 1740 rpm (29 Hz) Ramp up of 10 Hz (19 Hz − 29 Hz) Ramp down of 10 Hz (29 Hz − 19 Hz)
IRBD	IR + BD	MF1	IR + OR + LB
IRLB	IR + LB	MF2	IR + OR + BD
IROR	IR + OR	MF3	OR + BD + LB
ORBD	OR + BD	MF4	BD + LB + IR
ORLB	OR + LB	MF5	IR + OR + BD + LB

The combination of various fault conditions was experimentally simulated to analyse the dual and multiple fault conditions as given in Table 1. These faults include a combination of inner race fault (IR), outer race fault (OR), ball fault (BD), and lack of lubrication (LB). There are a total of five dual fault conditions that are considered. The combination of inner race fault and ball fault is depicted as ‘IRBD’; a combination of inner race fault and lack of lubrication is depicted as ‘IRLB’; a combination of inner race fault and outer race fault is depicted as ‘IROR’; a combination of the outer race and ball fault is depicted as ‘ORBD’, and a combination of outer race fault and lack of lubrication is depicted as ‘ORLB’ as given in Table 1. Similarly, a total of five multiple fault conditions were considered in the present work. Various fault conditions were combined to produce multiple fault conditions, where more than two faults can co-exist. As given in Table 1, ‘MF1’ - ‘MF4’ consist of three fault conditions, and ‘MF5’ consists of four fault conditions.

The different fault conditions are shown in Figure 2, with one example of dual fault as ‘IROR’, and one example of multiple faults as ‘MF2’. For each experiment, except for the lack of lubrication condition, 100 % lubrication was done. Here 100% lubrication means 2.3 gm as recommended by the manufacturer, and lack of lubrication means 0 % lubrication. A grease SKF VKG 9/0.5IN was utilised for the lubrication of bearings.

Figure 2. Bearing conditions (a) healthy (b) inner race fault (c) outer race fault (d) ball fault (e) dual fault (f) multiple fault.

3. Proposed methodology

In this work, a deep learning-based strategy using CNN was adopted to diagnose ball bearing in various fault conditions. Here a 2-D CNN was designed, and image data was used for vibration and IRT. For the conversion of 1-D vibration data into images, the scalograms were obtained through CWT. Similarly, thermal images were extracted with proper RoI from the thermal video recorded by thermal camera.

3.1 Time-Frequency image extraction using CWT for vibration data

Traditional time domain and frequency domain analysis has the restriction on information from both the domain. Time-frequency-based analysis founds it application in effective way for the diagnosis of rotating machine components such as bearings [39]. For time-frequency analysis, various techniques can be used, such as Short-Time Fourier Transform (STFT), CWT, Winger Ville distribution (WVD), and Pseudo-Winger Ville Distribution (PWBD) [40,41]. Using these techniques, 1-D data can be converted into 2-D images and further utilised for feature extraction. Such an example was presented in [42], where a 1-D acoustic emission data was converted into a 2-D image and was utilised for classification using a CNN. A traditional Fast Fourier Transform (FFT) uses a basis function complex exponential form that includes sine and cosine but does not talk about occurrence time for the frequencies in the signal. The STFT uses windowing to analyse a section of the signal to catch the frequency content at a particular time but suffers from the curse of the uniqueness of good resolution either in the time or frequency domain. Against this, CWT provides better temporal and frequency resolution [43].

The decomposition of signals through CWT is found effective as it decomposes signals into a series of superpositions of the mother wavelet by doing scaling and translation [44]. Decomposition based on wavelet uses an oscillatory and time-localised function called mother wavelet that helps to cope with the issues associated with other decomposition methods [45]. The source function is continuous in time as well as frequency domain. Mathematically CWT of a signal x(t) is given as:

(1) $X_w(\tau ,b)=\frac{1}{\sqrt{b}}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}x(t){\psi }^{\ast }(\frac{t-\tau }{b})dt$(1)

Here ${\psi }^{\ast }$is regarded as complex conjugate of the mother wavelet $\psi (t)$, which is nothing but a continuous function in time as well as frequency domain, and $\tau$is translational factor or time shift, and b is scale factor. The $x(t)$represents the original signal.

The purpose of scale factor is either dilation or compression of the original signal. The core target of the mother wavelet is basically to yield a source function to produce the daughter wavelets, which can be related to the mother wavelet as its translated and scaled versions. The choice of proper mother wavelet function depends on various factors such as variance in time and frequency domain and particular application. A ‘morlet’ wavelet is one of the widely used wavelets and is given as:

(2) $\psi (t)=e^{j{w}_{0}t}{e}^{-\frac{t^2}{2}}$(2)

Here, $w_0$can be regarded as centre of support found in Fourier space. In the present work, an analytic form of ‘morlet’ wavelet was used (popularly known as ‘amor’ wavelet) as a mother wavelet. It has a unique feature that provides equal variance in time and frequency. To convert 1-D vibration signal into a 2-D scalograms, the window-based conversion was adopted using the following equation:

(3) $X=\frac{N-w}{c.w}+1$(3)

Where X is the total number of 2-D images extracted as time-frequency scalograms, N is the total number of data points, w is the window length, and c is the overlap factor which is taken as 50 % for the extraction of CWT coefficients. Figure 3 shows the extraction procedure for CWT-based scalograms. In the present case, the number of 2-D images was formed to tally the number in IRT to establish a fair comparison. So, here the calculated values of X are 2100, N is 1,920,000, and hence the round value of w was taken as 1826. The window-based strategy was adopted to eliminate information loss due to edging. The obtained scalograms contain information about both frequency and time at which a particular frequency is present in the signal.

Figure 3. Extraction of CWT-based scalogram based on overlapping window frames.

3.2 Image extraction for IRT data

As an inherent property, heat is radiated by each object, and its amount depends on the surface temperature and surface conditions. This phenomenon encourages the use of IRT as a prominent condition monitoring tool as faults in the components of rotating machines, such as bearings, etc. [46]. The difference in temperature distribution in faulty conditions as compared to healthy conditions is an important source of information. Basically, infrared thermography makes it possible to visualise the superficial temperatures of a component in a non-invasive way with high resolution. This is achieved by measuring the radiation emitted by the component surface within the infrared region of the electromagnetic spectrum and by subsequently converting these radiations into electrical signals.

A black body is assumed to be the absorber of total incident radiation in the radiation process. Stefan-Boltzmann law is used to describe the emission rate per unit area as:

(4) $Q=\epsilon \sigma T^4\mathrm{W}/\mathrm{m}2$(4)

Here, Q is the total radiated energy per unit area, where ε is called emissivity, $\sigma$ is called the Stefan-Boltzmann constant with an approximate value of 5.67 x 10⁻⁸ Wm⁻² K^−4, and T is the absolute temperature. Emissivity is a material property that is different for different materials and is important for accurate temperature measurement as well as capturing accurate thermal images [47]. The value of emissivity for a perfectly black body is considered as 1. The factors associated with thermography are broadly mentioned in three categories as procedural, technical (material properties, distance, reflected temperature, specifications of camera etc.), and environmental (atmospheric temperature, humidity etc.). The heatmap generated using IRT can be used as an important source of information for diagnosing bearing faults. Thermal images are captured with the help of a thermal camera. This camera basically composed of the optical components (including lens, mirrors, etc.), detector elements, cooling arrangments, and related electronics. The present work extracted the thermal images of bearing from the complete frame by concentrating on RoI, as shown in Figure 4. The raw thermal image can be seen as a temperature map generated through the FLIR tool. The pixel values of RGB thermal are represented in Figure 4. It is clear from the map that the difference at pixel levels makes it a convenient tool for the diagnosis of different fault conditions.

Figure 4. Extraction of thermal images.

The setup was run for about 45 minutes before capturing the thermal data to attend the steady-state condition. However, the background of the whole image may be misled to the process, it is necessary to extract a RoI from the raw thermal images and convert it into greyscale imagess using the scale of [0, 255] and used as input to the CNN and classification of bearing fault conditions. The size of input images was fixed as 96 × 96 as per the input layer of CNN.

3.3 Convolutional neural network

In recent times, the convolutional neural network has been broadly acclaimed by the community of engineers and scientists. The primary reason for that is the capability of automatic feature extraction. In the basic structure of CNN, there are three main layers, i.e. the convolution layer, pooling layer, and fully connected layer [48]. CNN gives a robust framework for learning and hierarchically extracting the features and working closely with the vital eyes [49].

The input provided is in the form of 2-D images, which are further processed through various layers of the CNN structure. In the present case, the scalograms and thermal images are first converted into greyscale images as colour images lead to the curse of dimensionality without any substantial information addition. So, for the training and testing, greyscale images were used as they contain nearly the same information at pixel level as the colour image [50]. This practice has a significant advantage as computational costs get significantly reduced. After getting the input as an image, a convolution operation is performed along with non-linear mapping and pooling, this can be regarded as a single task, and this task is repeated depending on the number of such layers. These layers in the form of a triplet are stacked on one another in a CNN architecture, and the number of these triplets determines the depth of a network. Figure 5 shows the proposed architecture of CNN for vibration and IRT-based fault diagnosis. A detailed description of the proposed CNN architecture is also given in Table 2. In the architecture of constructed CNN, a total of 16 layers are there, including three convolutional layers with batch normalisation and ReLu (Rectified Linear unit) layers. A batch normalisation layer was used as it contributes to faster training with reduced dependency on initial weights and a smooth regularisation of the model. Similarly, three max-pooling layers, one fully connected layer, and one output or classification layer were used with the SoftMax function. Convolution is given as the response of an imaging system, a linear space invariant system, to the input signal. This response is given as:

Figure 5. Proposed CNN for bearing fault diagnosis based on (a) vibration and (b) IRT.

Table 2. Description of various layers in proposed CNN model.

Layer Name	Activations	Learnable
Image input layer	96x96x1	-
Convolutional layer (C1)	96x96x32	Weights: 3x3x1x32 Bias:1x1x32
Batch Normalization layer (B1)	96x96x32	Offset: 1x1x32 Scale:1x1x32
Relu layer (R1)	96x96x32	-
Maxpooling layer (P1)	48x48x32	-
Convolutional layer (C2)	48x48x32	Weights: 3x3x32x32 Bias:1x1x32
Batch Normalization layer (B2)	48x48x32	Offset: 1x1x32 Scale:1x1x32
Relu layer (R2)	48x48x32	-
Maxpooling layer (P2)	24x24x32	-
Convolutional layer (C3)	24x24x32	Weights: 3x3x32x32 Bias:1x1x32
Batch Normalization layer (B3)	24x24x32	Offset: 1x1x32 Scale:1x1x32
Relu layer (R3)	24x24x32	-
Maxpooling layer (P3)	24x24x32	-
Fully connected layer	1x1x6	Weights: 6x4608 Bias:6x1
Softmax layer	1x1x6	-
Output layer	-	-

(5) $\psi (a,b)=\sum _{m=0}^{\mathrm{\infty }}\sum _{n=0}^{\mathrm{\infty }}x(m,n)h(a-m,b-n)$(5)

Where h is regarded as the impulse response used to characterise the convolution process, m, n are the values of pixels in the input image, and a, b are pixel values in the resulting convoluted image. A finite convolution kernel extracts the particular feature by taking the particular portion of an image with a specified neighbourhood. Convolution operations involve many such convolutional kernels, which are nothing but a finite impulse response. The size of this kernel is specified by:

(6) $(2z+1)\hspace{0.17em}(2z+1)$(6)

Where $z=1,2,3$ etc. So, now the values of m and n vary from -z to z. In the present case the kernel size was 3 by 3.

Sometimes padding is needed to get the size of the input and output image to be equal. Here, the ‘same’ padding was used to get the image as the input image size. The convolution kernel used in CNN is applied at every position of the input image, reducing parameters compared to conventional neural networks.

In flipping the kernel, each kernel coefficient is multiplied with corresponding pixel values of input images, and that gives the output response, and all such output responses for a particular kernel result in a convoluted image or feature map. When flipping the kernel on the input image, a particular region is to be flipped at a time, which is called the receptive field. The output of the convolution layer is passed through the ReLu layer, which makes the non-linear input feature map into a linearly separable feature by replacing all negative values in the feature map with zero. No specific parameters are required for this layer. Mathematically it can be taken as:

(7) $R(x)=max\{0,x\}$(7)

Here, x describes the input values. The output of this layer passed through the pooling layer. Window size and strides are essential parameters of the pooling layer. The present work applied max-pooling with a window of size 2 × 2 and stride was 2. The fully connected layer performs the classification task along with the classification layer. The classification layer has nodes equal to the number of classes or labels. The other important parameters for the training of CNN are optimiser, learning rate, number of epochs, validation frequency etc.

For this work, ‘sgdm’ optimiser was utilised to give a better result than other commonly used optimisers. The learning rate was 0.001, the validation frequency was 30, and the number of epochs was 100. From the total number of images for each dataset, 1575 images were used for training, and the remaining 525 images were used for testing purposes. The current designed model of CNN was tested for the MNIST digit dataset, which achieved a validation accuracy of 99.68 %. The training was done on a system having an ‘intel 4208’ CPU with a memory of 32 GB and NVIDIA-RTX2080, 11 GB single GPU in Matlab 2021b environment.

4. Results and discussion

The designed CNN model was trained with the same number of images for vibration and IRT data. Results are compared in terms of classification accuracies. The obtained confusion matrices from tested data in both cases were analysed with other important conclusions.

4.1 Vibration-Based fault diagnosis

Vibration monitoring is a well-established and most widely used method for fault diagnosis in rotating machines. The main advantage of using vibration-based analysis is that the signal analysis provides various characteristic frequencies for a particular fault condition. Traditional signal processing techniques can efficiently characterise a single fault condition. However, when the situation becomes complex, such as more than one fault with varying speed conditions, the method demands extra effort, and a simple signal processing method becomes less effective in such cases, and a series of signal processing may require. Wavelet analysis is such a method where the analysis can be done efficiently. Further, the artificial intelligence-based method allows automated and intelligent methods to identify faults with a pattern recognition-based methodology. In this case, the time domain data can be used as images using CWT containing time-frequency information, called scalograms, as explained in the previous section.

These obtained scalograms in colour form were first converted into grey images to prepare them for CNN input. The scalograms of single fault conditions at 19 Hz are shown in Figure 6. It can be observed that some amount of modulation was observed with the highest in OR and BD. As explained earlier, the 75% data set from each level was taken as training data and 25% data for testing.

Figure 6. Raw vibration signals and scalograms of single fault conditions in bearing at 19 Hz.

Table 3 shows the confusion matrices of different fault conditions at 19 Hz and 29 Hz for single fault conditions, which give the predicted and actual class values. The result for accuracy shows a perfect classification, i.e. 100 % accuracy at constant speed conditions, but slightly lower accuracy at ramp up, where there is a little confusion between the inner race and ball fault. However, at ramp down, the accuracy was 100 %. Figure 7 shows the raw vibration signature and respective scalograms at 19 Hz for dual fault conditions. A significant amount of modulation is observed in all dual fault conditions. Table 4 gives the results for dual fault conditions. The classification rate is also near 100% for constant speeds with little confusion in ‘IRBD’ and ‘IROR’ classes for dual fault conditions. However, this confusion is more in the case of varying speed conditions, i.e. ramp up and ramp down. The result shows a promising trend of classification accuracy as most of the fault conditions were perfectly classified.

Figure 7. Raw vibration signals and scalograms of dual fault conditions in bearing at 19 Hz.

Table 3. Confusion matrices obtained through vibration-based monitoring for single faults.

19 Hz							29 Hz
Predicted class							Predicted class
True class	HL	100	0	0	0	0	True class	HL	100	0	0	0	0
	IR	0	100	0	0	0		IR	0	100	0	0	0
	OR	0	0	100	0	0		OR	0	0	100	0	0
	BD	0	0	0	100	0		BD	0	0	0	100	0
	LB	0	0	0	0	100		LB	0	0	0	0	100
		HL	IR	OR	BD	LB			HL	IR	OR	BD	LB
Ramp up							Ramp down
Predicted class							Predicted class
True class	HL	100	0	0	0	0	True class	HL	100	0	0	0	0
	IR	0	99.8	0	0.2	0		IR	0	100	0	0	0
	OR	0	0	100	0	0		OR	0	0	100	0	0
	BD	0	0	0	100	0		BD	0	0	0	100	0
	LB	0	0	0	0	100		LB	0	0	0	0	100
		HL	IR	OR	BD	LB			HL	IR	OR	BD	LB

Table 4. Confusion matrices obtained through vibration-based fault diagnosis for dual faults.

19 Hz								29 Hz
Predicted class								Predicted class
True class	HL	100	0	0	0	0	0	True class	HL	100	0	0	0	0	0
	IRBD	0	99.8	0	0.2	0	0		IRBD	0	100	0	0	0	0
	IRLB	0	0	100	0	0	0		IRLB	0	0	100	0	0	0
	IROR	0	0	0	100	0	0		IROR	0	0.2	0	99.8	0	0
	ORBD	0	0	0	0	100	0		ORBD	0	0	0	0	100	0
	ORLB	0	0	0	0	0	100		ORLB	0	0	0	0	0	100
		HL	IRBD	IRLB	IROR	ORBD	ORLB			HL	IRBD	IRLB	IROR	ORBD	ORLB
Ramp up								Ramp down
Predicted class								Predicted class
True class	HL	100	0	0	0	0	0	True class	HL	100	0	0	0	0	0
	IRBD	0	98.5	0	1.5	0	0		IRBD	0	99.8	0	0.2	0	0
	IRLB	0	0	100	0	0	0		IRLB	0	0	100	0	0	0
	IROR	0	1	0	99	0	0		IROR	0	0.2	0	99.8	0	0
	ORBD	0	0	0	0	100	0		ORBD	0	0	0	0	100	0
	ORLB	0	0	0	0	0	100		ORLB	0	0	0	0	0	100
		HL	IRBD	IRLB	IROR	ORBD	ORLB			HL	IRBD	IRLB	IROR	ORBD	ORLB

Figure 8 shows the raw vibration signature and respective scalograms at 19 Hz for multiple fault conditions. A much higher modulation is observed in all multiple fault conditions, with most in ‘MF2’ and ‘MF5’. The classification results obtained through the proposed CNN are given in Table 5 for multiple faults. Here more confusion is observed among the classes than in the dual fault conditions. At constant speed, the confusion persists among ‘MF1’, ‘MF2’, and ‘MF5’. However, the confusion at varying speed conditions is among all the multiple fault conditions, with the maximum in the case of ‘MF2’ and ‘MF5’.

Figure 8. Raw vibration signals and scalograms of multiple fault conditions in bearing at 19 Hz.

Table 5. Confusion matrices obtained through vibration-based fault diagnosis for multiple faults.

19 Hz								29 Hz
Predicted class								Predicted class
True class	HL	100	0	0	0	0	0	True class	HL	100	0	0	0	0	0
	MF1	0	100	0	0	0	0		MF1	0	99.8	0	0	0.2	0
	MF2	0	0	99.9	0	0	0.1		MF2	0	0.2	99.7	0	0	0.1
	MF3	0	0	0	100	0	0		MF3	0	0	0	100	0	0
	MF4	0	0.1	0	0	99.9	0		MF4	0	0.1	0	0	99.9	0
	MF5	0	0.1	0.6	0	0	99.3		MF5	0	0	0.2	0	0	99.8
		HL	MF1	MF2	MF3	MF4	MF5			HL	MF1	MF2	MF3	MF4	MF5
Ramp Up								Ramp Down
Predicted class								Predicted class
True class	HL	100	0	0	0	0	0	True class	HL	100	0	0	0	0	0
	MF1	0	99.9	0	0	0.1	0		MF1	0	99.8	0	0	0	0.2
	MF2	0	0.1	98.2	0	0	1.7		MF2	0	0	99.5	0	0	0.5
	MF3	0	0	0	100	0	0		MF3	0	0	0	100	0	0
	MF4	0	0	0	0	100	0		MF4	0	0.2	0.1	0	99.6	0.1
	MF5	0	0	1.8	0	0	98.2		MF5	0	0.1	0.8	0	0	99.1
		HL	MF1	MF2	MF3	MF4	MF5			HL	MF1	MF2	MF3	MF4	MF5

4.2 Infrared thermography-based fault diagnosis

Although vibration monitoring is considered a prominent and well-established technique and is also proven here, but is associated with some inherent issues, the common issue involved with vibration-based monitoring are physical mounting and sensor damage for long-time contact. Also, finding out the characteristic frequencies to determine the actual fault type when the component gets hampered by multiple faults becomes challenging, as discussed in the previous section. Being a non-invasive method, IRT has an advantage in those situations for fault diagnosis. The temperature distribution pattern in various fault conditions makes it possible to detect them on a trained system accurately. However, at the laboratory level, it is recommended to run the setup for a specified time to achieve the steady-state conditions for any clear discrimination between the said fault conditions. However, this limitation is accessible in the actual industrial environment, where the machines work for longer hours.

The thermal images of all conditions at 19 Hz are shown in Figure 9 for single faults. An important observation from these images is a significant heat transfer from the bearing to the adjacent housing part, especially in case of lack of lubrication. Also, from Figure 9, a more uniform heat distribution is observed in case of lack of lubrication as against the other conditions that indicate a higher heat generation, and that condition can be easily detected with IRT-based monitoring. In addition, the main problem arises when other geometrical faults aid the lack of lubrication in the bearing. Such conditions make it very difficult for an accelerometer to capture the data accurately and smoothly. IRT, in such cases, finds its place by doing a perfect job for pattern recognition by generating the data in thermal images. Table 6 gives the classification results for the single faults using IRT. All the fault conditions were classified with 100% accuracy at all speed conditions. The thermal images of different dual fault conditions at 19 Hz are shown in Figure 10. A slight difference was observed in the heat distribution in RoI for the different dual fault conditions. However, with naked eyes, the task of discrimination is problematic and should be made automated for intelligent fault diagnosis.

Figure 9. Thermal images of different single fault conditions in bearing at 19 Hz.

Figure 10. Thermal images of different dual fault conditions in bearing at 19 Hz.

Table 6. Confusion matrices obtained through IRT-based fault diagnosis for single faults.

19 Hz							29 Hz
Predicted class							Predicted class
True class	HL	100	0	0	0	0	True class	HL	100	0	0	0	0
	IR	0	100	0	0	0		IR	0	100	0	0	0
	OR	0	0	100	0	0		OR	0	0	100	0	0
	BD	0	0	0	100	0		BD	0	0	0	100	0
	LB	0	0	0	0	100		LB	0	0	0	0	100
		HL	IR	OR	BD	LB			HL	IR	OR	BD	LB
Ramp up							Ramp down
Predicted class							Predicted class
True class	HL	100	0	0	0	0	True class	HL	100	0	0	0	0
	IR	0	100	0	0	0		IR	0	100	0	0	0
	OR	0	0	100	0	0		OR	0	0	100	0	0
	BD	0	0	0	100	0		BD	0	0	0	100	0
	LB	0	0	0	0	100		LB	0	0	0	0	100
		HL	IR	OR	BD	LB			HL	IR	OR	BD	LB

The classification results of different dual fault conditions are given in Table 7. It was observed that the classification accuracies for each class are 100%, with no confusion among the classes. The pattern of perfect classification persists for constant speed conditions as well as varying speed conditions, i.e. ramp up and ramp down.

Table 7. Confusion matrices obtained through IRT-based fault diagnosis for dual faults.

19 Hz								29 Hz
Predicted class								Predicted class
True class	HL	100	0	0	0	0	0	True class	HL	100	0	0	0	0	0
	IRBD	0	100	0	0	0	0		IRBD	0	100	0	0	0	0
	IRLB	0	0	100	0	0	0		IRLB	0	0	100	0	0	0
	IROR	0	0	0	100	0	0		IROR	0	0	0	100	0	0
	ORBD	0	0	0	0	100	0		ORBD	0	0	0	0	100	0
	ORLB	0	0	0	0	0	100		ORLB	0	0	0	0	0	100
		HL	IRBD	IRLB	IROR	ORBD	ORLB			HL	IRBD	IRLB	IROR	ORBD	ORLB
Ramp Up								Ramp Down
Predicted class								Predicted class
True class	HL	100	0	0	0	0	0	True class	HL	100	0	0	0	0	0
	IRBD	0	100	0	0	0	0		IRBD	0	100	0	0	0	0
	IRLB	0	0	100	0	0	0		IRLB	0	0	100	0	0	0
	IROR	0	0	0	100	0	0		IROR	0	0	0	100	0	0
	ORBD	0	0	0	0	100	0		ORBD	0	0	0	0	100	0
	ORLB	0	0	0	0	0	100		ORLB	0	0	0	0	0	100
		HL	IRBD	IRLB	IROR	ORBD	ORLB			HL	IRBD	IRLB	IROR	ORBD	ORLB

The thermal images of different multiple fault conditions at 19 Hz are shown in Figure 11. The most friction-contributing fault is the ball fault, and this fault is included in all multiple faults, excluding ‘MF1’. The heat distribution pattern seems more uniform in ‘MF3’, ‘MF4’, and ‘MF5’. The classification results for multiple fault conditions are given in Table 8. It can be observed from the results that the capability of IRT to discriminate the different fault conditions as the classification results are perfect for all the fault conditions at constant and varying speed conditions.

Figure 11. Thermal images of different multiple fault conditions in bearing at 19 Hz.

Table 8. Confusion matrices obtained through IRT-based fault diagnosis for multiple faults.

19 Hz								29 Hz
Predicted class								Predicted class
True class	HL	100	0	0	0	0	0	True class	HL	100	0	0	0	0	0
	MF1	0	100	0	0	0	0		MF1	0	100	0	0	0	0
	MF2	0	0	100	0	0	0		MF2	0	0	100	0	0	0
	MF3	0	0	0	100	0	0		MF3	0	0	0	100	0	0
	MF4	0	0	0	0	100	0		MF4	0	0	0	0	100	0
	MF5	0	0	0	0	0	100		MF5	0	0	0	0	0	100
		HL	MF1	MF2	MF3	MF4	MF5			HL	MF1	MF2	MF3	MF4	MF5
Ramp Up								Ramp Down
Predicted class								Predicted class
True class	HL	100	0	0	0	0	0	True class	HL	100	0	0	0	0	0
	MF1	0	100	0	0	0	0		MF1	0	100	0	0	0	0
	MF2	0	0	100	0	0	0		MF2	0	0	100	0	0	0
	MF3	0	0	0	100	0	0		MF3	0	0	0	100	0	0
	MF4	0	0	0	0	100	0		MF4	0	0	0	0	100	0
	MF5	0	0	0	0	0	100		MF5	0	0	0	0	0	100
		HL	MF1	MF2	MF3	MF4	MF5			HL	MF1	MF2	MF3	MF4	MF5

Figure 12. Validation accuracy of vibration and IRT-based fault diagnosis (a) single faults (b) dual faults (c) multiple faults.

The comparison of validation accuracies for vibration and IRT is shown in Figure 12. It can be observed that with the use of CWT-based time-frequency scalograms, the classification task becomes much more efficient. In the case of single fault conditions, the accuracy is 100 %, except for the ramp-up condition. The results show the classification accuracy is quite adequate in dual fault conditions with a maximum value of 99.97 % at 19 Hz and 29 Hz and a minimum at ramp-down conditions with 99.59 %. Similarly, the maximum accuracy for vibration monitoring for multiple fault conditions is 99.88 % at 29 Hz, and the minimum is 99.39 % at the ramp-down condition.

The reason for 100 % classification in all the conditions using thermography may be due to the advantage of thermal profile change in different conditions. Although the results give 100 % classification accuracy in all the conditions of single, dual, and multiple faults in the IRT-based classification, it also has an inherent issue against vibration monitoring. It is difficult to generalise the results for any rotating machines, as in the case of vibration monitoring, the characteristic frequency relates to the geometrical parameters of bearing. However, the advantage of thermal-based monitoring can be taken for specific types of rotating machines and generalised for similar kinds of machines that the industries usually contain. However, the thermographic analysis for rotating machines is developing, and researchers are working on empirical relations with the geometrical measures of rotating machine components.

The presented proposed methodology based on CNN was also compared with the recent research work as given in Table 9. As mentioned earlier, very little work is reported on multiple faults at individual component levels such as bearing. However, few researchers have studied system-level multiple faults in the rotating machines. Shao et al [51]. used IRT for the diagnosis of various dual fault conditions in a rotor-bearing system. The system was tested using CAE and an enhanced CNN at two-speed conditions. Similarly, in work done by Xin et al [52], various faults were studied using IRT data at continuously increasing speed. The classification task was achieved through a Gaussian deep belief network, and an accuracy of 99.24 % was achieved. A deep CNN and SVM were used for dual faults in bearing by Xue et al. at a constant speed of 1465 rpm and achieved an accuracy of 98.71 % [53].

Table 9. Comparison of the proposed work with previous research work on bearing fault diagnosis.

References	Input modality	Considered fault enumeration	Speed condition	Classifier	Maximum accuracy
[51]	Vibration and IRT	Dual faults	Constant speed	CAE and CNN	94.54 %
[52]	IRT	Dual and multiple faults	Accelerated speed	Gaussian DBN	99.24 %
[53]	Vibration	Dual faults	Constant speed	CNN	98.71 %
[54]	Vibration	Single and multiple faults	Constant speed	SRC	95.67 %
[55]	Acoustic emission	Dual and multiple faults	Constant speed	SVM	100 %
[56]	Fusion of vibration and IRT	Single faults	Constant speed	SVM	100 %
[57]	Vibration	Single and dual faults	Constant speed	CNN+SVM	98.50 %
Proposed work	Vibration and IRT	Single, dual, and multiple faults	Constant, accelerated and decelerated speed	CNN	100 %

Sparse Representation Classifier (SRC), in combination with wavelet kernel-based extreme learning machine, was used for the classification of three single and one multiple fault conditions in bearing with an accuracy of 95.67 % [54]. Acoustic emission was explored for the diagnosis of various dual and multiple fault conditions using SVM with validation at different constant speed conditions, and different data sets and 100 % accuracy were formulated in six data sets out of 10 datasets [55]. Various single fault conditions were also studied in our previous using SVM with data from individual modalities i.e. vibration and IRT, and with their fusion and an accuracy of 100 % was achieved [56]. However, with SVM, mandatory manual feature extraction is associated. Similarly, single and dual faults were studied using CNN assisted with Particle Smarm Optimised – Support Vector Machine (PSO-SVM) and feature level fusion by converting the 1-dimensional vibration data into images [57]. In the proposed work, single, dual and multiple fault conditions were studied using a CNN with vibration and IRT. The validation was presented at constant as well as varying speed conditions. The results for time-frequency-based diagnosis were quite adequate in dual as well as multiple fault conditions. Meanwhile, the proposed IRT-based fault diagnosis provides classification results as 100 % at a constant as well as varying speeds.

5. Conclusions

In a rotating machine, components such as bearing, and various faults can co-exist in combination. A commensurate modality sensor has limitations for the accurate and reliable diagnosis of bearing faults. In the proposed work, a vibration monitoring method and non-invasive infrared thermography were presented, and the results are validated for a different combination of faults at constant and varying speeds. A CNN network was utilised for the automated fault diagnosis using the extracted vibration and thermal images. The time-domain information lags the complete information about the frequency content at a particular time, which may hinder the correct diagnosis of the fault. So, a time-frequency scalogram using CWT is utilised as the input to the CNN. Similarly, a non-invasive IRT was employed, and thermal images were extracted with a proper RoI. Adequate results were obtained using CWT-based time-frequency scalograms with maximum accuracy in the case of dual faults as 99.97 % at a constant speed and 99.94 % at varying speeds, respectively. These values for multiple faults were 99.88 % and 99.67 %, respectively.

Further, 100 % classification accuracy was achieved in the proposed IRT-based fault diagnosis for constant as well as varying speed conditions. These results demonstrate the reliability and robustness of the proposed method for bearing fault diagnosis. Also, the results demonstrate that the IRT-based diagnosis was more effective in fault detection and coping with other issues inherited with vibration monitoring. However, the thermal-based analysis may be limited by the cost aspect, but the one-time investment could be helpful for quick and effective decision-making regarding the effective and reliable multiple fault diagnosis of bearing. Future work will involve the fusion of vibration and IRT through experimental simulation of the multiple fault conditions by incorporating more complex working conditions.

Acknowledgments

The authors are thankful to the ‘Ministry of Education, Government of India, for the support in conducting this work.

Disclosure statement

No potential conflict of interest was reported by the author(s).