Improving hyperspectral sub-pixel target detection in multiple target signatures using a revised replacement signal model

ABSTRACT

The rich spectral data found in the hyperspectral data cube make them useful in real-world applications, such as target detection. Target pixels detection among an unknown background such as ground objects from hyperspectral data cube is of great interest for remote sensing community. The commonly used hyperspectral target detection methods often overlook the problem of prior knowledge of the target and could reduce the efficiency of these methods. It has to be noted that the spatial resolution of the hyperspectral data cube is usually limited; therefore, the sub-pixel targets only occupy part of the pixel. The replacement signal model is an essential model for sub-pixel targets. In this study, we developed a revised replacement signal model based on an automatic target generation procedure for improving hyperspectral sub-pixel target detection using the HyMap data cube. The effects of various real targets on hyperspectral data cube are evaluated to obtain consistent results. In experiments with seven targets, the proposed method achieves the average area under the ROC curve of 99%. Comparison results illustrated that the proposed method has competitive target detection performance in comparison with other state-of-the-art methods.

KEYWORDS:

• Target detection
• sub-pixel
• replacement signal model
• hyperspectral data cube

Introduction

Airborne or spaceborne hyperspectral imaging sensors provide remote sensing image cubes from the reflected solar radiation from the Earth’s surface (Bingwen et al., 2013; Borengasser et al., 2007; Zhang & Du, 2012). A hyperspectral image cube contains very narrow bands covering from the visible spectrum through the infrared spectrum (Bakhshi et al., 2019). The rich spectral data found in hyperspectral images makes them useful in real-world applications, such as land-cover classification (Zomer et al., 2009), anomaly detection (Li & Du, 2015), change detection (Hasanlou & Seydi, 2018; Seydi et al., 2020) and target detection (Chang, 2016; Wei & Xuchu, 2006). Considering these applications, target detection is an important field of hyperspectral remote sensing (Anderson et al., 2019). Target detection in hyperspectral data is the identification of known object spectral signature within pixel spectra from a hyperspectral image (Basener et al., 2017). Due to the spectral mixture between objects in the hyperspectral image, target detection is an extremely difficult issue (Nasrabadi, 2014; Wang et al., 2020).

Pre-processing, target map generation, and evaluation are the three main stages for target detection. Among these stages, the way the target map is generated is the most crucial issue for any target detection studies (Sun et al., 2017). To solve this problem, different methods have been proposed in recent years. Traditional methods in target detection have been studied based on the linear unmixing model (Hsuan Ren et al., 2003), including maximum likelihood (Settle, 1996), subspace projection (Schowengerdt, 1997), and singular value decomposition (Boardman, 1989). The linear unmixing approaches require the deep knowledge of the endmember signatures. Generally, an endmember is defined as the pure signature of an object. In practical experiences obtaining such knowledge of the endmember signatures may not be possible (Demirci et al., 2014). Moreover, there are four main categories of target detection approaches, including spectral angle mapper (also known as SAM) (Richards & Jia, 2006), matched filter (also known as MF) (Manolakis & Shaw, 2002), adaptive cosine estimator (also known as ACE) (Kraut et al., 2005), and constrained energy minimization (also known as CEM) (Settle, 1996). SAM is the simplest target detection method, which is based on spectral similarity and measures the angle between known object spectra and the observed pixel spectra. SAM is limited by the assumption of a zero-mean which is an unrealistic assumption for target detection (Jha & Nidamanuri, 2020; Kruse et al., 1993). Unlike the previous approach, MF is obtained by correlating a known object spectrum, with an unknown object spectra (Jin et al., 2009). Unlike the MF which assumes ambient noise to be additive, ACE is generated from the generalized likelihood ratio test, which incorporated a substitution model for ambient (Kraut et al., 2005). CEM method suppresses the interference of undesired objects in the received signal, thereby accentuating the target. The CEM method requires only the knowledge of specific targets of interest. Due to the CEM method is very sensitive to noise, it is not a robust classifier for the same type of targets (Johnson, 2003). To remedy this drawback, three target detection methods have been proposed based on the CEM method, including sum CEM, winner-take-all CEM, and multiple-target CEM (Hsuan Ren et al., 2003). Considering these developed methods, winner-take-all CEM usually gives the best performance (Chang, 2013).

The investigation of previous studies proves that there are still key challenges for hyperspectral target detection. The main challenges are as follows:

(1) Sub-pixel detection. The spectral mixture between objects without prior knowledge which has not yet been considered well in the relevant studies. It is worth mentioning that the spatial resolution of hyperspectral images is usually limited, so the sub-pixel targets only occupy the pixel partially (B. Du et al., 2010). In this regard, the one-step generalized likelihood ratio test has recently been extended for sub-pixel target detection based on the replacement signal model (Besson & Vincent, 2020). The sub-pixel target detection is important in many applications, such as battleground reconnaissance (Kumar & Ghosh, 2017), monitoring of mineral material (Myers et al., 2019), and so on.

(2) Complex background. Complex background modeling for target detection is an important problem for hyperspectral data processing in complex environments (Eismann, 2006).

(3) Multiple signatures. Poor performance due to the lack of a robust mathematical model in pure and mixed pixels that are composed of multiple signatures (e.g. pickup truck) (Vincent & Besson, 2020).

The main purpose of this paper is to establish a robust approach to sub-pixel target detection based on revising the replacement signal model using a hyperspectral image in a complex background. It has to be noted that the replacement signal model is an important model for sub-pixel targets in hyperspectral data. The contributions of this study are as follows.

(1) The proposed a revised replacement signal model based on the automatic target generation procedure method for unsupervised target endmember extraction.

(2) A comprehensive assessment is presented based on state-of-the-art target detection methods. Consequently, the proposed revised replacement signal model surpasses the state-of-the-art methods for different target detection tasks.

(3) The proposed a simple and effective pre-processing step based on the quick atmospheric correction method for target detection enhancement.

(4) To obtain consistent results, the effects of various real targets on hyperspectral data cube are evaluated.

The manuscript is organized as follows. In Section 2, we introduce the dataset for the experimental study. In Section 3, we introduce the structure of the proposed method. Furthermore, Section 4 presents some experimental results and discussions to evaluate the performance of the proposed method. Finally, Section 5 provides the conclusion of this study.

Hyperspectral image dataset

In this study, the hyperspectral image with coverage areas of approximately 2.0 km² in Cooke City town, MT, USA is used as the experimental data. It was provided via the HyMap airborne hyperspectral imaging sensor by Snyder et al. (2008). HyMap sensor flown at 1.4 km above the ground level. The Cooke City dataset is illustrated in Figure 1(a), where there are 280 × 800 pixels. Each pixel is observed at 126 bands covering the electromagnetic spectra from 0.45 to 2.48 μm and the ground sample distance is 3.0 m. In our experiments, the three vehicles (Figure 1(b-d)) and four fabric panels (Figure 1(e-h)) in different sizes and colors were selected as targets. The reference spectra of the vehicles and the fabric panels were measured by an Analytical Spectral Devices, Inc. FieldSpec Pro, and a Cary 500 spectrophotometer, respectively (Snyder et al., 2008). The vehicles were occupied at most a few pixels, but fabric panels F1 and F2 were nearly a full pixel while fabric panels F3 and F4 were occupied less than a pixel. The Cooke City dataset is still a challenging dataset for hyperspectral target detection (Yang et al., 2020). All related datasets can be found in https://rslab.ut.ac.ir.

Figure 1. Cooke City dataset with target locations identified by the color squares (left). Plots showing the reference spectra of targets (right). The greyscale image (a), and the targets (b-h)

Proposed method

In this study, we developed a revised replacement signal model based on the automatic target generation procedure for hyperspectral sub-pixel target detection using the Cooke City dataset. The proposed method is composed of two effective concepts of the automatic target generation procedure for endmember extraction and the replacement signal model that extracts the sub-pixel target. The proposed method for sub-pixel target detection is presented in Figure 2. As shown in Figure 2, the proposed method for sub-pixel target detection is made up of three stages: (a) pre-processing, (b) target detection, and (c) evaluation.

Figure 2. Overview of the proposed method

Pre-processing

At first, the water absorption and low signal-to-noise bands were identified and removed from the Cooke City dataset for further processing. As shown in Figure 3, 10 bands, namely bands 1, 2, 3, 63, 64, 65, 66, 95, 96, and 97, contained the water absorption and low signal-to-noise bands and therefore were discarded (Heiden et al., 2001). After removing the water absorption and low signal-to-noise bands, 116 bands were retained.

Figure 3. Examples of the greyscale images for bad band 63 (a), and 64 (b)

Atmospheric correction is an important pre-processing step to remove the atmospheric and solar radiation effect on the recorded data and to derive the spectral reflectance signature of the ground level. The spectral unmixing performance can also be affected by the problem of atmospheric correction. Therefore, atmospheric correction is an essential pre-processing step in many spectral unmixing tasks.

The Quick Atmospheric Correction (QUAC) method is a semi-empirical atmospheric correction method for hyperspectral data cube in the visible spectrum through the shortwave infrared spectrum (Carr et al., 2015). Unlike other first-principles radiation transport algorithms, e.g. Fast Line-of-sight Atmospheric Analysis of Spectral Hypercubes (also known as FLAASH), the QUAC method determines atmospheric compensation parameters directly from the observed pixel spectra, without using ancillary data. QUAC supports hyperspectral sensors, such as HyMap airborne hyperspectral image. The key QUAC assumption, which empirically holds for most scenes, is the consistency of the average of the endmember reflectance spectra, which are not highly structured, is always the same (Bernstein et al., 2012).

Target detection

Endmember estimation

To estimate the number of distinct endmembers in the hyperspectral data cube, the virtual dimensionality provides an estimate of the initial target endmembers. To determine the virtual dimensionality, the “Harsanyi-Farrand-Chang” (also known as HFC) approach based on the “Neyman-Pearson” detection theory is an effective method (Chang & Du, 2004; Harsanyi et al., 1993). According to estimates (Cao et al., 2018), the selection of the initial target endmembers based on the HFC approach has a great impact on spectral unmixing. As a binary hypothesis problem, the virtual dimensionality determined by the HFC approach is q (number of endmembers). The HFC approach has an initial parameter that specified the false alarm probability. The false alarm probability is defined as follows (Gerg, 2008):

(1) $P_f=\underset{\tau }{\overset{\mathrm{\infty }}{\int }}{p}_0\left(z\right)dz$(1)

where $p_0\left(z\right)$ is a Gaussian distribution; $\tau$ is determined by selecting suitable values for $P_f$. In practice, $P_f$ is set to 10⁻³. Another widely used endmember estimation method is the NWHFC approach (Chang & Du, 2004). NWHFC is a developed structure of the HFC approach, included a noise-whitening processing step (Jafarzadeh & Hasanlou, 2019). Due to the noise-whitening processing step in the NWHFC approach, the NWHFC does not have stable performance in the target detection task. In this regard, (Chang, 2017) showed that the HFC method compatible with the automatic target generation procedure for unsupervised endmember extraction.

Endmember extraction

The Automatic Target Generation Procedure (ATGP) is an unsupervised endmember extraction method in the target detection task, where can provide more accurate initial points to identify endmembers (Ren & Chang, 2003). Compared to conventional supervised and unsupervised endmember extraction approaches, such as the Pixel Purity Index (PPI) that require a very large number of skewers to find maximal/minimal orthogonal projections (Chang & Li, 2016), ATGP method requires less prior information to extract pure pixel vectors as the endmember spectrum from the hyperspectral data cube (Cao et al., 2018). ATGP method works by iterative orthogonal projections of the hyperspectral data cube then finding the largest magnitude vector in a sequence of orthogonal projection subspaces (Q. Du et al., 2008; Gerg, 2008). ATGP method was implemented in the four following steps (Chang et al., 2015):

(a) Selecting the initial target pixel vector and an error threshold: $t_0=\mathrm{a}\mathrm{r}\mathrm{g}\left\{ma{x}_x\left[x^{T}x\right]\right\}$

(b) Finding p ^th target: $t_p=\mathrm{a}\mathrm{r}\mathrm{g}\left\{ma{x}_x\left[{\left(P_{U_{p-1}}^{\mathrm{\perp }}x\right)}^T\left(P_{U_{p-1}}^{\mathrm{\perp }}x\right)\right]\right\}$;$U_{p-1}=\left[t_0,t_1,t_2,\dots ,t_{p-1}\right]$

(c) Stopping rule: If $t_0^T{P}_{U_{p-1}}^{\mathrm{\perp }}{t}_0>\epsilon ,$ let $U_p=\left[U_{p-1}{t}_p\right]=\left[t_1,t_2,\dots ,t_p\right]$

(d) When p equals to the number of the endmembers q, the ATGP is terminated.

where x is the observed pixel,$\epsilon$ is the error threshold and q is the virtual dimensionality determined by the HFC approach.

Figures 4 and 5 demonstrate the prediction and the reference spectral signature for targets by the proposed strategy. Table 1 shows a summary of the evaluation results based on the coefficient of determination (R²) and Root Mean Square Error (RMSE). The results demonstrate that the proposed strategy provides good performance on the spectral signature prediction.

Table 1. The evaluation results for the predicted spectral signature based on the proposed strategy in the Cook City dataset. Good values for R² and RMSE are 100%, and 0, respectively

	V1	V2 (cabin)	V2 (back)	V3	F1	F2	F3	F4
R^2 [%]	74.0	85.0	60.0	71.7	91.9	67.5	74.0	76.9
RMSE	0.05	0.08	0.02	0.08	0.04	0.02	0.1	0.6

Figure 4. Plots showing the reflectance spectra of targets. Blue spectra denote the reference signature, and red spectra denote the prediction signature of V1 (a), V2-cabin (b), V2-back (c), and V3 (d)

Figure 5. Plots showing the reflectance spectra of targets. Blue spectra denote the reference signature, and red spectra denote the prediction signature of F1 (a), F2 (b), F3 (c), and F4 (d)

Another widely used endmember extraction method is the vertex component analysis approach (Nascimento & Dias, 2005). The vertex component analysis approach is a developed structure of the ATGP method, included a sequence of successive orthogonal projection subspaces based on the maximal orthogonal projections (Tang et al., 2012). According to estimates (Cao et al., 2018), the vertex component analysis approach does not have stable performance when random initial values are used.

Sub-pixel abundance estimation

The replacement signal model is a sub-pixel target detection method based on the spectrum of the target/background and the unknown target abundance. For target detection, the replacement signal model, defined as (Manolakis et al., 2002):

(2) $x=\alpha s+\left(1-\alpha \right)v$(2)

where x is the spectral vector of the pixel under test (known as the observed pixel spectrum); $0\le \alpha \le 1$ is the fraction of the pixel area filled by the target; $s$ is the spectrum of the target, and $v$ is the spectrum of the background. The fraction of the pixel area filled by the target is the unknown target abundance (Vincent & Besson, 2019). In this regard, the target spectrum s is assumed to be known from the ATGP method and the generalized likelihood ratio test (known as GLRT) (Kelly, 1986) used for the sub-pixel abundance estimation. The revised replacement signal model based on GLRT and the spectrum of the target from the ATGP method, defined as:

(3) $R-GLRT\left(x\right)=\text{break}\frac{\left({\left[{\left(s-m\right)}^T{G}^T\left(x-m\right)\right]}^2\right)}{\left([{\left(s-m\right)}^T{G}^T\left(x-m\right)\right].\left[1+\left(\frac{1}{M}\right).{\left(x-m\right)}^T{G}^{-1}\left(x-m\right)\right])}$(3)

where x is the observed pixel spectrum; s is the spectrum of the target from ATGP method; m is the mean; G is the background covariance matrix, and M is the total number of pixels.

According to previous studies, the spectrum of the target is known from laboratory or field spectra measurements (Vincent & Besson, 2020). In practical experiences, obtaining this knowledge of the target endmember’s spectrum may not be possible. In this study, we focus on tackling this problem in a hyperspectral data cube by proposing a revised replacement model based on the ATGP method for unsupervised target endmember extraction. The original replacement signal model cannot perform well due to the sum-to-one constraint in linear endmember mixing. In this regard, the revised replacement signal model is to improve the detection tasks by softening this unitary constraint based on ATGP and GLRT. As a result, the sub-pixel abundance vectors are subjected to the soft sum-to-one constraint. More precisely, our unique advantages of the proposed method over the original replacement signal model are twofold: (1) estimation of target spectrum (s) based on unmixing, and (2) soft sum-to-one constraint.

Evaluation

The performance of the revised replacement model based on the ATGP method was compared with those of three representative methods based on the literature, including winner-take-all CEM, ACE, and MF. In this study, due to the number of targets is too small to get statistical assessment; the performance evaluation is conducted based on the area under the Receiver Operating Characteristics (ROCs) and target probability map. This means that a smaller false alarms score is better. To the best of the authors’ knowledge, the false alarms score is an effective metric for hyperspectral target detection in the literature (Besson & Vincent, 2020; Vincent & Besson, 2020; 2019).

Results and discussions

In this section, we have quantitatively and qualitatively evaluated the performance of the revised replacement model based on the ATGP method for hyperspectral target detection. The proposed method was executed in MATLAB R2017b on a computer with Intel(R) Core(TM) i5-3320 M CPU (2.60 GHz) and 4 GB RAM. Unmixing computation time of the proposed method on the Cooke City dataset is less than 120 minutes. Moreover, detection computation time is less than 2 seconds. Figures 6-12 show the test scene and the predicted probability maps for seven different targets using the proposed approach compared with the state-of-the-art methods. Moreover, Tables 2 and 3 show a summary of the area under the ROC curve for different targets and methods. According to Figures 6-8, the comparison results from the first, second, and third targets (i.e. V1, V2, and V3) show that the proposed approach can achieve the highest probability for pure pixel and mixed pixel among the state-of-the-art methods. The results of the special pixels under test (e.g. V2) demonstrate that the proposed method is robust to pure and mixed pixels that are composed of two different signatures. The proposed approach achieves the area under the ROC curve of 99.2%, 99.1%, and 99.1% for the first, second, and third targets, respectively, while the area under the ROC curve of ACE is 99.1%, 99.0%, and 99.0%. ACE is generated from the generalized likelihood ratio test and the prior information of the target, while the proposed approach is an unsupervised method. Based on Table 2, the average area under the ROC curve for vehicles is about 50.3%, 99.0%, 50.6%, and 99.1% for the winner-take-all CEM, ACE, MF, and the proposed method, respectively.

Table 2. Area under the ROC curve for vehicles. The bold values denote the best result

	Winner-take-all CEM (%)	ACE (%)	MF (%)	The proposed method (%)
V1	50.1	99.1	50.5	99.2
V2	50.1	99.0	50.8	99.1
V3	50.8	99.0	50.7	99.1
Average	50.3	99.0	50.6	99.1

Table 3. Area under the ROC curve for fabric panels. The bold values denote the best result

	Winner-take-all CEM (%)	ACE (%)	MF (%)	The proposed method (%)
F1	50.0	99.0	50.3	99.1
F2	50.5	98.9	50.9	98.9
F3	50.0	98.9	50.4	98.9
F4	50.6	99.0	50.7	99.0
Average	50.2	98.9	50.6	98.9

Figure 6. Results of the first target (V1). The Probability prediction values for target pixels are defined in a range from 0% to 100%

Figure 7. Results of the second target (V2). The Probability prediction values for target pixels are defined in a range from 0% to 100%

Figure 8. Results of the third target (V3). The Probability prediction values for target pixels are defined in a range from 0% to 100%

Figure 9. Results of the fourth target (F1). The Probability prediction values for target pixels are defined in a range from 0% to 100%

Figure 10. Results of the fifth target (F2). The Probability prediction values for target pixels are defined in a range from 0% to 100%

Figure 11. Results of the sixth target (F3). The Probability prediction values for target pixels are defined in a range from 0% to 100%

Figure 12. Results of the seventh target (F4). The Probability prediction values for target pixels are defined in a range from 0% to 100%

According to Figures 9–12 and Table 3, the proposed approach (Figures 9–Figures 12(j)) and ACE (Figures 9–Figures 12(h)) obtained better detection results than other methods in the fabric panels. The proposed approach and ACE achieve the average area under the ROC curve of 98.9% for the different targets, while the average area under the ROC curve of MF and CEM is 50.6%, and 50.2, respectively. MF (Figures 9–Figures 12(h)) generally performed better than CEM but still did not detect the fabric panels in the Cook City dataset. The comparison results from the fourth, fifth, sixth, and seventh targets (i.e. F1, F2, F3, and F4) show that the proposed approach can achieve the highest probability for pure pixel and mixed pixel among the state-of-the-art methods. Moreover, the results of the special pixels under test (e.g. F3 and F4) demonstrate that the proposed method is robust to pure and mixed pixels that are composed of multiple targets.

To compare the performance of the proposed approach with the state-of-the-art methods the test targets from the Cook City dataset are also evaluated according to Table 4 based on probability prediction values. The comparison results show that the proposed method can achieve the highest average probability prediction values for pure and mixed pixels among the state-of-the-art methods for different targets.

Table 4. A comparison between probability prediction values in the proposed approach and the state-of-the-art methods for different targets. The bold values denote the best result

	Winner-take-all CEM (%)		ACE (%)		MF (%)		The proposed method (%)
	Pure	Mixed	Pure	Mixed	Pure	Mixed	Pure	Mixed
V1	3.0	2.3	16	8.1	3.3	2.6	30	9.3
V2	0.0	0.0	0.0	1.5	0.0	0.0	5.5	4.0
V3	0.0	0.0	3.0	4.0	0.0	0.5	6.0	4.3
F1	0.0	0.0	42.9	40.9	3.9	3.1	44.0	40.9
F2	5.4	1.3	35.7	6.9	5.4	1.5	35.6	6.9
F3	0.0	0.0	1.2	0.3	0.1	0.1	1.2	0.3
F4	0.0	0.0	29.2	14.6	0.0	0.0	29.2	14.6
Average	1.2	0.5	18.3	10.9	1.8	1.1	21.6	11.5

Conclusions

In this study, a revised replacement model based on the ATGP method for unsupervised target endmember extraction was presented for unsupervised sub-pixel target detection from a hyperspectral data cube. The proposed method is conducted to improve the performance of hyperspectral unmixing in target detection. Moreover, the proposed method does not require prior knowledge and can be applied in complex scenes. Comparison results illustrated that the proposed method has competitive target detection performance in comparison with other state-of-the-art methods. Based on the analysis of the experimental results, the advantages of the proposed method are briefly summarized as follows: (1) The proposed method without prior knowledge can achieve better target detection results than winner-take-all CEM, ACE, and MF. (2) Unlike state-of-the-art methods, the proposed method is robust to pure and mixed pixels that are composed of two different signatures (e.g. V2). (3) The proposed method is robust to pure and mixed pixels that are composed of multiple targets (e.g. F3, and F4). Due to the unmixing stage in the proposed method, the proposed method has an inevitably high time consumption, which is its main limitation. Future work involves acceleration strategies that will be investigated towards the time consumption optimization for the unmixing stage.

Nomenclature

Table

Pf	False alarm probability
P0(z)	Gaussian distribution with mean zero and variance ${\sigma }_{z_l}^2$
G	Background covariance matrix
M	Total number of pixels
q	Virtual dimensionality determined by the HFC approach
x	Observed pixel spectrum
s	Spectrum of the target
v	Spectrum of the background
ε	An error threshold
ACE	Adaptive cosine estimator
ATGP	Automatic target generation process
CEM	Constrained energy minimization
GLRT	Generalized likelihood ratio test
HFC	Harsanyi-Farrand-Chang
MF	Matched filter
R^2	Coefficient of determination
RMSE	Root mean square error
SAM	Spectral angle mapper

Acknowledgments

The authors thank David K. Snyder from the Rochester Institute of Technology for providing the Cook City dataset.

Disclosure statement

No potential conflict of interest was reported by the authors.