A Privacy-preserving Image Retrieval Method Based on Improved BoVW Model in Cloud Environment

ABSTRACT

With the rapid development of cloud computing technology, more and more users choose to outsource image data to clouds. To protect users’ privacy and guarantee data’s confidentiality, images need to be encrypted before being outsourced to CSP, but this brings new difficulties to some basic yet important data services, such as content-based image retrieval (CBIR). In this paper, a privacy-preserving image retrieval method based on an improved BoVW model is proposed. An improved BoVW method based on Hamming embedding can provide binary signatures that refine the matching based on visual words; therefore, retrieval precision is improved significantly; orthogonal transformation is utilized to implement privacy-preserving image retrieval, where image features are divided into two different fields with orthogonal decomposition, for which encryption and distance comparison are executed separately, and two kinds of operation results are fused in the final vector with orthogonal composition. As a result, cloud server can extract components from encrypted features directly and compare distance with those of query image, without violating the privacy of images and features. Any algorithm can be used to encrypt features, which enhance the practicability of the proposed method. The security analysis and experimental results prove its security and retrieval performance.

KEYWORDS:

• Image retrieval
• Secure retrieval
• BoVW
• Cloud environment
• Orthogonal transformation

1. INTRODUCTION

Along with the development of cloud computing technology, public cloud service can provide unlimited storage space and computing ability for massive multimedia. At the same time, it brings serious security problems because data owners lose physical control of data while CSP (cloud service provider) is considered “honest but curious”, which means that it may lead to unauthorized access to data and cause leakage of personal privacy [1–3]. Given such circumstance, data must be encrypted before being outsourced to the CSP to protect its confidentiality. Although data security can be guaranteed by encryption, it also brings many new challenges to data management and data sharing service, such as image retrieval. Content-based image retrieval (CBIR) is a very promising method in image retrieval field which is characterized by extracting image features and comparing the distance between features automatically, and it has grown rapidly and made progresses in both the derivation of new features and the construction of signatures based on these features [4–6]. However, conventional CBIR methods cannot be applied under cloud environment directly because encrypted data fail to preserve the distance between feature vectors. If users want to retrieve images from CSP, encrypted image need to be decrypted first, then retrieval can be operated on plaintext, which makes the sensitive information being exposed to attackers, breaks privacy, and hence is not desired. Therefore, it is important to develop technologies of CBIR over encrypted domain, which is also called private/privacy-preserving CBIR (PCBIR) [7–9].

Many advances have been achieved in image retrieval over cryptographic domain. Early technology mainly originated from retrieval on text document for example, Song et al. [10] proposed a ciphertext scanning method based on streaming cipher to make sure whether the search term exists in the ciphertext; Boneh et al. [11] proposed a keyword search method based on public-key encryption; Swaminathan et al. [12] ranked the order the documents securely and extracted the most relevant documents from an encrypted collection; Cao et al. [13] proposed privacy-preserving multi-keyword ranked search over encrypted data. Although these methods can be extended to image retrieval based on user-assigned tags, extension to CBIR is not straightforward. CBIR typically relies on comparing the distance of image features, but comparing similarity among high-dimensional vectors using cryptographic primitives is challenging.

Recently, several methods have been proposed to solve the problem of PCBIR. Lu et al. [14] proposed three distance-preserving methods: bit plane randomization, random projection, and random unary encoding, which are applied on low-level features such as color histogram. Karthik et al. [15] proposed a transparent privacy-preserving hash method, which keeps the statistical rules of encrypted AC coefficients but ignores the spatial information distribution of the image. Xu et al. [16] proposed a secure retrieval method for JPEG images, which preserves the distribution of the AC coefficients and the statistical rule of color after the decoding of encrypted images. Ferreira et al. proposed a method in [17], where color information is encrypted by deterministic encryption techniques to support color-feature-based CBIR, and texture information is encrypted by probabilistic encryption algorithms for better security. Zhang et al. [18] use Pallier encryption algorithm to protect some lower level features such as color, texture, and shape, and achieve secure retrieval effects. Some local-feature-based PCBIR methods are also proposed. Hsu et al. [19] put forward a homomorphic encryption-based secure SIFT method which has good security at the cost of serious ciphertext extensions. Xia et al. [20] proposed to use SIFT features and transform earth mover’s distance (EMD) in a way that the cloud server can evaluate the similarity between images without learning sensitive information. Huang et al. [21] convert the high-dimensional VLAD descriptors to compact binary codes, and then adapt the asymmetric scalar-product-preserving encryption to design PCBIR to achieve the privacy requirements in the cloud environment. In contrast with global feature approaches, local-feature-based PCBIR methods achieve higher retrieval accuracy, but it requires quite complex methods to implement distance preserving, which is not suitable for large-scale image retrieval under cloud environment.

It should be noted that the above research results are all relying on specific encryption methods, such as shuffling and homomorphic encryption, which limit its universality. For example, homomorphic encryptions are too computation- and communication-intensive to be used in low-profile devices and large-scale systems, while shuffling is not suitable for some situations that have high requirements to security. Besides, these methods are all relying on global features or local features to evaluate the similarity of images, whose retrieval accuracy can hardly meet requirements of practical applications because of “semantic gap” between the visual features and the richness of human semantics [22]. Aiming to solve these problems, we propose a privacy-preserving image retrieval method under cloud environment in this paper, where an improved BoVW model is employed to improve retrieval precision, orthogonal transformation is combined together to implement privacy-preserving retrieval under cloud environment. An improved BoVW method based on Hamming embedding can provide binary signatures that refine the matching based on visual words; therefore, retrieval precision is improved significantly; orthogonal transformation are used to divide features into two different domains, for which encryption and distance comparison are executed separately. As a result, cloud server can extract components from encrypted features directly and compare distance with those of query image, without violating the privacy of images and features. Because the encryption operation and distance comparison operation are independent, any algorithm can be used to encrypt features, which enhance the practicability of the proposed method. The experimental results prove its effectiveness and security.

The organization of this paper is as follows: Chapter 2 introduces system architecture and preliminaries; Chapter 3 proposes our scheme. Chapter 4 provides experimental results and performance analysis, and Chapter 5 presents conclusions.

2. SYSTEM ARCHITECTURE AND PRELIMINARIES

2.1 System Architecture

The system model used in this paper is given in Figure 1.

Figure 1: System model of the proposed method

There are three entities involved in this model: content owner, CSP, and user. Content owner trains visual dictionaries, extracts features from images, and constructs search index, which are then encrypted together with images and outsourced to the CSP. CSP stores cipher images and secure index and performs image retrieval when receiving user request. Users send retrieval request to content owner and CSP, decrypt cipher-images returned by CSP, and get requested images.

2.2 Preliminaries

2.2.1 BoVW Algorithm

Bag of visual words (BoVW) model is extended from natural language processing and information retrieval field to computer vision in [23]. Descriptors are quantized into visual words with the k-means algorithm, and then image can be represented by the frequency histogram of visual words obtained by assigning each descriptor of the image to the closest visual word. Fast access to the frequency vectors is achieved by an inverted file system. BoVW has been successfully adopted to enable fast indexing and retrieval of large image collections [24]. The retrieval process consists of four steps: image content is described by means of a set of visual descriptors such as SIFT and SURF.; descriptors are clustered into visual words which form a vocabulary; descriptors are compared and assigned to one or more visual words so as to map the image into a histogram of visual word frequencies; images with the closest histogram distance will be returned as retrieval results.

Although BoVW has shown good performance in image retrieval task, it still suffers from some problems, such as insufficient discriminative power of visual words, quantization error caused by assigning descriptors to visual words, and low efficiency caused by comparing distance between high dimensions of vectors. Many methods are proposed to improve the performance of BoVW, such as [25,26]. Among these methods, [27] is the one that achieves higher accuracy for large-scale image search based on Hamming embedding (HE). The main idea of HE is to select low number of centroids k in k-means clustering to form a rough visual dictionary and refine the quantized index $q$ (${\text{x}}_i$) (the index of the centroid closest to the descriptor ${\text{x}}_i$) with those of a d_b-dimensional binary signature, thus the Euclidean distance between two descriptors can be mapped into the Hamming distance, and mismatch points can be removed by setting proper threshold; thus, retrieval performance is improved. This method includes four steps:

1. Assign descriptors x to their closest centroid, resulting in $q\left(x\right)$; project x by means of a $d_b\times d$ orthogonal projection matrix P, producing a vector $z=Px=[z_1,\dots ,z_{d_b}{]}^{\mathrm{T}}$; compute the median value ${\tau }_i$ of all vectors that belong to the same centroid;

2. Compute the signature $b\left(x\right)=\left(b_1\left(x\right),\dots ,b_{d_b}\left(x\right)\right)$ as follows: (1) $b_i\left(x\right)=\left\{\begin{array}{l}1\text{ if }{z}_i>{\tau }_{q\left(x_i\right)}\\ 0\text{ otherwize }\end{array}\right.$(1)

3. Use HE matching function to match the distance of $x,y$: (2) $f_{HE}\left(x,y\right)=\left\{\begin{array}{l}\begin{array}{c}(tf-idf{\left(q\left(x\right)\right)}^{2}ifq(x)=q(y)\\ \text{ and }h\left(b\left(x\right),b\left(y\right)\right)\le h_t\end{array}\\ 0\text{ otherwise }\end{array}\right.$(2) where $tf-idf\left(\right)$ weights the visual words according to their frequency, and $h\left(\right)$is the Hamming distance defined as follows: (3) $h\left(b\left(x\right),b\left(y\right)\right)=\sum _{i=1}^{d_b}{b}_i\left(x\right)\oplus b_i\left(y\right)$(3) ${\text{h}}_t$is a fixed Hamming threshold such that $0\le h_t\le d_b$.

4. Given a query image represented by its local descriptor $y_{i^{\mathrm{\prime }}}$, and a set of database image $j=1,\dots ,n$ represented by their local descriptor $x_{i,j}$, then a voting system is used to evaluate the similarity between the query image and each database image. The image score $s_j^{\ast }$ is given as follows: (4) $s_j^{\ast }=g_j\left(\sum _{i^{\mathrm{\prime }}=1}^{m\mathrm{\prime }}\sum _{i=1}^{m_j}f\left(x_{i,j},y_{i^{\mathrm{\prime }}}\right)\right)$(4) where g_j() is a post-processing function, m_j is the number of local descriptors of the j-th database image, m′ is the number of local descriptors of the query image. The score reflects the number of matches between the query and each database image.

2.2.2 Distance-Preserving Method Based on Orthogonal Transformation

Orthogonal transformation is a kind of vector representation method that consists of two processes: orthogonal decomposition and orthogonal composition. Any vector can be expressed as a sum of a set of component coefficients by orthogonal decomposition, and component coefficients are fused into one composite vector by orthogonal composition.

Suppose vector $X=\left(x_1,x_2,..,x_n\right)$, orthogonal matrix $B=\left(b_1,b_2,\dots ,b_n\right)$ , then $X$ can be expressed as follows: (5) $X=B\cdot Y$(5) Vector $Y$ can be expressed as follows: (6) $Y=B^{-1}\cdot X$(6) If we divide $B$ into two sub-matrix $R=\left(b_1,b_2,\dots ,b_m\right)$ and $S=\left(b_{m+1},b_{m+2},\dots ,b_n\right)$, then $Y$ is divided into $Y=(Y_1,Y_2{)}^{\mathrm{T}}$ correspondingly, and we can obtain the following: (7) $X=R\cdot Y_1+S\cdot Y_2$(7) If encryption is operated on $Y_1$ and feature is extracted from $Y_2$, respectively, then we can obtain the following: (8) $\begin{array}{rl}{X}_{ef}& =R\cdot Enc\left(Y_1,K_e\right)+S\cdot FeaExt\left(Y_2,K_f\right)\\ & =R\cdot Y_{1e}+S\cdot Y_{2f}=B\cdot Y_{ef}\end{array}$(8) Two different operations will be independent and will not interfere with each other because of the independence of the orthogonal decomposition, while two kinds of operation results are fused in final vector $X_{ef}$ because of the fusion of the orthogonal composition. We can also obtain the following: (9) $D\left(X_{ef},K_d,R\right)=R\cdot Dec\left(Y_{1e},K_d\right)+Y_{2f}$(9) (10) $FeaExt\left(X_{ef},K_f,S\right)=S\cdot FeaExt\left(Y_{2f},K_f\right)$(10) From these equations, we can see that features can be extracted from encrypted vectors directly, and the distance between the component vectors is similar to the one between the original vectors because of the distance-preserving characteristic of the orthogonal transformation.

3. THE PROPOSED METHOD

Based on algorithms mentioned in Section 2, we proposed a privacy-preserving CBIR method combined with an improved BoVW model and orthogonal transformation.

The specific retrieval process is given as follows:

3.1 Configuration Stage

In configuration stage, visual dictionary, encryption/ decryption key, and Gaussian orthonormal matrix are constructed for later usage.

• $\left(VD\right)\leftarrow VDGen\left(TB\right)$: Content owner generates visual dictionary VD from training image dataset TB, where two steps are included: SIFT descriptors are extracted from TB, which have been widely used due to its distinctiveness and robust matching across a substantial range of affine distortion, addition of noise, and change in illumination; Descriptors are quantized into visual words to form a visual dictionary with k-means clustering method.

• $\left(K_I,K_j\right)\leftarrow KeyGen\left(1^{\alpha }\right)$: Encryption key $K_I$, $K_J$ are generated with $KeyGen\left(\right)$ function, where α is a security parameter used to generate key;

• $\left(P,B\right)\leftarrow MatrixGen\left(\right)$: A random Gaussian orthonormal matrix is generated as P, B, R, and S are selected from B in terms of users’ practical needs on security or retrieval performance.

• $\left({\tau }_{q\left(x_j\right)}\right)\leftarrow MedianMatrixGen\left(P,q\left(x_j\right),x_j\right)$: The descriptors $x_j$ from $VD$ are assigned to their closest centroid $q\left(x_j\right)$ and projected by $P$ to$z=P{x}_j=[z_1,\dots ,z_{d_b}{]}^T$, then median value ${\tau }_{q\left(x_j\right)}$ of all vectors that belong to the same centroid $q\left(x_j\right)$ is generated.

3.2 Content Owner Side

• $\left(S_{x_j}\right)\leftarrow SigBuild\left(Img,VD,P,{\tau }_{q\left(x_j\right)}\right)$: Sift descriptors $x_j$ are extracted from images $Img$ that will be outsourced to CSP, assigned to particular cluster centers $q\left(x_j\right)$($j=1,\dots ,m_j$) in $VD$, and $x_j$ is mapped to vectors with a orthogonal projection matrix P, then binary signature of $x_j$ is generated by comparing with median matrix ${\tau }_l$ according to Eq.(1). Therefore a descriptor $x_j$ can be represented as a signature$S_{x_j}=\left(q\left(x_j\right),b\left(x_j\right)\right)$, where $b\left(x_j\right)=\left(b_1\left(x_j\right),\dots ,b_{d_b}\left(x_j\right)\right)$.

• $\left(q_{ef}\left(x_j\right),b_{ef}\left(x_j\right)\right)\leftarrow \text{ Secure }IndexBuild\left(S_{x_j},K_I,R\right)$: Orthogonal Decomposition is applied to $S_x$ to obtain encryption field $S_{1x_j}=\left(q_1\left(x_j\right),b_1\left(x_j\right)\right)$, encryption algorithm AES are executed on it to obtain$q_{1e}\left(x_j\right),b_{1e}\left(x_j\right))$ according to Equation (8), finally encrypted features $\left(q_{ef}\left(x_j\right),b_{ef}\left(x_j\right)\right)$ are obtained by means of orthogonal composition. TF-IDF is used to construct secure image index.

• $\left(Im{g}_e\right)\leftarrow \text{ SecureImg }Build\left(Img,K_J\right)$: Images that will be outsourced to CSP are encrypted by $K_J$ and AES encryption algorithm, cipher-images $Im{g}_e$ are obtained.

Data owner uploads cipher-images $Im{g}_e$ and corresponding secure image index to CSP.

3.3 User Side

• User sends query request to the data owner. After identity authentication, owner sends secure parameters (R, S, P,$K_I$) to users securely.

• $\left({\text{S}}_q\right)\leftarrow SigBuild\left(Im{g}_q,VD,P\right)$: Sift descriptors $x_q$ are extracted from$Im{g}_q$, and binary signature $S_q=\left(q\right(x_q),b(x_q\left)\right)$ is generated with the method we mentioned in 3.2.

• $\left(q_{ef}\right(x_q),b_{ef}(x_q\left)\right)\leftarrow \text{ TrapdoorBuild }\left(s_q,R,K_I\right)$: Orthogonal Decomposition is applied on signature $S_q$ to obtain encryption field $S_{1q}$, encryption algorithm AES and encryption key $K_I$ are executed on $S_{1q}$, then orthogonal composition is applied to obtain encrypted features $\left(q_{ef}\right(x_q),b_{ef}(x_q)$) as trapdoor. User sends trapdoor to the CSP as query.

• $Img\leftarrow Dec\left(Im{g}_{eq},K_J\right)$: After receiving retrieval results $Im{g}_{eq}$ from CSP, user decrypts cipher images with $K_J$ and obtains requested images.

3.4 CSP Side

• CSP obtains sub-matrix S content data owner securely, operates orthogonal decomposition on encrypted feature $\left(q_{ef}\left(x_{i,j}\right),b_{ef}\left(x_{i,j}\right)\right)$ and search trapdoor $\left(q_{ef}\right(x_q),b_{ef}(x_q)$ sent by users according to Equation (10), then distance comparison field is obtained to compare the distance between $\left(q_{2f}\left(x_{i,j}\right),b_{2f}\left(x_{i,j}\right)\right)$, $\left(q_{2f}\right(x_q),b_{2f}(x_q\left)\right)$.

• $\left(Im{g}_{eq}\right)\leftarrow DistComp\left(\right(q_{2f}\left(x_{i,j}\right),b_{2f}\left(x_{i,j}\right))$, $\left(q_{2f}\right(x_q),b_{2f}(x_q\left)\right)$: The matching scores are available by the similarity comparison between $\left(q_{2f}\left(x_{i,j}\right),b_{2f}\left(x_{i,j}\right)\right)$, $\left(q_{2f}\right(x_q),b_{2f}(x_q\left)\right)$ according to Equations (11–13). The higher the matching scores, the more similar the corresponding image is to the query image, and the encrypted images $Im{g}_{eq}$ with the highest scores are sent to the requested user. (11) $f\left(x_{i,j},x_{\mathrm{q}}\right)=\begin{array}{c}\left\{\begin{array}{l}\frac{(idf{\left(q_{2f}\left(x_{i,j}\right)\right)}^2\times \exp \left(\frac{-L{\left(x_{i,j},x_{\mathrm{q}}\right)}^2}{{\sigma }^2}\right)}{\sqrt{tf}}\\ 0\text{ otherwise}\end{array}\right.\\ if{q}_{2f}\left(x_{i,j}\right)q_{2f}\left(x_q\right)andL\left(x_{i,j},x_q\right)\le L_d\end{array}$(11) (12) $L\left(x_j,x_{\mathrm{q}}\right)=\sqrt{{\left(b_{2f}\left(x_{i,j}\right)-b_{2f}\left(x_{\mathrm{q}}\right)\right)}^2}$(12) (13) $s=\sum _{q=1}^{m\mathrm{\prime }}\sum _{j=1}^{m_j}f\left(x_{i,j},x_q\right)$(13) where $\text{idf }\left(\right)$ is a measure of the general importance of a visual word and indicates the ratio of the number of images containing a given word to the total number of images; tf describes the number of occurrences of a given visual word in an image; $L_d$ and σ are empirical values obtained from experiment; $m_j$ is the number of local descriptors of the $j$-th database image, $m^{\mathrm{\prime }}$ is the number of local descriptors of the query image.

4. EXPERIMENTAL RESULTS

In this chapter, we present an experimental evaluation of the proposed method and compare retrieval results with several classical privacy-preserved image retrieval methods, including random projection, bit-plane randomization, and randomized unary encoding proposed in [14], Pallier algorithm put forward in [19]. We perform experiment on INRIA Holidays dataset [28] containing 1491 images, which are divided into 500 queries and 991 corresponding relevant images. The performance of our scheme is evaluated in terms of security and retrieval performance.

4.1 Security

The orthogonal decomposition is implemented on image features instead of images, and images can be encrypted by any encryption method. We use AES-128 encryption algorithm to encrypt images. Even with the most powerful biclique attacks, the computational complexity is $2^{126.1}$, which means the security of AES will not be broken. For the encrypted features, since S is known by CSP, and as a sub-matrix of B, R, and S are not completely independent, so there is a potential secure problem. The complexity of attack B and R in the case of known the sub-matrix S is as follows: (14) $d_{\mathit{B}}\sim 2^{n^{2}k-k_m}+2^{\frac{n\left(n+1\right)\left(k-1\right)}{2}+k_0}$(14) (15) $d_{\mathit{R}}\sim 2^{3klo{g}_2^{m\left(n-m\right)}+m^{2}k-k_m}+2^{\frac{m\left(m+1\right)k}{2}+k_0}$(15) where the size of B is n, element of B is k-bit fixed word-length integer, m is the number of columns of the sub-matrix S, and $k_m$ is a factor that includes the efficiency improvement achieved over an exhaustive search by using a different algorithm. Therefore, the complexity of attacking B and attacking R tend to O($n^{2}k$) and O($m^{2}k$), which means even if orthogonal sub-matrix S is known, the computational complexity of attacking the orthogonal matrix B and the orthogonal sub-matrix R is exponentially increasing; therefore, the security of features can also be guaranteed.

Three different metrics are used to evaluate the security of features: autocorrelation function, information entropy, and histogram distribution.

(1) Autocorrelation function

The autocorrelation function of a feature vector measures how correlated the neighboring feature elements are [29]. Suppose n is the size of feature vectors, T is the delay of signals, then the autocorrelation function R(T) is calculated as follows: (16) $R\left(T\right)=\sum _{i=1}^{n}x\left(i\right)x\left(i-T\right)$(16) The autocorrelation function for the raw color histogram, randomized features of three different retrieval methods of Lu, random uniform vector, and the encrypted features using algorithm proposed in this paper are shown in Figure 2. We can see that the raw color feature has non-negligible correlation, which indicates there exists strong association between adjacent vectors, while our method has similar autocorrelation as random features encrypted by Pallier algorithm and is much lower than Lu’s three methods, which shows encrypted features using our method has low autocorrelation thus has good confidentiality.

Figure 2: Autocorrelation function on different algorithms

(2) Information entropy

Information entropy describes the uncertainty of random variables. A higher entropy of encrypted features means that it has a distribution closer to uniform, thus higher randomness and higher security. The entropy is defined as follows: (17) $H=-\sum _{\mathrm{i}=\text{0}}^{n}p\left(x\right(i\left)\right)\times {\log }_{2}p\left(x\right(i\left)\right)$(17) where $x\left(i\right)$ is the feature, n is the length of $x\left(i\right)$, $p\left(x\right(i\left)\right)$is the probability of $x\left(i\right)$. A higher entropy of encrypted features means it has a distribution closer to uniform, thus higher randomness and higher security. The entropy of encrypted features for different algorithm is given in Table 1.

Table 1: Entropy for different methods

Feature type	Entropy
Color histogram	1.18
Visual words	2.59
Bit-plane randomization	6.12
Random projection	5.94
Randomized unary encoding	6.69
Proposed scheme	6.83
Uniform random features	7.96

From this table, we can see that both the color histogram and visual words have low entropy, which means that there is strong inherent correlation among feature values. The features encrypted by the proposed method achieve higher entropy than Lu’s three methods but lower than uniform random features encrypted by Pallier’s algorithm. Although its security is not as good as features encrypted by complex homomorphic algorithm, its time complexity consumed by encryption is much lower than Pallier’s algorithm; therefore, it is a more practical method.

(3) Histogram distribution

Histogram distribution is usually employed to evaluate security of encrypted data. In the proposed method, secure image index of each image is outsourced to CSP, which consists of two parts: $\left(q_{ef}\right(x_j),b_{ef}(x_j)$), $q_{ef}\left(x_j\right)$ means the encrypted cluster centers that feature $x_j$ belonging to; $b_{ef}\left(x_j\right)$ means the encrypted binary signature of x_j. The CSP can obtain only the frequency histogram distribution of encrypted visual words, but it can hardly acquire image features from its encrypted binary signature. Therefore it is difficult for the CSP to only use frequency histogram distribution to recover the image features and infer the image content without correct decryption key.

Figure 3 shows the comparative results of frequency histogram distribution between the encrypted visual words and the original visual words of an image. From Figure 3, we can see that the histogram of encrypted visual words is different from that of original visual words, and it is difficult for attackers to infer the image content information only through frequency histogram distribution of the encrypted visual words.

Figure 3: The comparative results of histogram distribution

4.2 Analysis of Time Complexity

Assume the size of the selected operation data X is n, the dimension of sub-matrix S is k(k < n), g is the number of bit-planes to randomize, m is the dimension of projected features, M is the eigenvector of the feature vector, then the time complexity comparison results with other methods are given in Table 2.

Table 2: Time complexity comparison of different methods

Method	Time complexity	Practical computation times (s)
Proposed scheme	O(kn)	0.21
Bit-plane randomization	O(gn)	0.01
Random projection	O(mn)	0.12
Randomized unary encoding	O(mnM)	3.65
Pallier Homomorphic encryption	O($n^4(n^n{)}^2$)	1478.75

From Table 2, we can see that the time complexity of the Pallier homomorphism method is the highest; Randomized unary encoding is higher; the proposed method is lower than these methods and similar to random projection proposed by Lu.

4.3 Retrieval Precision

Retrieval precision is evaluated by mean average precision (mAP) that is widely used to measure image retrieval performance for a group of queries [30]. It is defined as follows: (18) $\text{mAP}=\frac{\sum _{q=1}^{Q}aveP\left(q\right)}{Q}$(18) where Q is the number of queries, avePq represents the average value of all the precisions measured each time a new relevant image is retrieved. In this experiment, a group of 500 query images are retrieved in a database containing 991 images, and the MAP of query images for different encryption methods are computed as Table 3 shows.

Table 3: mAP for different methods

Method	mAP
Original HE method	80.59
Proposed scheme	79.44
Bit-plane randomization	38.15
Random projection	53.80
Randomized unary encoding	51.67
Homomorphic encryption	53.94

From the table, we can see that the retrieval result of our scheme is much better than those of other methods. Meanwhile, we also compute the mAP of the unencrypted HE method, and its MAP is 80.59 which is roughly equal to the result of our encrypted scheme, namely, our method can hardly reduce retrieval accuracy of original HE.

Through the above analysis of retrieval performance, it is proved that the proposed method has a good trade-off between security and retrieval performance, it not only guarantees the security of retrieval in cloud environment but also ensures the high precision of retrieval for practical applications.

5. CONCLUSION

A privacy-preserving image retrieval method based on improved BoVW model in cloud computing is proposed in the paper. An improved BOVW method based on Hamming embedding which provides binary signatures refining visual words are combined with orthogonal transformation to achieve privacy-preserving image retrieval under cloud environment. Experiments show that our scheme has obvious advantages in security and retrieval precision compared with other methods. Future research will focus on exploring retrieval efficiency further.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research is supported by the National Natural Science Foundation of China [grant number 41571426], National Key Research and Development Program of China under [grant number 2017YFB0504202], and Wuhan Applied Basic Research Programs [grant number 2017010201010114].

Notes on contributors

Jiaying Gong

Jiaying Gong received her master’s degree in communication and information system from the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, China. Her research interest is multimedia information security. E-mail: [email protected]

Yanyan Xu

Yanyan Xu is a professor of the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, China. Her main research interests include cloud computing security and big data privacy protection, multimedia network communications.

Xiao Zhao

Xiao Zhao is a master’s degree candidate of communication and information system in the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, China. His research interests include multimedia information security and multimedia network communication. E-mail: [email protected]