A comprehensive study on face recognition: methods and challenges

References

• Jiang Z. Camera network analysis for visual surveillance in industrial electronic context. Multimed Tools Appl. 2017:1–16. https://link.springer.com/article/10.1007/s11042-017-5298-1.
   Google Scholar
• Peacock C, Goode A, Brett A. Automatic forensic face recognition from digital images. Sci Justice. 2004;44(1):29–34.
   PubMed Web of Science ®Google Scholar
• Burton AM, Wilson S, Cowan M, et al. Face recognition in poor-quality video: evidence from security surveillance. Psychol Sci. 1999;10(3):243–248.
   Web of Science ®Google Scholar
• Pentland A, Choudhury T. Face recognition for smart environments. Computer. 2000;33(2):50–55.
   Web of Science ®Google Scholar
• Feng W, Zhou J, Dan C, et al. Research on mobile commerce payment management based on the face biometric authentication. Int J Mob Commun. 2017;15(3):278–305.
   Web of Science ®Google Scholar
• Jain AK, Li SZ. Handbook of face recognition. illustrated, reprint ed. New York: Springer; 2011.
   Google Scholar
• Besbes F, Trichili H, Solaiman B. Multimodal biometric system based on fingerprint identification and iris recognition. In: 2008 3rd international conference on information and communication technologies: from theory to applications. 2008. p. 1–5.
   Google Scholar
• Darwin C, Prodger P. The expression of the emotions in man and animals. illustrated, annotated ed. London: Oxford University Press; 1998.
   Google Scholar
• Sirovich L, Kirby M. Low-dimensional procedure for the characterization of human faces. J. Opt. Soc. Am. A. 1987;4(3):519–524.
   PubMedGoogle Scholar
• Cootes TF, Walker K, Taylor CJ. View-based active appearance models. In: Fourth IEEE international conference on automatic face and gesture recognition. 2000. p. 227–232.
   Google Scholar
• Zaharia T, Preteux FJ. 3D-shape-based retrieval within the MPEG-7 framework. In: Nonlinear image processing and pattern analysis xii, Vol. 4304. 2001. p. 133–145.
   Google Scholar
• Ojala T, Pietikinen M, Harwood D. A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 1996;29(1):51–59.
   Web of Science ®Google Scholar
• Smith RS, Windeatt T. Facial expression detection using filtered local binary pattern features with ECOC classifiers and platt scaling. In: Proceedings of the first workshop on applications of pattern analysis. 2010. p. 111–118.
   Google Scholar
• Sun N, Zheng W, Sun C, et al. Gender classification based on boosting local binary pattern. In: International symposium on neural networks. 2006. p. 194–201.
   Google Scholar
• Jabid T, Kabir MH, Chae O. Gender classification using local directional pattern (LDP). In: 20th international conference on pattern recognition. 2010a. p. 2162–2165.
   Google Scholar
• Dalal N, Triggs B. Histograms of oriented gradients for human detection. 2005.
   Google Scholar
• Tan X, Triggs B. Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process. 2010;19(6):1635–1650.
   PubMed Web of Science ®Google Scholar
• Grgic M, Delac K, Grgic S. SCface–surveillance cameras face database. Multimed Tools Appl. 2011;51(3):863–879.
   Web of Science ®Google Scholar
• Phillips PJ, Martin A, Wilson CL, et al. An introduction evaluating biometric systems. Computer. 2000;33(2):56–63.
   Web of Science ®Google Scholar
• Prabhakar S, Pankanti S, Jain AK. Biometric recognition: security and privacy concerns. IEEE Secur Priv. 2003;2:33–42.
   Google Scholar
• Alhussein M. Automatic facial emotion recognition using weber local descriptor for e-Healthcare system. Cluster Comput. 2016;19(1):99–108.
   Web of Science ®Google Scholar
• Hossain MS. Patient state recognition system for healthcare using speech and facial expressions. J Med Syst. 2016;40(12):272.
   PubMed Web of Science ®Google Scholar
• Brunelli R, Poggio T. Face recognition: features versus templates. IEEE Trans Pattern Anal Mach Intell. 1993;15(10):1042–1052.
   Web of Science ®Google Scholar
• Ahonen T, Hadid A, Pietikainen M Face recognition with local binary patterns. In: European conference on computer vision. 2004. p. 469–481.
   Google Scholar
• Ahonen T, Hadid A, Pietikainen M. Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell. 2006;28(12):2037–2041.
   PubMed Web of Science ®Google Scholar
• Yin X, Liu X. Multi-task convolutional neural network for pose-invariant face recognition. IEEE Trans Image Process. 2018;27(2):964–975.
   PubMed Web of Science ®Google Scholar
• Ayyavoo T, Suseela JJ. Illumination pre-processing method for face recognition using 2D DWT and CLAHE. IET Biometrics. 2018;7(4):380–390.
   Web of Science ®Google Scholar
• Cui X, Zhou P, Yang W. Local dominant orientation feature histograms for face recognition. In: Applied informatics. Vol. 4. 2017. p. 14.
   Google Scholar
• Xie Z, Jiang P, Zhang S. Fusion of LBP and HOG using multiple Kernel learning for infrared face recognition. In: IEEE/ACIS 16th international conference on computer and information science. 2017. p. 81–84.
   Google Scholar
• Fanaee F, Yazdi M, Faghihi M. Face image super-resolution via sparse representation and wavelet transform. Signal Image Video Process. 2018;13:1–8.
   Web of Science ®Google Scholar
• Juefei-Xu F, Luu K, Savvides M, et al. Investigating age invariant face recognition based on periocular biometrics. In: International joint conference on biometrics. 2011. p. 1–7.
   Google Scholar
• Turk M, Pentland A. Eigenfaces for recognition. J Cogn Neurosci. 1991a;3:71–86.
   PubMed Web of Science ®Google Scholar
• Moghaddam B, Pentland A. Probabilistic visual learning for object representation. IEEE Trans Pattern Anal Mach Intell. 1997;19(7):696–710.
   Web of Science ®Google Scholar
• Sung KK, Poggio T. Example-based learning for view-based human face detection. IEEE Trans Pattern Anal Mach Intell. 1998;20(1):39–51.
   Web of Science ®Google Scholar
• Osuna E, Freund R, Girosit F. Training support vector machines: an application to face detection. In: IEEE computer society conference on computer vision and pattern recognition. 1997. p. 130–136.
   Google Scholar
• Yang MH, Roth D, Ahuja N. A SNoW-based face detector. Adv Neural Inf Process Syst. 2000;862–868.
   Google Scholar
• Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2001;1:I–I.
   Google Scholar
• Viola P, Jones MJ. Robust real-time face detection. Int J Comput Vis. 2004;57(2):137–154.
   Web of Science ®Google Scholar
• Yin Q, Tang X, Sun J. An associate-predict model for face recognition. Comput Vis Pattern Recognit. 2011:497–504. https://ieeexplore.ieee.org/document/5995494/citations.
   Google Scholar
• Blanz V, Vetter T. A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th annual conference on computer graphics and interactive techniques. 1999. p. 187–194.
   Google Scholar
• Starovoitov VV, Samal D. A geometric approach to face recognition. In: Nonspecific interstitial pneumonia. 1999. p. 210–213.
   Google Scholar
• Weng R, Lu J, Tan YP. Robust point set matching for partial face recognition. IEEE Trans Image Process. 2016;25(3):1163–1176.
   PubMed Web of Science ®Google Scholar
• Yang F, Yang W, Gao R, et al. Discriminative multidimensional scaling for low-resolution face recognition. IEEE Signal Process Lett. 2018;25(3):388–392.
   Web of Science ®Google Scholar
• Jabid T, Kabir MH, Chae O. Local directional pattern (LDP) – a robust image descriptor for object recognition. In: Seventh IEEE international conference on advanced video and signal based surveillance. 2010b. p. 482–487.
   Google Scholar
• Sariyanidi E, Gunes H, Cavallaro A. Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE Trans Pattern Anal Mach Intell. 2015;37(6):1113–1133.
   PubMed Web of Science ®Google Scholar
• Goyani MM, Patel NM. Robust facial expression recognition using local haar mean binary pattern. J Inf Sci Eng. 2018;16:54–67.
   Google Scholar
• Rao LK, Rohini P, Reddy LP. Local color oppugnant quantized extrema patterns for image retrieval. Multidimens Syst Signal Process. 2018;30:1–23.
   Web of Science ®Google Scholar
• Gross R, Matthews I, Baker S. Appearance-based face recognition and light-fields. IEEE Trans Pattern Anal Mach Intell. 2004;26(4):449–465.
   PubMed Web of Science ®Google Scholar
• Sheikh Z, Thakare V. Wavelet based feature extraction technique for face recognition and retrieval: a review. Int Org Sci Res J Comput Eng. 2016;49–54.
   Google Scholar
• Pentland A, Moghaddam B, Starner T, et al. View-based and modular eigenspaces for face recognition. 1994.
   Google Scholar
• Belhumeur PN, Hespanha JP, Kriegman DJ. Eigenfaces vs. fisherfaces: recognition using class specific linear projection, Vol. 19. 1997. (Tech. Rep.).
   Google Scholar
• Bartlett MS, Movellan JR, Sejnowski TJ. Face recognition by independent component analysis. IEEE Trans Neural Networ. 2002;13(6):1450. A Publication of the IEEE Neural Networks Council.
   PubMed Web of Science ®Google Scholar
• Bay H, Ess A, Tuytelaars T, et al. Speeded-up robust features (SURF). Comput Vis Image Underst. 2008;110(3):346–359.
   Web of Science ®Google Scholar
• Susan S, Jain A, Sharma A, et al. Fuzzy match index for scale-invariant feature transform (SIFT) features with application to face recognition with weak supervision. IET Image Process. 2015;9(11):951–958.
   Web of Science ®Google Scholar
• Tu J, Zhang Z, Zeng Z, et al. Face localization via hierarchical condensation with fisher boosting feature selection. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, Vol. 2. 2004. p. II–II.
   Google Scholar
• Konen W, Malsburg Cv.d.. Learning to generalize from single examples in the dynamic link architecture. Neural Comput. 1993;5(5):719–735.
   Web of Science ®Google Scholar
• Wiskott L, Fellous JM, Krger N, et al. Face recognition by elastic bunch graph matching. In: International conference on computer analysis of images and patterns. 1997. p. 456–463.
   Google Scholar
• Lin SH, Kung SY, Lin LJ. Face recognition/detection by probabilistic decision-based neural network. IEEE Trans Neural Networ. 1997;8(1):114–132.
   PubMedGoogle Scholar
• Liu C, Wechsler H. Face recognition using evolutionary pursuit. In: European conference on computer vision, Vol. 1407. 1998. p. 596–612.
   Google Scholar
• Yang MH. Kernel eigenfaces vs. Kernel fisherfaces: face recognition using Kernel methods. In: Proceedings of fifth IEEE international conference on automatic face gesture recognition. Washinton, DC; 2002. p. 0215.
   Google Scholar
• Srisuk S, Petrou M, Kurutach W, et al. Face authentication using the trace transform. In: IEEE computer society conference on computer vision and pattern recognition, Vol. 1. 2003.
   Google Scholar
• Queirolo CC, Silva L, Bellon OR, et al. 3D face recognition using simulated annealing and the surface interpenetration measure. IEEE Trans Pattern Anal Mach Intell. 2010;32(2):206.
   PubMed Web of Science ®Google Scholar
• Liu M, Wang R, Huang Z, et al. Partial least squares regression on Grassmannian manifold for emotion recognition. In: Proceedings of the 15th ACM on international conference on multimodal interaction. 2013. p. 525–530.
   Google Scholar
• Lei Z, Pietikinen M, Li SZ. Learning discriminant face descriptor. IEEE Trans Pattern Anal Mach Intell. 2014;36(2):289–302.
   PubMed Web of Science ®Google Scholar
• Akhtar Z, Rattani A. A face in any form: new challenges and opportunities for face recognition technology. Computer. 2017;50(4):80–90.
   Web of Science ®Google Scholar
• Deng J, Guo J, Xue N, et al. Arcface: additive angular margin loss for deep face recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2019. p. 4690–4699.
   Google Scholar
• Wang H, Wang Y, Zhou Z, et al. Cosface: large margin cosine loss for deep face recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. p. 5265–5274.
   Google Scholar
• Georgescu MI, Ionescu RT, Popescu M. Local learning with deep and handcrafted features for facial expression recognition. IEEE Access. 2019;7:64827–64836.
   Web of Science ®Google Scholar
• Guo Y, Zhao G, Pietikainen M. Discriminative features for texture description. Pattern Recognit. 2012;45(10):3834–3843.
   Web of Science ®Google Scholar
• Shan C, Gritti T. Learning discriminative LBP-histogram bins for facial expression recognition. In: BMVC. 2008. p. 1–10.
   Google Scholar
• Shan S, Zhang W, Su Y, et al. Ensemble of piecewise FDA based on spatial histograms of local (Gabor) binary patterns for face recognition. In: Null. 2006. p. 606–609.
   Google Scholar
• Turtinen M, Pietikinen M. Contextual analysis of textured scene images. In: British machine vision conference, Vol. 2. 2006.
   Google Scholar
• Yang H, Wang Y. A LBP-based face recognition method with Hamming distance constraint. In: Fourth international conference on image and graphics. 2007. p. 645–649.
   Google Scholar
• Liao S, Chung AC. Face recognition by using elongated local binary patterns with average maximum distance gradient magnitude. In: Asian conference on computer vision. 2007. p. 672–679.
   Google Scholar
• Tan X, Triggs B. Fusing Gabor and LBP feature sets for kernel-based face recognition. In: International workshop on analysis and modeling of faces and gestures. 2007. p. 235–249.
   Google Scholar
• Trefn J, Matas J. Extended set of local binary patterns for rapid object detection. In: Computer vision winter workshop. 2010. p. 1–7.
   Google Scholar
• Mu Y, Yan S, Liu Y, et al. Discriminative local binary patterns for human detection in personal album. In: IEEE conference on computer vision and pattern recognition. 2008. p. 1–8.
   Google Scholar
• Guo Z, Zhang L, Zhang D, et al. Rotation invariant texture classification using adaptive LBP with directional statistical features. In: 17th IEEE international conference on image processing. 2010. p. 285–288.
   Google Scholar
• Ojansivu V, Heikkil J. Blur insensitive texture classification using local phase quantization. In: International conference on image and signal processing. 2008. p. 236–243.
   Google Scholar
• Pietikinen M, Ojala T, Xu Z. Rotation-invariant texture classification using feature distributions. Pattern Recognit. 2000;33(1):43–52.
   Web of Science ®Google Scholar
• Ahonen T, Pietikinen M. Soft histograms for local binary patterns. In: Proceedings of the finnish signal processing symposium. 2007. Vol. 5, p. 1.
   Google Scholar
• Yan S, Shan S, Chen X, et al. Locally assembled binary (LAB) feature with feature-centric cascade for fast and accurate face detection. In: IEEE conference on computer vision and pattern recognition. 2008. p. 1–7.
   Google Scholar
• Wolf L, Hassner T, Taigman Y. Descriptor based methods in the wild. In: Workshop on faces in'real-life'images: detection, alignment, and recognition. 2008.
   Google Scholar
• Fu X, Wei W. Centralized binary patterns embedded with image Euclidean distance for facial expression recognition. In: Fourth international conference on natural computation, Vol. 4. 2008. p. 115–119.
   Google Scholar
• An KH, Chung MJ. Cognitive face analysis system for future interactive TV. IEEE Trans Consum Electr. 2009;55(4):2271–2279.
   Web of Science ®Google Scholar
• Guo Z, Zhang L, Zhang D. Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recognit. 2010;43(3):706–719.
   Web of Science ®Google Scholar
• Nanni L, Brahnam S, Lumini A. A local approach based on a local binary patterns variant texture descriptor for classifying pain states. Expert Syst Appl. 2010;37(12):7888–7894.
   Web of Science ®Google Scholar
• Wang SH, Phillips P, Dong ZC, et al. Intelligent facial emotion recognition based on stationary wavelet entropy and Jaya algorithm. Neurocomputing. 2018;272:668–676.
   Web of Science ®Google Scholar
• Zhang YD, Yang ZJ, Lu HM, et al. Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine, and stratified cross validation. IEEE Access. 2016;4:8375–8385.
   Web of Science ®Google Scholar
• Papandreou G, Maragos P. Adaptive and constrained algorithms for inverse compositional active appearance model fitting. In: IEEE conference on computer vision and pattern recognition. 2008. p. 1–8.
   Google Scholar
• Liu X. Generic face alignment using boosted appearance model. In: IEEE conference on computer vision and pattern recognition. 2007. p. 1–8.
   Google Scholar
• Liu X. Discriminative face alignment. IEEE Trans Pattern Anal Mach Intell. 2009;31(11):1941.
   PubMed Web of Science ®Google Scholar
• Liebelt J, Xiao J, Yang J. Robust AAM fitting by fusion of images and disparity data. In: IEEE computer society conference on computer vision and pattern recognition, Vol. 2. 2006. p. 2483–2490.
   Google Scholar
• Xiao J, Baker S, Matthews I, et al. Real-time combined 2D+ 3D active appearance models real-time combined 2d+ 3d active appearance models. In: Computer vision pattern recognition. 2004. p. 535–542.
   Google Scholar
• Hamsici OC, Martinez AM. Active appearance models with rotation invariant Kernels. In: IEEE 12th international conference on computer vision. 2009. p. 1003–1009.
   Google Scholar
• Lee HS, Kim D. Tensor-based AAM with continuous variation estimation: application to variation-robust face recognition. IEEE Trans Pattern Anal Mach Intell. 2009;31(6):1102–1116.
   PubMed Web of Science ®Google Scholar
• Jiao F, Li S, Shum HY, et al. Face alignment using statistical models and wavelet features. In: IEEE computer society conference on computer vision and pattern recognition proceedings, Vol. 1. 2003. p. I–I.
   Google Scholar
• Zhang L, Ai H. Multi-view active shape model with robust parameter estimation. In: 18th international conference on pattern recognition, Vol. 4. 2006. p. 469–468.
   Google Scholar
• Li Y, Ito W. Shape parameter optimization for adaboosted active shape model. In: Tenth IEEE international conference on computer vision, Vol. 1. 2005. p. 251–258.
   Google Scholar
• Brunet N, Perez F, De la Torre F. Learning good features for active shape models. In: IEEE 12th international conference on computer vision workshops. 2009. p. 206–211.
   Google Scholar
• Vogler C, Li Z, Kanaujia A, et al. The best of both worlds: combining 3d deformable models with active shape models. In: IEEE 11th international conference on computer vision. 2007. p. 1–7.
   Google Scholar
• Kaya Y, Kobayashi K. A basic study on human face recognition. In: Frontiers of pattern recognition. 1972. p. 265–289). Elsevier.
   Google Scholar
• Wang M, Deng W. Deep face recognition: a survey. 2018. arXiv preprint arXiv:1804.06655.
   Google Scholar
• Liu B, Deng W, Zhong Y, et al. Fair loss: margin-aware reinforcement learning for deep face recognition. In: Proceedings of the IEEE international conference on computer vision. 2019. p. 10052–10061.
   Google Scholar
• Nguyen DT, Pham TD, Baek NR, et al. Combining deep and handcrafted image features for presentation attack detection in face recognition systems using visible-light camera sensors. Sensors. 2018;18(3):699.
   Web of Science ®Google Scholar
• Wang X, Wang S, Wang J, et al. Co-mining: deep face recognition with noisy labels. In: Proceedings of the IEEE international conference on computer vision. 2019. p. 9358–9367.
   Google Scholar
• Sun Y, Liang D, Wang X, et al. Deepid3: face recognition with very deep neural networks. 2015. arXiv preprint arXiv:1502.00873.
   Google Scholar
• Verma VK, Srivastava S, Jain T, et al. Local invariant feature-based gender recognition from facial images. In: Soft computing for problem solving. Springer; 2019. p. 869–878.
   Google Scholar
• Peng C, Wang N, Li J, et al. DLFace: deep local descriptor for cross-modality face recognition. Pattern Recognit. 2019;90:161–171.
   Web of Science ®Google Scholar
• Král P, Vrba A, Lenc L. Enhanced local binary patterns for automatic face recognition. In: International conference on artificial intelligence and soft computing. 2019. p. 27–36.
   Google Scholar
• Zhou D, Yang D, Zhang X. Exploring joint encoding of multi-direction local binary patterns for image classification. Multimed Tools Appl. 2017;77(15):1–25.
   Web of Science ®Google Scholar
• Gross R, Matthews I, Cohn J, et al. Multi-pie. Image Vis Comput. 2010;28(5):807–813.
   Web of Science ®Google Scholar
• Weyrauch B, Heisele B, Huang J, et al. Component-based face recognition with 3D morphable models. In: Conference on computer vision and pattern recognition workshop. 2004. p. 85–85.
   Google Scholar
• Martinez AM. The AR face database. 1998. (CVC Technical Report24).
   Google Scholar
• Gao W, Cao B, Shan S, et al. The CAS-PEAL large-scale Chinese face database and evaluation protocols. Joint Research and Development Laboratory, CAS; 2004. (Technique Report No. JDL-TR_04_FR_001).
   Google Scholar
• Dornaika F, Lazkano E, Sierra B. Improving dynamic facial expression recognition with feature subset selection. Pattern Recognit Lett. 2011;32(5):740–748.
   Web of Science ®Google Scholar
• Lajevardi SM, Hussain ZM. Contourlet structural similarity for facial expression recognition. In: 2010 IEEE international conference on acoustics, speech and signal processing. 2010. p. 1118–1121.
   Google Scholar
• Xing Y, Luo W. Facial expression recognition using local Gabor features and adaboost classifiers. In: 2016 international conference on progress in informatics and computing (pic). 2016. p. 228–232.
   Google Scholar
• Tyagi V. Content-based image retrieval: ideas, influences, and current trends. Singapore: Springer; 2018.
   Google Scholar
• Lahasan B, Lutfi SL, San-Segundo R. A survey on techniques to handle face recognition challenges: occlusion, single sample per subject and expression. Artif Intell Rev. 2019;52(2):949–979.
   Web of Science ®Google Scholar
• Liu M, Fu H, Wei Y, et al. Light field-based face liveness detection with convolutional neural networks. J Electron Imaging. 2019;28(1):013003.
   Web of Science ®Google Scholar