Masked Face Recognition using deep learning models

Omer T. Hamajan; Abbas M. Ali

doi:10.21271/ZJPAS.36.2.2

Authors

Omer T. Hamajan Department of Software Engineering and Informatics, College of Engineering, Salahaddin University-Erbil, Iraq
Abbas M. Ali Department of Software Engineering and Informatics, College of Engineering, Salahaddin University-Erbil, Iraq

DOI:

https://doi.org/10.21271/ZJPAS.36.2.2

Keywords:

Convolution Neural Network (CNN), Deep Learning, Masked Face Dataset, Masked Face Recognition, YOLO, Mask R-CNN

Abstract

Face recognition has become indispensable in our daily lives as a quick and painless technique of confirming our identities since in the era of wearing face masks the traditional face recognition system may not effectively recognize the concerned person as an important part of the face (mouth, nose, and chin) which makes a substantial contribution to the face recognition process are occluded and partially hidden. The objective of our research is to tackle the challenges of the partially occluded face with a mask by training our custom dataset using those powerful pre-trained deep learning models like YOLO and Mask R-CNN which have not been used before for this purpose and to compare which one is outperforming the better results. To this end, models like (YOLOv5, YOLOv7, YOLOv8, and Mask R-CNN) have been employed and trained on the created dataset to check the accuracy and robustness of the occluded face recognition process. In addition, an online dataset such as (mfr2) which contains celebrities, and politicians masked and unmasked faces after expansion with more images has been used. The experimental findings demonstrate that the proposed algorithms give an accurate result, we achieve an accuracy of 97.5% using YOLOv8s, an accuracy of 89.7% using YOLOv7, and an accuracy of 89% using YOLOv5x, while an accuracy of 94.5% using Mask R-CNN. The study concludes that YOLOv8s outperforms the other models in masked face recognition.

References

X. Peng, M. Bennamoun, and A. S. Mian, “A training-free nose tip detection method from face range images,” Pattern Recognit., vol. 44, no. 3, pp. 544–558, 2011, doi: 10.1016/j.patcog.2010.09.015.

X. Lu, A. K. Jain, and D. Colbry, “Matching 2.5D face scans to 3D models,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 1, pp. 31–42, 2006, doi: 10.1109/TPAMI.2006.15.

M. L. Koudelka, M. W. Koch, and T. D. Russ, “A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., vol. 2005-Septe, pp. 1–8, 2005, doi: 10.1109/CVPR.2005.566.

H. T. Likassa, W. Xian, and X. Tang, “New Robust Regularized Shrinkage Regression for High-Dimensional Image Recovery and Alignment via Affine Transformation and Tikhonov Regularization,” Int. J. Math. Math. Sci., vol. 2020, 2020, doi: 10.1155/2020/1286909.

L. Song, Di. Gong, Z. Li, C. Liu, and W. Liu, “Occlusion robust face recognition based on mask learning with pairwise differential siamese network,” Proc. IEEE Int. Conf. Comput. Vis., vol. 2019-Octob, pp. 773–782, 2019, doi: 10.1109/ICCV.2019.00086.

M. Haghighat, S. Zonouz, and M. Abdel-Mottaleb, “CloudID: Trustworthy cloud-based and cross-enterprise biometric identification,” Expert Syst. Appl., vol. 42, no. 21, pp. 7905–7916, Jul. 2015, doi: 10.1016/j.eswa.2015.06.025.

E. A. M. Al-Rahawe, E. A. M. Alrahawe, V. T. Humbe, and G. N. Shinde, “MULTI-BIOMETRIC TRAITS FUSION: A REVIEW Multimodal biometric system View project publication View project MULTI-BIOMETRIC TRAITS FUSION: A REVIEW,” Int. J. Res. Anal. Rev., 2020, [Online]. Available: www.ijrar.org.

Y. Lecun, Y. Bengio, and R. 4g332, “Convolutional Networks for Images, Speech, and Time-Series.”

A. Alzu’bi, F. Albalas, T. Al-Hadhrami, L. B. Younis, and A. Bashayreh, “Masked face recognition using deep learning: A review,” Electron., vol. 10, no. 21, pp. 1–35, 2021, doi: 10.3390/electronics10212666.

A. Anwar and A. Raychowdhury, “Masked Face Recognition for Secure Authentication,” pp. 1–8, 2020, [Online]. Available: http://arxiv.org/abs/2008.11104.

Y. Taigman, M. A. Ranzato, T. Aviv, and M. Park, “Taigman_DeepFace_Closing_the_2014_CVPR_paper,” doi: 10.1109/CVPR.2014.220.

Y. Sun, Y. Chen, X. Wang, and X. Tang, “Deep learning face representation by joint identification-verification,” Adv. Neural Inf. Process. Syst., vol. 3, no. January, pp. 1988–1996, 2014.

M. Wang and W. Deng, “Deep face recognition: A survey,” Neurocomputing, vol. 429, pp. 215–244, 2021, doi: 10.1016/j.neucom.2020.10.081.

J. M. Alrikabi and K. H. Alibraheemi, “A Combination Approach for Masked Face Recognition Based on Deep Learning,” Adv. Mech., vol. 9, no. 3, p. 2021, 2021.

W. Hariri, “Efficient masked face recognition method during the COVID-19 pandemic,” Signal, Image Video Process., 2021, doi: 10.1007/s11760-021-02050-w.

Y. Zhang, J. Gao, and H. Zhou, “Breeds Classification with Deep Convolutional Neural Network,” ACM Int. Conf. Proceeding Ser., pp. 145–151, 2020, doi: 10.1145/3383972.3383975.

B. Mandal, A. Okeukwu, and Y. Theis, “Masked Face Recognition using ResNet-50,” 2021, [Online]. Available: http://arxiv.org/abs/2104.08997.

O. Mohammed Aarif K et al., “Gabor-Deep CNN based Masked Face Recognition for Fraud Prevention,” in Proceedings - 5th International Conference on Computing Methodologies and Communication, ICCMC 2021, Apr. 2021, pp. 990–995, doi: 10.1109/ICCMC51019.2021.9418044.

S. Liu and S. S. Agaian, “COVID-19 face mask detection in a crowd using multi-model based on YOLOv3 and hand-crafted features,” no. April 2021, p. 20, 2021, doi: 10.1117/12.2586984.

G. N. Priya and R. S. D. W. Banu, “Detection of Occluded Face Image using Mean Based Weight Matrix and Support Vector Machine,” J. Comput. Sci., vol. 8, no. 7, pp. 1184–1190, 2012, doi: 10.3844/jcssp.2012.1184.1190.

N. Alyuz, B. Gokberk, and L. Akarun, “3-D face recognition under occlusion using masked projection,” IEEE Trans. Inf. Forensics Secur., vol. 8, no. 5, pp. 789–802, 2013, doi: 10.1109/TIFS.2013.2256130.

X. Guo, “A KNN Classifier for Face Recognition,” 2021 IEEE 3rd Int. Conf. Commun. Inf. Syst. Comput. Eng. CISCE 2021, no. Cisce, pp. 292–297, 2021, doi: 10.1109/CISCE52179.2021.9445908.

H. N. Vu, M. H. Nguyen, and C. Pham, “Masked face recognition with convolutional neural networks and local binary patterns,” Appl. Intell., 2021, doi: 10.1007/s10489-021-02728-1.

Z. Zhu et al., “Masked Face Recognition Challenge: The WebFace260M Track Report,” 2021, [Online]. Available: http://arxiv.org/abs/2108.07189.

G. Yang et al., “Garbage Classification System with YOLOV5 Based on Image Recognition,” 2021 6th Int. Conf. Signal Image Process. ICSIP 2021, vol. 1, no. January 2020, pp. 11–18, 2021, doi: 10.1109/ICSIP52628.2021.9688725.

K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 2, pp. 386–397, 2020, doi: 10.1109/TPAMI.2018.2844175.

Q. Zhang, X. Chang, and S. B. Bian, “Vehicle-Damage-Detection Segmentation Algorithm Based on Improved Mask RCNN,” IEEE Access, vol. 8, pp. 6997–7004, 2020, doi: 10.1109/ACCESS.2020.2964055.

D. Mallick Satya, “Custom Object Detection Training using YOLOv5.” https://learnopencv.com/custom-object-detection-training-using-yolov5/ (accessed Jan. 30, 2023).

“Roboflow.” https://docs.roboflow.com/ (accessed Feb. 07, 2023).