Computers, Materials & Continua
DOI:10.32604/cmc.2021.015249
images
Article

3D Semantic Deep Learning Networks for Leukemia Detection

Javaria Amin1, Muhammad Sharif2, Muhammad Almas Anjum3, Ayesha Siddiqa1, Seifedine Kadry4, Yunyoung Nam*,5 and Mudassar Raza2

1University of Wah, Wah Cantt, Pakistan
2COMSATS University Islamabad, Wah Campus, Pakistan
3National University of Technology (NUTECH), IJP Road Islamabad, Pakistan
4Faculty of Applied Computing and Technology, Noroff University College, Kristiansand, Norway
5Department of Computer Science and Engineering, Soonchunhyang University, Asan, 31538, Korea
*Corresponding Author: Yunyoung Nam. Email: ynam@sch.ac.kr
Received: 12 November 2020; Accepted: 13 February 2021

Abstract: White blood cells (WBCs) are a vital part of the immune system that protect the body from different types of bacteria and viruses. Abnormal cell growth destroys the body’s immune system, and computerized methods play a vital role in detecting abnormalities at the initial stage. In this research, a deep learning technique is proposed for the detection of leukemia. The proposed methodology consists of three phases. Phase I uses an open neural network exchange (ONNX) and YOLOv2 to localize WBCs. The localized images are passed to Phase II, in which 3D-segmentation is performed using deeplabv3 as a base network of the pre-trained Xception model. The segmented images are used in Phase III, in which features are extracted using the darknet-53 model and optimized using Bhattacharyya separately criteria to classify WBCs. The proposed methodology is validated on three publically available benchmark datasets, namely ALL-IDB1, ALL-IDB2, and LISC, in terms of different metrics, such as precision, accuracy, sensitivity, and dice scores. The results of the proposed method are comparable to those of recent existing methodologies, thus proving its effectiveness.

Keywords: YOLOv2; darknet53; Bhattacharyya separately criteria; ONNX

1  Introduction

Blood is a fluid that transports oxygen, providing energy to body cells that then produce carbon dioxide. It also plays a pivotal role in the immune system; blood circulating in living organisms contains 55% plasma, 40% red cells, 4% platelets, and 1% white blood cells (WBCs) [1]. The five primary types of WBCs are acidophilus, lymphocytes, monocytes, basophils, and neutrophils. These blood cells contain nuclei that differ from those of other cells [2]. WBC abnormalities are diagnosed by a blood smear test. Peripheral blood analysis is utilized for detection of diseases, such as malaria, leukemia, and anemia [3,4]. Such disorders are revealed by an increase and decrease the number of WBCs in the human body. Variations occur in the morphological structure of blood cells in terms of color, shape, and size, and such variations aid in the diagnosis of abnormalities in the WBCs [5]. Thus, segmentation and classification methods are used for the detection of WBCs. The manual evaluation of WBCs is laborious and time consuming [6], and computerized methods are a useful alternative that also minimize the workload of hematologists [7]. Segmentation and classification of WBCs are performed using conventional and deep learning methodologies. In conventional approaches, features are extracted manually; however, in deep learning, images features are learned automatically through a pipeline to improve efficiency [8]. In this study, an automated approach based on deep learning is proposed to segment and classify WBCs more accurately. The foremost contributions of the proposed work are as follows:

•   The Open Neural Network Exchange (ONNX) is applied with a YOLOv2 model, which detects the different types of WBCs. The features are extracted using activation-5 of the ONNX model. The extracted features are fed to the YOLOv2 model. The proposed framework accurately detects the region of interest (ROI).

•   The features are extracted using darknet-53, and the prominent features are selected based on Bhattacharyya separately criteria and fed to the shallow classifiers for the classification of WBCs.

2  Existing Literature

In the literature, significant work has been done for the detection of WBCs, and some of the recent works are discussed in this section [9,10]. The detection of WBCs comprises four primary steps: pre-processing, localization/segmentation, extracting discriminant features, and classification. Pre-processing is a crucial step that is performed for noise removal and eradicating unwanted distortion to enhance the lesion region used in the subsequent segmentation step [11]. Segmentation is another vital step; it is used to group the homogeneous pixels and segment the required region from the input images. WBC cells are difficult to segment because of variations in their appearance [12]. Traditionally, WBCs were detected manually by pathologists, which is time-consuming and can be inaccurate [13]. Recently, automated approaches have been used for the detection of WBCs. Unsupervised clustering methods [14], thresholding approaches [15], shape-based approaches [16], and saliency-based models [17] are commonly used to localize WBCs. Watershed and histogram orientation approaches are used for the segmentation of WBCs. A large amount of data is presented into a set of vectors in the feature extraction process [18]. Selection of the optimum diagnostic features is an important task for the detection of WBCs [19]. Several types of features with different classifiers were used to differentiate the types of WBCs [20]. Supervised methods, such as SVM, Bayesian, random forest [21], and Bayesian [22], are used for the classification of WBCs. However, even the best feature extraction and selection methods struggle with accurate classification” or something similar [23]. Deep learning (DL) approaches are used widely to extract high-level information automatically [24] for the detection of ROIs, such as in WBC detection and classification [25]. Contour aware neural networks are used to segment the WBCs. Pixel by pixel classification is performed using a fully convolutional neural network (FCN) [26]. Mask R-CNN exhibits better classification as compared with other DL techniques [27].

3  Proposed Methodology

The proposed approach comprises localization, segmentation, high-level feature extraction/selection, and classification steps for the analysis of WBCs. In the proposed approach, WBCs are detected/localized using ONNX as the backbone of YOLOv2. The localized cells are segmented using the proposed 3-D semantic segmentation model. Finally, the WBCs are classified using multi-SVM. An overview of the proposed method is presented in Fig. 1.

images

Figure 1: Proposed method architecture for WBCs localization and the segmentation

3.1 Localization of the WBCs

In this research, WBCs are recognized by the suggested WBC-ONNX-YOLOv2 model, as shown in Fig. 2, where features are extricated from activation-5 LeakyReLU of the ONNX model. The extracted features are further fed to the YOLOv2 architecture. The proposed model has 26 layers in the ONNX model, namely 1 input, 6 Conv, 6 Bn, 6 activation, 2 elementwise-affine, and 5 max-pooling layers, and 9 YOLOv2 layers, namely 2 ReLU, 2 Bn, 2 Conv, 1 classification, 1 transform, and 1 output layer.

images

Figure 2: ONNX-YOLOv2 for multi-class detection

The layer-wise proposed model architecture is presented in Tab. 1.

Table 1: The layered architecture of the proposed localization model

images

The proposed model is trained using selected parameters as reported in Tab. 2.

Table 2: Proposed localization model training parameters

images

It is trained on 100 epochs, because after 100 epochs, the model performance is almost stable. The number of iterations with the respective loss during training is illustrated graphically in Fig. 3.

images

Figure 3: Number of the iteration with respect to training loss

3.2 3D-Segmentation of the Leukocytes

The semantic segmentation model is proposed for the segmentation of WBCs, in which deeplabv3 is used as a bottleneck in the Xception model. The pre-trained Xception model contains 205 layers, comprising 1 input, 88 2-D Conv, 46 Bn, 46 ReLU, 3 max-pooling, 12 addition, 4 crop 2D, 2 transpose Conv, 2 depth Conv, softmax, and pixel classification layers. The segmentation model was trained from scratch on the blood smear images. The training parameters of the presented model are listed in Tab. 3.

Table 3: Training parameters of the segmentation model

images

The proposed model learning with convolutional layers is plotted with activation units, as presented in Fig. 4.

images

Figure 4: Segmentation model with activation units

3.3 Deep Features Extraction and Classification

The deep features are extracted using a pre-trained darknet53 model, which contains 184 layers, namely 1 input, 53 Conv, 1 global pooling, 52 Bn, 52 LeakyReLU, and 23 addition layers, and softmax with cross-entropy loss. The features are extracted from Conv53 layers with dimensions of 1×1000. The selection of informative features from a pool of features is difficult. Therefore, the Bhattacharyya rank-based feature selection approach is used, in which the optimum 500 (50%) best features are selected out of 1000 features to improve the classification accuracy, also providing cost-effective and fast predictors. The best-selected features are further supplied to the multi-kernel SVM classifiers, such as Cubic-SVM, Quadratic SVM, O-SVM, and Gaussian SVM to classify the different types of blood cells, as depicted in Fig. 5.

images

Figure 5: Feature extraction & selection and classification process

The SVM classifier with different kernels is trained on the best-selected feature vectors with optimum parameters, as listed in Tab. 4.

Table 4: Parameters of SVM selection

images

Table 5: Localization results of different kinds of WBCs

images

images

Figure 6: Localization results on benchmark datasets (a) log average rate (b) average precision of different types of WBCs (c) IoU (d) average precision of blast cells

4  Experimental Setup

In this research, three publicly available benchmark datasets are used for the method evaluation. ALL-IDB1 contains 107 blood smear images, of which 33 are blasts and 74 are non-blast cells, and ALL-IDB2 contains 260 blood smear images, comprising 130 blasts and 130 non-blast cells [2831]. The LISC dataset contains blood smear images of WBCs, including eosinophils, neutrophils, monocytes, lymphocytes, and basophils. The numbers of images for each type of WBC are not equal. To balance the different types of imaging data of WBCs, data augmentation is performed by rotating the images at different angles, such as 45, 90, 180, and 360. After augmentation, 6250 images of five types of WBCs are obtained, with each type having 1250 blood smear images [32].

4.1 Results & Discussion

The proposed work performance is validated by performing three experiments. The first experiment is performed to validate the presented localization technique by different metrics such as mean precision (mAP) and intersection over the union (IoU). The second experiment is validated to compute the segmentation model performance, while the third experiment is performed to compute the classification model performance. All experiments in this research are performed on the MATLAB 2020 Ra toolbox with 1050 K Nvidia Graphic Card.

4.2 Experiment #1: Localization of Leukocytes

Experiment 1 was performed to validate the performance of the localization approach on three benchmark datasets, LISC, ALL-IDB1, and ALL-IDB2, using IoU and mAP as metrics, as shown in Tab. 5. In this experiment, six types of WBCs were localized, and the localization results are graphically depicted in Fig. 6.

images

Figure 7: Localization results and corresponding confidence score of the proposed method on LISC dataset. Column (a) and (d) represent input images; (b) and (e) localization results; (c) and (f) confidence score

The localization outcomes in Tab. 5 indicate that the method achieved the highest 0.97 IoU on blast cells.

The proposed method localizes the WBCs with confidence scores, as shown in Figs. 7 and 8.

The localization results in Figs. 7 and 8 reveal that the proposed method achieved the highest confidence scores of 0.97349, 0.96849, 0.95933, 0.95867, 0.94616, and 0.89726 for eosinophils, basophils, lymphocytes, monocytes, neutrophil, and blast cells, respectively.

images

Figure 8: Localized region on LISC and ALL-IDB datasets (a) blast images (b) localization (c) confidence scores

4.3 Experiment 2: Segmentation of Leukocytes

In this experiment, the 3D segmented region is validated using different types of performance metrics, namely IoU, mean, weighted, and global accuracy, and F1-scores, as mentioned in Tab. 6. The results of the proposed segmented WBCs are mapped pixel-by-pixel with ground annotated images, as illustrated in Fig. 9.

Table 6: Segmentation results of the WBCs

images

images

Figure 9: 3D-segmentation outcomes (a) WBCs (b) 3D segmentation (c) ground annotated masks

The segmentation results in Tab. 6 indicate that the proposed method achieved the highest segmentation accuracy, obtained by the pixel-by-pixel comparison of the segmented images with ground annotated images.

4.4 Experiment #3: Classification Based on the Extracted Feature

In this experiment, an optimized feature vector is fed to a multi-kernel SVM for WBC classification, and the outcomes are computed in terms of accuracy, precision, recall, and F1 scores from the LISC dataset, as displayed in Tabs. 79. The discrimination outcomes on the LISC and ALL-IDB1&2 datasets with class labels are presented in Fig. 10.

Table 7: WBCs classification results

images

Table 8: WBCs classification results

images

Table 9: WBCs classification results

images

A quantitative analysis is performed using an SVM with three different types of kernels, namely cubic, quadratic, and optimized. The SVM with the optimized kernel achieved a maximum overall accuracy of 98.4%. The classification results are also compared with the latest published work, as shown in Tab. 10.

Tab. 10 compares the classification results with the latest published existing work. The existing work achieved accuracies of 0.995, 0.984, 0.984, 0.961, and 0.950 for lymphocytes, monocytes, basophils, eosinophils, and neutrophils, respectively. In contrast, the proposed method exhibited improved classification accuracy, with 0.996, 1.00, 0.987, 1.00, and 0.988 for lymphocytes, monocytes, basophils, eosinophils, and neutrophils, respectively.

The classification results on the ALL-IDB1&2 datasets are presented in Tabs. 11 and 12.

The classification results of blast/non-blast cells are presented in Tabs. 11 and 12. An accuracy of 99.57% was achieved on the ALL-IDB1 dataset and 98.25% on the ALL-IDB2 dataset, and the results are compared with a recently published work, as provided in Tab. 13.

Tab. 13 presents a comparison of the numerical results, wherein the competitive results obtained from the proposed method are compared to those of the latest published work.

images

Figure 10: Confusion matrix (a) LISC dataset (b) ALL-IDB1 dataset

Table 10: Proposed work comparison with latest published work on LISC dataset

images

Table 11: WBCs classification results

images

Table 12: WBCs classification results

images

Table 13: Results comparison

images

5  Conclusion

In this study, deep learning approaches are proposed for the detection of WBCs. Detecting WBCs is challenging because blood smear images contain different color distributions in the cytoplasm and nucleus regions, making it difficult to segment these regions accurately. A 3-D semantic segmentation model is proposed, in which deeplabv3 is used as a bottleneck and the Xception model is used as a classification head to accurately segment the WBCs. Feature extraction/selection is another challenge for the classification of WBCs. The features are extracted from the pre-trained darknet-53 model, and informative features are selected using Bhattacharyya separability criteria and passed to the SVM with different types of kernels, namely cubic, quadratic, and optimized. The proposed classification method achieved an accuracy of 99.57% on the ALL-IDB1 dataset, 98.25% for the ALL-IDB2 dataset, and 98.4% for LISC datasets using the optimizable SVM kernel. The overall experimental outcomes demonstrate that the proposed technique achieved competitive outcomes by optimizing the SVM kernel. The proposed new framework based on a CNN can be used for the detection of different types of cancer, such as lung and bone cancer. It detects and classifies leukocytes at an early stage, thereby increasing the survival rate of patients.

Acknowledgement: This research was supported by Korea Institute for Advancement of Technology (KIAT).

Funding Statement: This research was supported by Korea Institute for Advancement of Technology (KIAT) grant funded by the Korea Government (MOTIE) (P0012724, The Competency Development Program for Industry Specialist) and the Soonchunhyang University Research Fund.

Conflicts of Interest: All authors declare that they have no conflicts of interest to report regarding the present study.

References

 1.  M. Srivastava, Analysis on Bio-Mathematics. Chhattisgarh, India: Shashwat Publication, 2020. [Online]. Available: https://shashwatpublication.com/books/anasysis-on-bio-mathematics.

 2.  S. Ali, A. Tanveer, A. Hussain and S. U. Rehman, “Identification of cancer disease using image processing approahes,” International Journal of Intelligent Information Systems, vol. 9, no. 2, pp. 1–10, 2020.

 3.  E. Abdulhay, M. A. Mohammed, D. A. Ibrahim, N. Arunkumar and V. Venkatraman, “Computer aided solution for automatic segmenting and measurements of blood leucocytes using static microscope images,” Journal of Medical Systems, vol. 42, no. 4, pp. 1–12, 2018.

 4.  M. A. Mohammed, K. H. Abdulkareem, S. A. Mostafa, M. K. A. Ghani, M. S. Maashi et al., “Voice pathology detection and classification using convolutional neural network model,” Applied Sciences, vol. 10, pp. 1–13, 2020.

 5.  M. Subathra, M. A. Mohammed, M. S. Maashi, B. Garcia-Zapirain, N. Sairamya et al., “Detection of focal and non-focal electroencephalogram signals using fast walsh-hadamard transform and artificial neural network,” Sensors, vol. 20, no. 17, pp. 1–20, 2020.

 6.  M. K. Abd Ghani, M. A. Mohammed, N. Arunkumar, S. A. Mostafa, D. A. Ibrahim et al., “Decision-level fusion scheme for nasopharyngeal carcinoma identification using machine learning techniques,” Neural Computing and Applications, vol. 32, no. 3, pp. 625–638, 2020.

 7.  O. I. Obaid, M. A. Mohammed, M. Ghani, A. Mostafa and F. Taha, “Evaluating the performance of machine learning techniques in the classification of wisconsin breast cancer,” International Journal of Engineering & Technology, vol. 7, pp. 160–166, 2018.

 8.  N. Arunkumar, M. A. Mohammed, M. K. Abd Ghani, D. A. Ibrahim, E. Abdulhay et al., “K-means clustering and neural network for object detecting and identifying abnormality of brain tumor,” Soft Computing, vol. 23, no. 19, pp. 9083–9096, 2019.

 9.  H. Eilertsen, P. C. Sæther, C. E. Henriksson, A. S. Petersen and T. A. Hagve, “Evaluation of the detection of blasts by sysmex hematology instruments, cellavision DM96, and manual microscopy using flow cytometry as the confirmatory method,” International Journal of Laboratory Hematology, vol. 41, no. 3, pp. 338–344, 2019.

10. H. H. Inbarani and A. T. Azar, “Leukemia image segmentation using a hybrid histogram-based soft covering rough k-means clustering algorithm,” Electronics, vol. 9, no. 1, pp. 1–22, 2020.

11. J. Bai, F. Lu and K. Zhang, “ONNX: Open neural network exchange,” GitHub Repository, 2019. [Online]. Available: https://github.com/onnx/onnx.

12. J. Redmon and A. Farhadi, “YOLO9000: Better, faster, stronger,” in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, Hawaii, pp. 7263–7271, 2017.

13. L. Chen, Y. Zhu, G. Papandreou, F. Schroff and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in Proc. of the European Conf. on Computer Vision, Glasgow, United Kingdom, pp. 801–818, 2018.

14. F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Montreal, Canada, pp. 1251–1258, 2017.

15. J. Redmon, “Darknet: Open source neural networks in C,” 2016. [Online]. Available: https://pjreddie.com/darknet.

16. S. Theodoridis and K. Koutroumbas, “Pattern recognition,” IEEE Transactions on Neural Networks, vol. 19, no. 1, pp. 1–957, 2008.

17. S. Kollem, K. R. Reddy and D. S. Rao, “A review of image denoising and segmentation methods based on medical images,” International Journal of Machine Learning and Computing, vol. 9, no. 3, pp. 288–295, 2019.

18. K. Al-Dulaimi, J. Banks, K. Nguyen, A. Al-Sabaawi, I. Tomeo-Reyes et al., “Segmentation of white blood cell, nucleus and cytoplasm in digital haematology microscope images: A review-challenges, current and future potential techniques,” IEEE Reviews in Biomedical Engineering, vol. 14, pp. 1–16, 2020.

19. M. Dutta, S. Karmakar, P. Banerjee and R. Ghatak, “Detection of leukemia in blood samples applying image processing using a novel edge detection method,” in Proc. of the Global AI Congress, Singapore, pp. 1–16, 2019.

20. M. Nassar, M. Doan, A. Filby, O. Wolkenhauer, D. K. Fogg et al., “Label-free identification of white blood cells using machine learning,” Cytometry Part A, vol. 95, no. 8, pp. 836–842, 2019.

21. M. Khodashenas, H. Ebrahimpour-komleh and A. Nickfarjam, “White blood cell detection and counting based on genetic algorithm,” in Advances in Science and Engineering Technology International Conf., Dubai, United Arab Emirates, pp. 1–4, 2019.

22. A. Şengür, Y. Akbulut, Ü. Budak and Z. Cömert, “White blood cell classification based on shape and deep features,” in Int. Artificial Intelligence and Data Processing Symp., Malatya, Turkey, pp. 1–4, 2019.

23. J. Nalepa and M. Kawulok, “Selecting training sets for support vector machines: A review,” Artificial Intelligence Review, vol. 52, no. 2, pp. 857–900, 2019.

24. E. Hussain, L. B. Mahanta, C. R. Das, M. Choudhury and M. Chowdhury, “A shape context fully convolutional neural network for segmentation and classification of cervical nuclei in pap smear images,” Artificial Intelligence in Medicine, vol. 107, pp. 1–11, 2020.

25. H. Kutlu, E. Avci and F. Özyurt, “White blood cells detection and classification based on regional convolutional neural networks,” Medical Hypotheses, vol. 135, no. 10, pp. 1–11, 2020.

26. V. Shankar, M. M. Deshpande, N. Chaitra and S. Aditi, “Automatic detection of acute lymphoblasitc leukemia using image processing,” in IEEE Int. Conf. on Advances in Computer Applications, Coimbatore, India, pp. 186–189, 2016.

27. N. Dhieb, H. Ghazzai, H. Besbes and Y. Massoud, “An automated blood cells counting and classification framework using mask R-CNN deep learning model,” in 31st Int. Conf. on Microelectronics, Cairo, Egypt, pp. 300–303, 2019.

28. R. D. Labati, V. Piuri and F. Scotti, “All-IDB: The acute lymphoblastic leukemia image database for image processing,” in 18th IEEE Int. Conf. on Image Processing, Brussels, Belgium, pp. 2045–2048, 2011.

29. F. Scotti, “Robust segmentation and measurements techniques of white cells in blood microscope images,” in 2006 IEEE Instrumentation and Measurement Technology Conf. Proc., Sorrento, Italy, pp. 43–48, 2006.

30. F. Scotti, “Automatic morphological analysis for acute leukemia identification in peripheral blood microscope images,” in CIMSA. IEEE Int. Conf. on Computational Intelligence for Measurement Systems and Applications, Giardini Naxos, Italy, pp. 96–101, 2005.

31. V. Piuri and F. Scotti, “Morphological classification of blood leucocytes by microscope images,” in 2004 IEEE Int. Conf. on Computational Intelligence for Measurement Systems and Applications, Boston, MA, USA, pp. 103–108, 2004.

32. S. H. Rezatofighi, K. Khaksari and H. Soltanian-Zadeh, “Automatic recognition of five types of white blood cells in peripheral blood,” in Int. Conf. Image Analysis and Recognition, Berlin, Heidelberg, pp. 161–172, 2010.

33. L. C. de Faria, L. F. Rodrigues and J. F. Mari, “Cell classification using handcrafted features and bag of visual words,” in Anais do XIV Workshop de Visão Computacional, Brasil, pp. 1–6, 2018.

34. S. Shafique and S. Tehsin, “Acute lymphoblastic leukemia detection and classification of its subtypes using pretrained deep convolutional neural networks,” Technology in Cancer Research & Treatment, vol. 17, pp. 1–7, 2018.

35. Z. F. Mohammed and A. A. Abdulla, “An efficient CAD system for all cell identification from microscopic blood images,” Multimedia Tools and Applications, vol. 80, pp. 6355–6368, 2020.

36. L. H. Vogado, R. M. Veras and K. R. Aires, “LeukNet-A model of convolutional neural network for the diagnosis of leukemia,” in Anais Estendidos do XXXIII Conf. on Graphics, Patterns and Images, Bairro Agronomia, Porto Alegre, pp. 119–125, 2020.

37. C. Di Ruberto, A. Loddo and G. Puglisi, “Blob detection and deep learning for leukemic blood image analysis,” Applied Sciences, vol. 10, pp. 1–13, 2020.

images This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.