Artificial intelligence–assisted classification of gliomas using whole slide images

Authors

• Laya Jose, Computational NeuroSurgery (CNS) Lab, Macquarie University, Sydney, AustraliaFollow
• Sidong Liu, Australian Institute of Health Innovation, Centre for Health Informatics (Liu), Macquarie University, Sydney, Australia
• Carlo Russo, Computational NeuroSurgery (CNS) Lab, Macquarie University, Sydney, Australia
• Cong Cong, Macquarie University, Sydney, Australia; and the School of Computer Science and Engineering, University of New South Wales, Sydney, Australia
• Yang Song, Macquarie University, Sydney, Australia; and the School of Computer Science and Engineering, University of New South Wales, Sydney, Australia
• Michael Rodriguez, Macquarie Medical School, Macquarie University, Sydney, Australia
• Antonio Di Ieva, Computational NeuroSurgery (CNS) Lab, Macquarie University, Sydney, Australia Edit author details Remove author

Published

29-11-2022

Type of Work

Article - Journal

Abstract

Context.—Glioma is the most common primary brain tumor in adults. The diagnosis and grading of different pathological subtypes of glioma is essential in treatment planning and prognosis.

Objective.—To propose a deep learning–based approach for the automated classification of glioma histopathology images. Two classification methods, the ensemble method based on 2 binary classifiers and the multiclass method using a single multiclass classifier, were implemented to classify glioma images into astrocytoma, oligodendroglioma, and glioblastoma, according to the 5th edition of the World Health Organization classification of central nervous system tumors, published in 2021.

Design.—We tested 2 different deep neural network architectures (VGG19 and ResNet50) and extensively validated the proposed approach based on The Cancer Genome Atlas data set (n = 700). We also studied the effects of stain normalization and data augmentation on the glioma classification task.

Results.—With the binary classifiers, our model could distinguish astrocytoma and oligodendroglioma (combined) from glioblastoma with an accuracy of 0.917 (area under the curve [AUC] = 0.976) and astrocytoma from oligodendroglioma (accuracy = 0.821, AUC = 0.865). The multiclass method (accuracy = 0.861, AUC = 0.961) outperformed the ensemble method (accuracy = 0.847, AUC = 0.933) with the best performance displayed by the ResNet50 architecture.

Conclusions.—With the high performance of our model (>80%), the proposed method can assist pathologists and physicians to support examination and differential diagnosis of glioma histopathology images, with the aim to expedite personalized medical care.

References

1. Louis DN, Perry A, Wesseling P, et al. The 2021 WHO classification of tumors of the central nervous system: a summary. Neuro Oncol. 2021;23(8): 1231–1251.

2. Perus LJ, Walsh LA. Microenvironmental heterogeneity in brain malignancies. Front Immunol. 2019;10:2294.

3. van den Bent MJ. Interobserver variation of the histopathologic diagnosis in clinical trials on glioma: a clinician’s perspective. Acta Neuropathol. 2010; 120(3):297–304.

4. Alom MZ, Yakopcic C, Nasrin MS, Taha TM, Asari VK. Breast cancer classification from histopathologic images with inception recurrent residual convolutional neural network. J Digit Imaging. 2019;32(4):605–617.

5. Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med. 2018;24(10):1559–1567.

6. Jin L, Shi F, Chun Q, et al. Artificial intelligence neuropathologist for glioma classification using deep learning on hematoxylin and eosin stained slide images and molecular markers. Neuro Oncol. 2021;23(1):44–52.

7. Ker J, Bai Y, Lee HY, Rao J,Wang L. Automated brain histology classification using machine learning. J Clin Neurosci. 2019;66:239–245.

8. Rathore S, Niazi T, Iftikhar MA, Chaddad A. Glioma grading via analysis of digital pathology images using machine learning. Cancers. 2020;12(3):578.

9. Mousavi HS, Monga V, Rao G, Rao AU. Automated discrimination of lower and higher grade gliomas based on histopathologic image analysis. J Pathol Inform. 2015;6:15.

10. National Cancer Institute. Genomic Data Commons Web site. https:// portal.gdc.cancer.gov/. Accessed April 28, 2019.

11. Barker J, Hoogi A, Depeursinge A, Rubin DL. Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Med Image Anal. 2016;30:60–71.

12. Wang X, Wang D, Yao Z, et al. Machine learning models for multiparametric glioma grading with quantitative result interpretations. Front Neurosci. 2019;12:1046.

13. Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge, MA: MIT Press; 2016.

14. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. 2012;25:1097–1105.

15. Dabeer S, Khan MM, Islam S. Cancer diagnosis in histopathologic image: CNN based approach. Inform Med Unlocked. 2019;16:100231.

16. Yue X, Dimitriou N, Arandjelovic O. Colorectal cancer outcome prediction from H&E whole slide images using machine learning and automatically inferred phenotype profiles [published online February 10, 2019]. arXiv.2019;190203582. doi:10.48550/ARXIV.1902.03582.

17. Ertosun MG, Rubin DL. Automated grading of gliomas using deep learning in digital pathology images: a modular approach with ensemble of convolutional neural networks. AMIA Annu Symp Proc. 2015;2015:1899–1908.

18. Pei L, Jones KA, Shboul ZA, Chen JY, Iftekharuddin KM. Deep neural network analysis of pathology images with integrated molecular data for enhanced glioma classification and grading. Front Oncol. 2021;11:2572.

19. Truong AH, Sharmanska V, Limb¨ack-Stanic C, Grech-Sollars M. Optimization of deep learning methods for visualization of tumor heterogeneity and brain tumor grading through digital pathology. Neurooncol Adv. 2020;2(1): vdaa110.

20. Louis DN, Perry A, Reifenberger G, et al. The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 2016;131(6):803–820.

21. Louis DN, Aldape K, Brat DJ, et al. Announcing cIMPACT-NOW: the consortium to inform molecular and practical approaches to CNS tumor taxonomy. Acta Neuropathol; 2017;133(1):1–3.

22. Takahashi R, Matsubara T, Uehara K. Data augmentation using random image cropping and patching for deep CNNs. IEEE Transact Circuits Syst Video Technol. 2019;30(9):2917–2931.

23. Liu S, Shah Z, Sav A, et al. Isocitrate dehydrogenase (IDH) status prediction in histopathology images of gliomas using deep learning. Sci Rep. 2020;10(1): 7733.

24. Zhuang F, Qi Z, Duan K, et al. A comprehensive survey on transfer learning. Proc IEEE. 2020;109(1):43–76.

25. Simonyan K, Zisserman A. Very deep convolutional networks for largescale image recognition [published online September 4, 2014]. arXiv. 2014; 14091556. doi:10.48550/ARXIV.1409.1556.

26. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proc IEEE Conference Comput Vis Pattern Recog. 2016:770–778. 

27. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. Proc IEEE Conference Comput Vis Pattern Recog. 2016:2818–2826.

28. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA. Inception-v4, inception-resnet and the impact of residual connections on learning. Paper presented at Thirty-first AAAI conference on Artificial Intelligence; February 4–9, 2017; San Francisco, CA.

29. Russakovsky O, Deng J, Su H, et al. ImageNet large scale visual recognition challenge. IntJ Comput Vis. 2015;115(3):211–252.

30. Zinkevich M, Weimer M, Li L, Smola A. Parallelized stochastic gradient descent. Proc 23rd Int Conf Neural Inf Processing Syst. 2010:2595–2603.

31. Kingma DP, Ba J. Adam: A method for stochastic optimization [published online December 22, 2014]. arXiv. 1412.6980. doi:0.48550/ARXIV.1412.6980.

32. Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data. 2019;6(1):1–48.

33. Chollet F. Deep Learning With Python. Vol. 361. New York: Manning; 2018.

34. Macenko M, Niethammer M, Marron JS, et al. A method for normalizing histology slides for quantitative analysis. IEEE Int Symp Biomed Imaging: Nano Macro. 2009:1107–1110.

35. Provost F, Kohavi R. Glossary of terms. J Machine Learning. 1998;30(2–3): 271–274.

36. Gosain A, Sardana S. Handling class imbalance problem using oversampling techniques: a review. Int Conf Adv Comput Commun Inform (ICACCI). 2017:79–85.

37. Jose L, Liu S, Russo C, Nadort A, Di Ieva A. Generative adversarial networks in digital pathology and histopathologic image processing: a review. J Pathol Inform. 2021;12.

38. Berstad TJD, Riegler M, Espeland H, et al. Tradeoffs using binary and multiclass neural network classification for medical multidisease detection. IEEE Int Symp Multimedia (ISM). 2018:1–8.

39. D’Alessio A, Proietti G, Sica G, Scicchitano BM. Pathological and molecular features of glioblastoma and its peritumoral tissue. Cancers. 2019; 11(4):469.

40. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. Adv Neural Inf Processing Syst. 2014;27.

41. Li Q, Yu Z, Wang Y, Zheng H. TumorGAN: A multi-modal data augmentation framework for brain tumor segmentation. Sensors. 2020;20(15): 4203.

42. Park JE, Eun D, Kim HS, Jang RW, Kim N. Generative adversarial network for glioblastoma ensures morphologic variations and improves diagnostic model for isocitrate dehydrogenase mutant type. Sci Rep. 2021;11(1):1–11.

43. Wong WS, Amer M, Maul T, Liao IY, Ahmed A. Conditional generative adversarial networks for data augmentation in breast cancer classification. Int Conf Soft Compu Data Mining 2020:392–402.

Citation

Jose, L., Liu, S., Russo, C., Cong, C., Song, Y., Rodriguez, M., & Di Ieva, A. (2023). Artificial intelligence-assisted classification of gliomas using whole-slide images. Archives of Pathology & Laboratory Medicine, 147(8), 916-924. https://doi.org/10.5858/arpa.2021-0518-OA

Link to Published Work

https://aplm.kglmeridian.com/view/journals/arpa/147/8/article-p916.xml?body=PDF