Automated diagnosis
Neural networks
Machine learning techniques, and neural networks in particular, for diagnosing melanoma have been in focus of research for more than two decades in dermatology[1][2]. A systematic review as early as 2009 has shown that automated classifiers can have a comparable diagnostic odds ratio to physicians[3]. Systems reviewed in this case commonly relied on manual or semi-automatic feature extraction of digital images[4]. Unprecedented image classification accuracy through so-called deep convolutional neural networks (CNN), which don't need a feature extraction step, has regained interest in automated analyses in the medical domain.
Comparison to human accuracy
Several studies found human level performance of such state-of-the-art CNNs to dermatologists for melanoma diagnosis[5][6][7][8][9] or multi-class accuracy[6]. Similar rates were found for classification of close-up images[5][10][11][10][12].
A more current endeavour was comparing multi-class accuracy of the ISIC 2018 challenge winners to human readers in a world wide and open online game. This study [13] showed for the first time that even average machine learning algorithms can show a higher accuracy in a multi-class setting encompassing all relevant pigmented lesion types.
Datasets
CNNs need a large number of images during training, are thus highly dependant on datasets providing those - and are currently limited to detecting diagnoses covered in them. At the moment the largest publicly available datasets are:
- HAM10000[14], available for download at the Harvard Dataverse and ISIC-Archive
- EDRA Interactive Atlas of Dermoscopy
- PH2 Database
The future central hub for any public dataset is probably the ISIC-Archive, where already datasets from Memorial Sloan Kettering Cancer Center, the SONIC study[15] and others are available in a structured way.
Problems
Although published accuracy rates for neural network classifiers - similar to studies >10 years ago - seem promising, researchers are raising concerns whether neural network classification can live up to its current promise to "solve" automated melanoma recognition[16][17]. This is due to many reasons, some exemplarily shown below:
- Instability: Current neural networks are highly sensitive to small image perturbations invisible to the human eye, shown by adversarial attacks against medical images. But also more evident differences in images can have an impact: E.g. if only melanoma images in the training dataset are photographed with skin markings or rulers, a network may also "diagnose" images of angiomas as melanomas if depicted alongside rulers and skin markings[17].
- Explainability: Although every parameter of a neural network is known, explainability of those systems is impeded by the vast number of parameters. One approach for intuitively explaining neural network decisions is by simply providing similar images with "content based image retrieval" (CBIR). A free non-commercial example application can be used at ypsono.com.
- Evaluation: Classic evaluation metrics of studies and machine learning challenges show comparability to human accuracy, but it is debateable if they provide valuable insight for performance and clinical applicability. Also, a recent Meta-Analysis has shown significant bias of metrics connected to known test datasets[18]
- Real world application: While recent studies show users accept automated classifiers in principle[19], more in depth early studies showed that only non-experts, and only in ambiguous cases, would change their decision based on an automated system[20]. It is completely unclear whether published metrics accurately reflect this population and cases, and whether automated decisions in those cases can provide beneficial and safe outcomes. An underestimated danger is also that an automated system can of course not diagnose a melanoma that is not being photographed at all, which was in fact the case in a prospective clinica trial in Vienna[21].
Device-based automated diagnostics
Apart from image analysis and classification of dermatoscopic images, several other techniques try to solve melanoma diagnosis (list non-exhaustive):
- Dermatofluoroscopy[22]
- Multispectral image analysis[23]
- Raman spectroscopy [24]
- Electrical Impedance Spectroscopy[25]
- Genomic analysis through adhesive tape[26]
They all try to provide a binary output with little to no user interaction. Albeit accuracy rates have been reported high for some of those, it is still unclear if and how they will provide meaningful input in clinical practice.
- References
- ↑ Binder et al.: Application of an artificial neural network in epiluminescence microscopy pattern analysis of pigmented skin lesions: a pilot study. Br. J. Dermatol. 1994;130:460-5. PMID: 8186110.
- ↑ Menzies et al.: The performance of SolarScan: an automated dermoscopy image analysis instrument for the diagnosis of primary melanoma. Arch Dermatol 2005;141:1388-96. PMID: 16301386. DOI.
- ↑ Rajpara et al.: Systematic review of dermoscopy and digital dermoscopy/ artificial intelligence for the diagnosis of melanoma. Br. J. Dermatol. 2009;161:591-604. PMID: 19302072. DOI.
- ↑ Rubegni et al.: Automated diagnosis of pigmented skin lesions. Int. J. Cancer 2002;101:576-80. PMID: 12237900. DOI.
- ↑ 5.0 5.1 Esteva et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017;542:115-118. PMID: 28117445. DOI.
- ↑ 6.0 6.1 Tschandl et al.: A pretrained neural network shows similar diagnostic accuracy to medical students in categorizing dermatoscopic images after comparable training conditions. Br. J. Dermatol. 2017;177:867-869. PMID: 28569993. DOI.
- ↑ Marchetti et al.: Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images. J. Am. Acad. Dermatol. 2018;78:270-277.e1. PMID: 28969863. DOI.
- ↑ Haenssle et al.: Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann. Oncol. 2018;. PMID: 29846502. DOI.
- ↑ Yu et al.: Acral melanoma detection using a convolutional neural network for dermoscopy images. PLoS ONE 2018;13:e0193321. PMID: 29513718. DOI.
- ↑ 10.0 10.1 Han et al.: Classification of the Clinical Images for Benign and Malignant Cutaneous Tumors Using a Deep Learning Algorithm. J. Invest. Dermatol. 2018;. PMID: 29428356. DOI.
- ↑ Fujisawa et al.: Deep learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumor diagnosis. Br. J. Dermatol. 2018;. PMID: 29901853. DOI.
- ↑ Tschandl et al.: Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks. JAMA Dermatol 2019;155:58-65. PMID: 30484822. DOI.
- ↑ Tschandl et al.: Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study. Lancet Oncol. 2019;20:938-947. PMID: 31201137. DOI.
- ↑ Tschandl et al.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 2018;5:180161. PMID: 30106392. DOI.
- ↑ Scope et al.: Dermoscopic patterns and subclinical melanocytic nests in normal-appearing skin. Br. J. Dermatol. 2009;160:1318-21. PMID: 19416274. DOI.
- ↑ Marghoob et al.: Comment on: Screening for malignant melanoma-a critical assessment in historical perspective. Dermatol Pract Concept 2018;8:73-74. PMID: 29785321. DOI.
- ↑ 17.0 17.1 Navarrete-Dechent et al.: Automated Dermatological Diagnosis: Hype or Reality?. J. Invest. Dermatol. 2018;. PMID: 29864435. DOI.
- ↑ Dick et al.: Accuracy of Computer-Aided Diagnosis of Melanoma: A Meta-analysis. JAMA Dermatol 2019;. PMID: 31215969. DOI.
- ↑ Fink et al.: Patient acceptance and trust in automated computer-assisted diagnosis of melanoma with dermatofluoroscopy. J Dtsch Dermatol Ges 2018;. PMID: 29927518. DOI.
- ↑ Dreiseitl et al.: Applying a decision support system in clinical practice: results from melanoma diagnosis. AMIA Annu Symp Proc 2007;191-5. PMID: 18693824.
- ↑ Dreiseitl et al.: Computer versus human diagnosis of melanoma: evaluation of the feasibility of an automated diagnostic system in a prospective clinical trial. Melanoma Res. 2009;19:180-4. PMID: 19369900. DOI.
- ↑ Forschner et al.: Diagnostic accuracy of dermatofluoroscopy in cutaneous melanoma detection: results of a prospective multicentre clinical study in 476 pigmented lesions. Br. J. Dermatol. 2018;. PMID: 29569229. DOI.
- ↑ Hauschild et al.: To excise or not: impact of MelaFind on German dermatologists' decisions to biopsy atypical lesions. J Dtsch Dermatol Ges 2014;12:606-14. PMID: 24944011. DOI.
- ↑ Lui et al.: Real-time Raman spectroscopy for in vivo skin cancer diagnosis. Cancer Res. 2012;72:2491-500. PMID: 22434431. DOI.
- ↑ Malvehy et al.: Clinical performance of the Nevisense system in cutaneous melanoma detection: an international, multicentre, prospective and blinded clinical trial on efficacy and safety. Br. J. Dermatol. 2014;171:1099-107. PMID: 24841846. DOI.
- ↑ Wachsman et al.: Noninvasive genomic detection of melanoma. Br. J. Dermatol. 2011;164:797-806. PMID: 21294715. DOI.
