ISIC project
The International Skin Imaging Collaboration (ISIC) is a combined academia and industry effort aimed at improving melanoma diagnoses and reducing melanoma mortality by facilitating the application of digital skin imaging technologies.
Standardization of Dermoscopy
Sponsored by the International Society for Digital Imaging of the Skin (ISDIS), ISIC working groups are developing proposed standards to address the technologies, techniques, and terminology used in skin imaging with special attention to the issues of privacy and interoperability (i.e., the ability to share images across technology and clinical platforms). Some of the articles published by the groups include Standardization of terminology in dermoscopy/dermatoscopy [1] in 2016 and Technique Standards for Skin Lesion Imaging [2] in 2017.
ISIC archive
In addition, ISIC has developed and is expanding a public archive containing the largest publicly available collection of quality controlled dermoscopic images of skin lesions. Presently, the ISIC Archive contains over 23,500 dermoscopic images, which were collected from leading clinical centers across the globe and acquired from a variety of devices within each center. Broad international participation in image contribution is intended to insure a representative, clinically relevant sample.
All incoming images to the ISIC Archive are screened for both privacy and quality assurance. Most images have associated clinical metadata, which has been vetted by recognized melanoma experts.
A subset of the images has undergone annotation and markup by recognized skin cancer experts. These markups include dermoscopic features (i.e., global and focal morphologic elements in the image known to discriminate between types of skin lesions).
All images on ISIC are available for everyone to access and to use for teaching purposes. In addition, ISIC is now linked to dermoscopedia. Images from dermoscopedia are uploaded to ISIC, and images from ISIC are available for use in dermoscopedia website.
The software infrastructure of the ISIC archive is based on the open-source Girder platform, and the source code for the Archive itself is freely available on GitHub.
Machine Learning Challenges
Since 2016, the ISIC Project has conducted an annual challenge for developers of artificial intelligence (AI) algorithms in the diagnosis of melanoma. In the first step of each challenge, a ‘training set’ of ISIC images matched to their diagnosis is used to train the algorithms. In the second stage, a ‘test set’ of ISIC images is used to evaluate the algorithms’ diagnostic accuracy and to compare the result with dermatologists’ diagnostic accuracy. Each year the challenges include more participants, more images, and more lesion types – While the 2016 challenge included only melanomas and pigmented nevi, the 2017 included melanomas, pigmented nevi, and seborrheic keratoses, and the 2018 challenge included 8 different lesion types.
The 2016 challenge included 900 images in the ‘training’ set and 350 images in the ‘test’ set. The dermoscopic features that were examined were only globules and streaks. The average sensitivity and specificity of dermatologists in classifying pigmented lesions was 82 % and 59 %, respectively. At 82 % sensitivity, dermatologist specificity was similar to the top individual algorithm (59 % vs. 62 %, P = .68) but lower than the best-performing fusion algorithm (a fusion of 16 automated predictions from the 25 participating teams, including both non-learned and machine-learning methods) that had a specificity of 76 % (P = .02) [3].
The 2017 challenge included 2000 images in the ‘training’ set and 600 images in the ‘test’ set. The dermoscopic features that were examined were pigment network, negative network, streaks and mili-like cysts. The specificity achieved by the top algorithm for melanoma diagnosis at a sensitivity level of 82% was 74.7%. Results of the dermatologists’ sensitivity and specificity will be published soon.
The 2018 challenge includes 2594 images in the ‘training’ set and 1000 images in the ‘test’ set. It is still ongoing and will hopefully include several hundred participating dermatologists who will ‘compete’ with AI algorithms in accurately diagnosing pigmented skin lesions.
- ↑ Kittler et al.: Standardization of terminology in dermoscopy/dermatoscopy: Results of the third consensus conference of the International Society of Dermoscopy. J. Am. Acad. Dermatol. 2016;74:1093-106. PMID: 26896294. DOI.
- ↑ Katragadda et al.: Technique Standards for Skin Lesion Imaging: A Delphi Consensus Statement. JAMA Dermatol 2016;. PMID: 27892996. DOI.
- ↑ Marchetti et al.: Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images. J. Am. Acad. Dermatol. 2018;78:270-277.e1. PMID: 28969863. DOI.
