A Sharper Eye: How Advanced Imaging Technologies and Artificial Intelligence Are Transforming Melanoma Diagnosis

By Kenneth Miller

Over the past decade, death rates from melanoma have fallen by an average of 2.9 percent a year, according to the National Cancer Institute, even as new cases have risen by an average of 1.5 percent each year. This encouraging trend in mortality has been driven, in part, by increases in early detection. Yet there is still substantial room for improvement. The American Cancer Society estimates that 6,850 people in the United States will die of melanoma in 2020. Many of those likely could have been saved if clinicians had detected their tumors sooner.

The problem is not that dermatologists are performing too few biopsies. Such procedures more than doubled between 2000 and 2015 among patients 65 or older (the age group at highest risk for melanoma), from about 2 million to 5 million per year, according to a 2018 study in the Journal of the American Academy of Dermatology. Yet for every additional 1,000 biopsies performed, another study of this cohort found, just 5.2 cases of melanoma in situ and 8.1 cases of invasive melanoma were discovered. An estimated 196,060 cases of melanoma will be diagnosed in the U.S. in 2020, just over half of them invasive.

In short: While thousands of patients fall through the diagnostic cracks each year, millions of others undergo biopsies that could have been avoided with better screening. Although the consequences of an undetected melanoma may be more severe for the individual patient, those of unnecessary biopsies place a significant burden, in financial cost and resource usage, on the health-care system as a whole.

A growing body of evidence suggests that advances in imaging technology, in tandem with dermatologically trained artificial intelligence (AI) programs, can play a key role in addressing persistent shortcomings in melanoma diagnosis. To shed light on these developments, we spoke with two leading experts in the field.

Allan C. Halpern, MD, is chief of dermatology at Memorial Sloan Kettering Cancer Center in New York City. Ashfaq A. Marghoob, MD, is director of clinical dermatology at Memorial Sloan Kettering Skin Cancer Center in Hauppauge, New York. The pair are co-editors of The Melanoma Letter. And both are leaders of the International Skin Imaging Collaboration (ISIC), a global research and development initiative founded by Dr. Halpern.

“Dermatology is an incredibly visual specialty, but it lags behind many others in terms of imaging,” Dr. Halpern notes. “ISIC’s mission is to harness AI and other drivers to move this discipline into the 21st century.”

Beyond the ABCs

To understand how imaging technology could improve diagnostic accuracy for melanoma in the future, it helps to know how far it has brought the field in the past few decades. “Until fairly recently, all we had was our clinical eye,” Dr. Marghoob explains. In the 1970s, when melanoma rates began to soar, clinical diagnosis was based on symptoms (bleeding, itching and ulceration) that arose when a cancer was often too advanced for effective intervention.

This began to change in 1985, when a group of dermatologists at New York University School of Medicine developed and published the first simple algorithm for detecting early melanomas. The ABCD criteria (for asymmetry, border irregularity, color variegation and diameter greater than 6 mm) “helped us find a few more melanomas, but many still escaped detection,” Dr. Marghoob recalls. “Those we did catch were often thick. And even at expert centers, we were removing about 15 benign nevi for every malignancy we found.”

In 1988 came the first description of total-body photography (TBP) as a screening tool, based on the insight that evolving size (the “E” added to ABCD) was another characteristic of malignant skin lesions. By tracking virtually all of a patient’s pigmented lesions over time — including those that initially might not have attracted notice — TBP improved diagnostic sensitivity, enabling clinicians to detect more and thinner melanomas. But because just 3 percent of changed lesions proved to be cancerous, this method brought only modest gains in specificity. For clinicians using TBP, Dr. Marghoob estimates, the ratio of benign to malignant lesions removed for biopsy was 10 to 1.

By the early 1990s, handheld dermatoscopes had become available, enabling clinicians to examine suspicious lesions in far greater detail than with a simple magnifying glass, and more easily than with a full-size stereomicroscope. “With dermoscopy, diagnostic accuracy went very high,” says Dr. Marghoob. “We got better at finding early melanomas that didn’t look like melanoma. We also got better at recognizing when moles that looked like melanoma weren’t cancers at all.” Among expert clinicians, studies showed, benign-to-malignant ratios dropped as low as 5 to 1.

Enhancing Specificity

More recently, advances in digital technology have brought refinements to the methods mentioned above. With total body digital photography (TBDP), dozens of cameras can instantly gather images from almost every inch of a patient’s skin. After creating a mole map (sometimes in 3-D, as is used at Memorial Sloan Kettering Cancer Center), some software programs can automatically alert the clinician to any changes since the last scan, although Dr. Halpern says, “Most clinical use of TBP relies on a clinician comparing the patient’s current examination to a set of photographs taken at a previous point in time.” Dermatoscopes now routinely accommodate a smartphone, which can record images or video, measure lesions precisely and store all this data electronically via a mobile app. Available technology enables digital dermoscopy images to be integrated with TBDP scans and accessed by tapping the area of concern on a touchscreen.

In addition, an array of new techniques (see “Tiebreaking Tech,” below) enables clinicians to analyze pigmented lesions with greater specificity, and thus to perform fewer unnecessary biopsies. “These instruments are useful when a lesion remains ambiguous even with dermoscopy,” Dr. Marghoob explains. “They can act as a tiebreaker when a mole doesn’t have clear features of cancer, but you can’t reassure yourself that it’s benign.”

The best-established of these approaches is reflectance confocal microscopy (RCM), which uses a near-infrared laser to provide cellular resolution of skin and cutaneous structures as deep as the papillary dermis. “At centers that are using total body digital photography, dermoscopy and RCM, the benign-to-malignant ratio is somewhere between 2 and 3 nevi removed for every melanoma,” says Dr. Marghoob.

Other emerging modalities — such as electrical impedance spectroscopy, tape stripping and optical coherence tomography — could eventually push the ratio even lower at such centers. 

Why AI?

Despite all these innovations, the benign-to-malignant ratio among dermatologists who don’t specialize in melanoma remains stubbornly high in the United States — averaging at least 50 to 1 for patients over 65, according to multiple studies. This is largely due to underutilization of even elementary imaging technology. Estimates for dermatoscope use in this country range from 40 to 80 percent, meaning that non-use ranges from 20 to 60 percent. Techniques like TBDP and RCM are found almost exclusively in academic medical centers.

“American dermatologists are at the bottom of the pile when it comes to imaging,” says Dr. Halpern. “Our group at Memorial Sloan Kettering is one of the few in the country that does it at a very high level. In Europe and Australia, there are at least a dozen.”

One often-cited constraint is the U.S. reimbursement structure, which covers biopsies but not dermoscopy. Another is the expense of devices such as TBDP and RCM scanners. But the most important factor may be the demand that imaging places on a clinician’s working hours. “All these methods are time sinks, in terms of attention and decision-making,” Dr. Marghoob observes. “It’s simpler to just remove the lesion and send it off to a dermatopathology lab.”

Artificial intelligence could change that calculation by offering instantaneous analysis of dermoscopic images. Such programs, as described in numerous studies over the past decade, use algorithms modeled on human decision-making processes to perform tasks requiring sophisticated pattern recognition. To function properly, these algorithms must be trained, tested and fine-tuned — often requiring the input of terabytes of data points. Only recently has progress in computing power made practical use of AI possible, in areas ranging from voice-recognition software to pharmaceutical development.

AI-linked technologies are currently being explored for a wide variety of health-care applications. In dermatology, such programs are in their infancy, but they already power the change-detection features in some TBDP and digital dermoscopy systems. They also enable a new crop of direct-to-consumer mobile apps to assess photos of a user’s moles for melanoma risk, although the accuracy of those products remains uncertain.

As with direct-to-consumer DNA testing, Dr. Halpern predicts, melanoma screening apps will grow in popularity even in the absence of scientific validation. And that adds to the urgency of developing AI-based diagnostic systems for clinicians. “AI is going to dramatically democratize access to triage, which will lead to an epidemic of worry,” he explains. “If the patient’s app says his mole is likely cancerous, it’s going to be tough to convince him he doesn’t need a biopsy. You’re going to need some technology that makes you better than that app.”

A Tool for Better Decision-Making

The kind of system that Dr. Halpern and Dr. Marghoob envision would not make decisions for the clinician; rather, it would assist in the decision-making process. Instead of inspecting hundreds of lesions on the mole map of a patient with dysplastic nevus syndrome, for example, a dermatologist could focus solely on those flagged by AI as high-risk.

“Let’s say the AI sees something that it assesses as having a significant chance of being melanoma,” Dr. Marghoob suggests. “Without immediately telling the physician the diagnosis, the system would say, ‘Here’s an area you should look at.’ You could click a button to ask, ‘What kind of structure does the AI see?’ The answer might be, ‘negative network.’ You’d inspect the spot to check whether you see it, too.”

 

AI could be trained to interpret imaging from RCM and other advanced technologies as well. Another potential use would be to enhance diagnostic accuracy via telemedicine — a modality that is poised to “rocket into the stratosphere,” as Dr. Marghoob puts it, in the wake of the coronavirus pandemic.

Ideally, AI software would be linked to a patient’s electronic medical records. Images and associated data would be filed automatically, encrypted for privacy but readily available to anyone authorized to access them. In addition to its other advantages, such a system would eliminate paperwork and reduce the chance of wrong-site surgeries and other physician errors. “This will be a tool to improve diagnosis, not to replace diagnosticians,” Dr. Halpern says.

Teaching Machines to Outperform Humans

In order to develop the kind of artificial intelligence described here, three things must happen. First, dermatologic imaging, and the language used to describe it, needs to be standardized so that the machines and the people who design, program and operate them can all be on the same page. Second, AI systems need to be trained with millions of images vetted by experts as representing either melanoma, lesions that mimic it or normal tissue (in every possible skin tone). And third, those systems need to be tested on human subjects in well-designed clinical trials.

Dr. Halpern and a few colleagues launched ISIC in 2015 to pursue these and other goals essential to the modernization of dermatologic imaging, bringing together experts on melanoma and data science from academia and industry. The organization is working to develop dermatology-specific DICOM (digital communication in medicine) standards comparable to those long used in radiology and other fields. Since 2016, ISIC has sponsored annual melanoma detection challenges — testing the prowess of competing AI algorithms with increasingly complex analytic tasks, based on images from an ever-expanding archive.

The most recent contest for which results have been published, held in 2018, assigned 511 human readers to diagnose dermoscopic images selected randomly in batches of 30 from a test set of 1,511. These diagnoses were compared with those of 139 algorithms created by 77 machine-learning labs, each of which received a training set of 10,015 images in advance. The ground truth of each lesion fell into one of seven predefined disease categories: intraepithelial carcinoma, including actinic keratoses and Bowen’s disease; basal cell carcinoma; benign keratocytic lesions; dermatofibroma; melanoma; melanocytic nevus; and vascular lesions.

The two main outcomes were the differences in the number of correct specific diagnoses per batch between all human readers (including 283 board certified dermatologists, 118 dermatology residents and 83 general practitioners) and the top three algorithms, and between human experts (clinicians with more than 10 years of experience) and the top three algorithms. The study, published in 2019 in Lancet Oncology, found that the top AI programs outperformed the top humans. The 27 experts achieved a mean of 18.78 correct answers, compared with 25.43 correct answers for the champion algorithms. Even the three highest-scoring experts couldn’t beat the best machines.

These findings don’t mean that AI is ready to outdo experts in real-life clinical settings, where dermatologists typically draw on other factors besides imaging (such as medical history, lifestyle, age, gender and genetics) to make diagnoses. To maximize utility, these programs must learn to consider such data as well. They must also prove their accuracy outside of the computer lab.

The results of ISIC’s challenges do suggest, however, that in the not-too-distant future, intelligent machines might lend a hand to clinicians of all skill levels. This could enable dermatologists to detect more early melanomas, reduce the number of unnecessary biopsies and spend more time paying attention to their patients.

“AI is going to impact dermatology whether we engage in its development or not,” says Dr. Halpern. “As a profession, we have an enormous stake in making sure it’s implemented advantageously rather than harmfully. It’s our responsibility to make sure this is something that happens with us, rather than to us.”

---

Kenneth Miller is a journalist based in LA. A contributing editor at Discover, he writes on science, medicine and other topics for a wide range of publications.