On 25 October Magnus Lynch, a consultant dermatologist and Mohs surgeon at Guy’s Hospital in London, published a test with the title “A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis.” It was on the Hacker News front page the same day, at 409 points when I looked. He drew 200 lesions from MILK10k, a dataset the International Skin Imaging Collaboration released in 2025 in which every lesion has a clinical photograph and a dermoscopic image, and every one of his 200 was confirmed by biopsy. He gave both images to two language models, GPT-6 Astra and Claude Fable 5.1, and to Inception v3, an image network Google published in 2015 that he fine-tuned on 68,011 images from the ISIC archive with every photograph of the test patients removed.
The headline result is clear, and I think it holds. The question underneath it is which result a patient should care about, because the post reports two, and they point in different directions until they are put on the same footing.
Naming the lesion
Each model had to pick one of six diagnoses: melanoma, basal cell carcinoma, squamous cell carcinoma, actinic keratosis, naevus or seborrhoeic keratosis. Inception v3 got 125 of the 200 right on its first choice. The language models got 84 and 85. Allowing a second guess, the counts were 173 against 134 and 138.
The gap is not uniform across the six classes. On basal cell carcinoma GPT-6 Astra was right on all 25, and the network on 24. The network’s lead comes mostly from the benign lesions and from squamous cell carcinoma. Lynch also reports how each language model leaned: GPT-6 Astra called 75 lesions basal cell carcinoma when 25 were, including 14 of the 30 melanomas, and Claude Fable 5.1 called 53 lesions actinic keratosis when 20 were.
Deciding who goes home
The second question in the test is the one a clinic actually has to answer: can this lesion be discharged with no treatment, biopsy or follow-up? Half of the 200 lesions were safe to discharge (50 naevi and 50 seborrhoeic keratoses) and half were not.
Here the language models look very different. GPT-6 Astra kept 97 of the 100 lesions that needed care and discharged 32 of the 100 that did not. Claude Fable 5.1 kept 96 and discharged 22. Both are cautious: they rarely let a dangerous lesion go, and they send most harmless ones on for more care. Inception v3, at the threshold it shipped with, discharged 76 of the safe lesions but kept only 77 of the unsafe ones, and it discharged 18 of the 30 melanomas. The language models discharged 3 and 4.
Read alone, that last row would make the network look dangerous, which is why the post does not stop there. Lynch moved the network’s threshold until it was as cautious as the language models, and it then kept 98 or 99 of the 100 unsafe lesions and discharged 2 or 1 of the 30 melanomas. That is the fair comparison, and the network holds up in it. I could not find in the post how many safe lesions the network discharged at that matched threshold, and that number is the one I would want next, because it says how much of the clinic’s time a safe network would actually save.
Scoring at a matched level of caution is how this field has long compared image networks with people. In 2019 Titus Brinker and colleagues tested a network trained on 12,378 open-source dermoscopic images against 157 dermatologists from 12 German university hospitals, on 100 images. The dermatologists’ mean sensitivity for melanoma was 74.1% at a mean specificity of 60%. Set to the same 74.1% sensitivity, the network’s mean specificity was 86.5%. Brinker’s study reported that second number. Lynch’s has not, yet, for Inception v3.
What the test cannot settle
Lynch lists the limits himself. Two hundred lesions show a pattern but cannot rank close results, and 30 melanomas leave wide uncertainty. Every lesion was biopsied, so the benign ones are harder than what a clinic usually sees. MILK10k has been public since 2025, so the language models may have seen it in training, which he notes would have helped them. Each language model was run once, and their answers vary between runs. Inception v3 is a baseline, not the state of the art. And the network’s area under the ROC curve fell from 0.96 on ordinary archive images to 0.86 on this test set, so these 200 lesions were hard for it too.
The safest reading is the one the post’s own numbers support. A small network trained for one job names skin lesions better than two general models that were shown the same two photographs with no training for the task. Whether any of them should decide who goes home depends on the threshold, and on that missing count.
References
- Lynch, M. (2026). A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis. Lynch Lab, 25 October 2026.
- MILK study team (2025). MILK10k. ISIC Archive. 10,480 images of 5,240 lesions, each a paired clinical and dermoscopic image.
- Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. (2016). Rethinking the Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
- Brinker, T. J., Hekler, A., Enk, A. H. et al. (2019). Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. European Journal of Cancer 113, 47–54.
- Gautam Parab (2026). Twice, Four Times or 1.7 Times: The Multiple Depends on Which Doctors. 24 October 2026.