The Curie Institute, a cancer research center in Paris, launched the AI4Curie challenge, about AI for prediction of immunotherapy response in lung cancer:
With a 60,000 Euros prize, the goal is to build a predictive model from histology images of lung tumors. Something like this:
Or like this:
You can learn more about cancer immunotherapy prediction in a previous blog post, and in this Youtube mini-series.
It’s always good to throw money at cancer problems, in any way. However, this way is not the best, and that’s a situation Curie Institute donors should think about.
I was disappointed by the lack of innovation from the jury. Last year, Roussy Cancer Institute, in Paris region too, launched the AI Challenge for Health, for prediction of cancer outcomes from CT scans, with a 1,500,000 Euros prize.
Those two challenges are strikingly similar, even though there are so many other applications of Artificial Intelligence to cancer, which also need exposure and funding. Why not neoantigens identification? Prediction of epigenetic modifications? CRISPR-Cas9 gene editing optimization?…why settling down for a “Me-too” challenge?
In the jury of this 2020 Curie challenge, it’s notable to find the CTO of Owkin, the Paris-based startup that won the 2019 Roussy challenge. So why does the Curie Institute need this new challenge, instead of simply re-purposing Owkin solution?
Selection criteria are not transparent: candidates are evaluated only on their proposals. The jury arbitrarily decides which applicant they prefer. This raises doubts about competition fairness. Will the ultimate standard be the legendary “copinage”?
In particular, how will the jury evaluate the criteria of “originality and innovativeness”, given the large literature covering the subject? (see this Google Sheet) What is exactly wrong with existing approaches?
Hint: Owkin wrote 2 papers around the topic. So I recommend participants to avoid excessive innovation and originality, and instead overfit jury expectations.
Why provide only histology images? Why not provide also transcriptomics data, cytometry, and so on? More comprehensive patient data could improve the odds of making correct predictions.
Hints: first, they probably didn’t collect transcriptomics data. This type of data is still not routinely collected in the clinic, because it is still considered expensive (although it costs less than $300, a tiny fraction of an oncology bill).
Second, they have the overly ambitious goal of developing a non-invasive diagnostics method, only based on images. That’s a long-term and uncertain objective, from both technology and commercial perspectives. Within several years, other methods (transcriptomics, image mass cytometry…) might have become routinely adopted in the clinic, and more effective than an image-only solution.
This situation is different from the new coronavirus SARS-CoV-2 pandemic, where hospitals are overwhelmed. In this case, expeditive and not-so-accurate diagnostics from CT scans are still better than nothing (see my previous blog post). Cancer centers are generally not over-capacity, and should deliver the best care possible, based on all data.
Why not simply upload this new dataset in public databases? That would truly accelerate research. Curie Institute motto is “prenons le cancer de vitesse” (let’s outpace cancer), but keeping this dataset private does not help. Is it a good practice for a charity relying on people’s generosity and taxes?
Thanks to donations and patient data, Curie Institute is launching a private business, instead of contributing to the public data pool, in the Cancer Genome Atlas for example.
Curie Institute is putting profits before patients, but this practice is widespread, and even affects public healthcare. In a previous blog post, I denounced attempts to monetize NHS data in England, instead of maximizing their use and potential benefits. Charity hospitals like Curie should do everything to develop cures and diagnostics quickly, as they claim in their fundraising campaigns.
Conclusion
The only benefits I see from this Curie challenge are academic and educational. Computer vision enthusiasts can learn about medical and histology images with this toy problem, supported by an abundant literature (see again this Google Sheet).
In our Slack and Telegram groups, we started several educational challenges months ago, including one on cancer outcome prediction with histology images (with no prize, and no deadline). Now, there are hundreds of registered users, and any expert and enthusiast can join.
Update: After this blog post, Institut Curie relaunched their challenge by incorporating my feedback:
- Prize grew from 60k to 1 Million Euros:
- The challenge now incorporates “multi-modal” data:
However, when contacted, organizers from Institut Curie declined to comment.
