Coronavirus pandemic is all the rage around the world, and so is the race for cures. For example, several drugs are being repurposed: anti-malaria chloroquine, anti-Ebola Remdesivir, anti-flu Favipiravir, anti-IL-6R Tocilizumab…
However, most clinical results either involve small cohorts, and/or lack a control group. These methodological weaknesses generate heated controversies on social medias.
Randomized Controlled Trials vs. Real World Evidence
These controversies reflect the old debate between Randomized Controlled Trials (RCT) and Real World Evidence (RWE) (for a quick introduction, see here, or listen to this podcast by a16z from 10:21).
RCT are often perceived as the gold standard, because randomization removes all biases. However, RCT also have a lot of shortcomings. They are slow, expensive, and using placebo can be unethical. For example, French-led clinical trial Discovery is struggling to recruit patients, who are reluctant to enroll with only 20% chances of getting Hydroxy-chloroquine.
Moreover, some effects are too small to be detected in clinical trials. For example, no serious cardiac adverse effects were recorded in 177 malaria clinical trials of 35,548 participants who received quinoline and related antimalarials (including chloroquine). On the other hand, in real-world practice, lethal cardiac adverse events happened.
Real-world evidence is not only useful for monitoring rare events of toxicity. It is also increasingly used to establish efficiency. For example, in precision oncology (my current field of interest), there’s a shortage of patients meeting specific inclusion criteria. As a result, it’s faster for Pharma companies to perform a single-arm trial, and build an artificial control arm from RWE using machine learning (see Roche video and paper). In 2019, there have even been FDA approval for extending Pfizer’s Ibrance to some breast cancers in men (a rare disease), based on RWE from Flatiron database.
For a global database of Coronavirus Real World Data
RWE methods can be applied to coronavirus data too. To that end, individual patient data needs to be available, with all relevant features (age, comorbidities…). I didn’t find such public database yet (if you find anything, tell me please!).
However, in this direction, some steps are being made. For example, Raoult’s IHU Institute in Marseille is publishing data daily in their ”Southern France Morning Post”, and in their latest paper with 1061 patients. Unfortunately, it’s aggregated data, so it’s insufficient. Individual patient data is necessary to apply algorithms.
The reason is quite intuitive: to simplify, consider 2 datasets with 2 patients each, and 2 risk factors: age and obesity. These 2 datasets can be equivalent at the aggregated level, but very different at the individual level. They give different clinical outcomes:
Besides IHU Marseille, other initiatives are emerging. For example, the Broad Institute of MIT and Harvard in Cambridge, Massachusetts, is publishing their COVID-19 Diagnostic Processing Dashboard. Unfortunately, they don’t include a Therapeutics Dashboard yet, which would show patient data and outcomes.
How to reduce Real-World Data bias: Propensity Score Matching
In order to mimic randomized controlled trials, the key step is to minimize bias between real-world treatment and control arms. To that end, we select appropriate subsets of patients.
One way to do that is with Propensity Score Matching. The propensity score is the probability for a patient to belong to the treatment arm, conditionally to his medically relevant features (age, comorbidities…). In a 1:1 perfectly randomized trial, this probability is 0.5. In real-world data, this probability can be biased. However, balance can be restored by matching patients in treatment and control arms respectively, along their propensity scores.
More precisely, according to a mathematical theorem by Rosenbaum and Rubin, it’s sufficient to adjust for propensity scores, if there are no hidden biases due to unobserved features. That’s why it’s important to collect as much data as possible, from different countries and hospitals, so that hidden biases can cancel each other out.
Having data only from Marseille can introduce unexpected confounders. For example, we can imagine that Marseille performance is due to their warm Mediterranean micro-climate, and not to their routine use of chloroquine. Raoult himself conjectured that the epidemic could vanish with spring season.
How to estimate propensity score with AI ?
Propensity scores have to be calculated from available data. That’s where AI and machine learning come into play. The basic algorithm is logistic regression: the output of this classifier is a probability, between 0 and 1, corresponding to the propensity score of each patient.
Logistic regression can be improved with feature extraction and dimensionality reduction, especially when a lot of features are involved. That’s the role of pre-processing algorithms like autoencoders and deep neural networks.
Finally, there are several methods to make the matching. For example, one possible way is to match each treatment patient with its nearest neighbor in the control dataset, except when they are too far from each other (distance above a tolerance threshold).
If you are interested to explore the topic further, join our Medicine & Data Science group on Slack.
