I became aware of chloroquine/hydoxychloroquine as a possible treatment for COVID-19 thanks to Google News’ feed on my phone presenting me yesterday with a wired article about some tech-bro tweets on the subject and today a with headline about Trump’s unwavering support for the treatment. This is the same Google news feed that has presented me with countless updates on a possible-but-unlikely Oasis reunion when I don’t even have Wonderwall on my Spotify guilty pleasures playlist. I frequently scroll through this feed in a depressed meditative state, and leave mildly reassured that if artificial intelligence can’t figure out what interests me, it’s probably not going to be stealing me job anytime soon. This time, Google actually managed to pique my curiosity: mostly because I wanted to see just how dumb this Trumpicon Valley solution really was. Good job, Googs!
Silicon Valley to the rescue!
The wired article explains how Silicon Valley became interested in chloroquine thanks to a series of tweets and retweets started by an MD who eschewed the opportunity to practice as a physician in order to fulfill his dream as a bitcoin investor. For those not familiar with bitcoin, it’s basically a super-cool high-tech pyramid scheme. I can think of no one better suited than this guy to be the face of heroism against a pandemic that is literally killing doctors who treat it. Anyway, he and his collaborators put together a google doc compiling existing information that supports the case for chloroquine as a COVID-19 treatment and have labored effortlessly to popularize/take credit for the idea. The document itself seems good enough to earn a passing grade on a med student end-of-rotation research-roundup assignment and has apparently gained considerably more press than the actual research.
Can we trust the French?
The most promising study for the benefit of hydroxycholoroquine in COVID-19 patients comes from France and was published March 17th. It studied hospitalized patients in multiple medical centers in France, comparing treated patients in one center to controls in other centers. This is a generally good structure for an observational trial because which center a patient presents to is less likely to be associated with the outcome than other inclusion factors (such as which patients agree to a treatment or which patients doctors decide to use a treatment on) and more closely simulates randomization. They also included patients that refused treatment at the main center in controls, which is a bit of a bummer because it makes for less elegant study design, but in this case appears more likely to have biased results away from showing a positive outcome (more on that later). The primary outcome was clearance of the virus at 6 days (established by negative PCR). The study found an impressively significant benefit of the hydroxychloroquine treatment (with 70% clearance in treated patients vs 12.5% clearance in controls and a p value of 0.001).
Does this difference matter? Yes. It’s not the most significant clinical outcome for the patient treated (intubation or death would be better for that purpose) but it is a surrogate for positive clinical outcomes in treated patients and a crucial outcome in terms of potential to decrease disease spread (Wuhan data shows a median viral clearance time of 20 days. In this treated population, median clearance time was reached at day 3.)
Are these results believable? Actually, yes. This isn’t the ideal type of evidence, but it’s much more convincing than I expected it to be. The gold standard would be a multi-center randomized controlled trial because randomization distributes potential confounders (known and unknown) equally between groups. In an observational study, you have to worry about differences between the groups which could be associated with both whether someone got the treatment (in this case, which center they presented at or if they refused) and the outcome (in this case, viral clearance). Given what we know about COVID-19, the differences we should worry about would be differences in age (with older people more severely affected) gender (with males more severely affected) and degree of infection at presentation (with patients with lower respiratory tract infections more severely affected).
In this study population, the control group was younger, less male, and less likely to have lower respiratory tract infection. In other words, we would expect every confounder we know about to bias the results away from showing benefit. Even if there are unknown confounders which would bias the results toward benefit, it’s hard to imagine they would have a larger effect than the differences we know about.
Some might argue that the small sample size (20 treated patients and 16 controls) makes these results unconvincing. The concern with small sample sizes is you are more vulnerable to random differences, but that’s what we calculate a p value for. If the effect size is big enough, results can be convincing even with a small sample. In this case, the probability of finding a difference as large as they did by chance–even in this small population–is a tenth of 1%.
What about Azithromycin? In this study, 6 of the 20 treated patients were also treated with Azithromycin. Azithromycin is an antibiotic that has previously shown in vitro benefit against Zika and Ebola and was given to patients in this case to prevent bacterial superinfection but the researcher’s found increased viral clearance in the patients treated with both hydroxycholorquine and azithromycin and Doctor Trump is all in on treating everyone with both. The benefit of azithromycin in these patients was clinically significant (with 100% of the 6 dual-treated patients clearing the virus by day 6) but this result is less convincing given it was an initial hypothesis of the study, raising the concern for a spurious result. The p value reported in this study is “<.001” which can only have been obtained by comparing the dual-treated patients to the non-treated patients. Doing a Fisher Exact Test comparing the hydroxychloroquine treated patients to the dual-treated patients at day 6 provides a p value of 0.11, which does not meet the general-accepted 5% threshold. Add to the fact that this is an observational trial and an outcome discovered after the fact rather than an a priori hypothesis, this result should be considered more interesting and hypothesis-generating than clear evidence of benefit.
Where do we go from here?
The gold standard for evidence of benefit hasn’t been met, doing so will take significant time, we’re in the middle of a pandemic, and many clinicians are already using hydroxychloroquine off-label to treat patients with COVID-19. The enlightened FDA and WHO officials will tell you not to jump to conclusions, to wait for randomized trial results and the established processes for treatment evaluation to run their course. The annoying silicon valley bros and Doctor Donalds of the world will tell you to listen to them and dive head-first into the chloroquine-fresh waters and let them heal the world. And any clinician treating patients who are going to the ICU and dying with COVID-19 will tell you it doesn’t feel right to not give something that might help them.
At this point, it may be difficult to execute a randomized controlled trial–specifically in the US–without too much cross-over of control patients wanting to be treated or doctors wanting to treat them. The best alternative might have to be larger observational studies using the differences in practices that already exist. Some medical centers treat all COVID-19 patients with hydroxychloroquine and others don’t regularly use that. Comparing outcomes between these centers will have its confounders, but if there’s a significant difference it will help establish the case for benefit. Much larger sample sizes will be available and different outcomes than viral clearance can be considered (e.g. intubations or deaths). Similarly, some centers have established or will establish a policy of treating with hydroxychloroquine mid-pandemic, and those centers’ outcomes can be compared to themselves before and after this change.
Evidence-based medicine isn’t simply waiting for the best possible evidence to absolve all doubt before offering a treatment: it’s using the best evidence available to you to make your clinical decisions. At this point, the best evidence says there’s likely a benefit of accelerating viral clearance when using hydroxychloroquine. It’s also a drug that’s been around for years with an acceptable safety profile. There are hypothetical harms (e.g. treatment with suboptimal doses could lead to resistant strains of the virus establishing dominance) but none are more convincing than the hypothetically and partially/incompletely-established benefit. It’s reasonable for clinicians to start treating in the absence of better options as we continue to collect data in efficient ways. If evidence for the benefit of hydroxychloroquine continues to mount, randomized trials would be beneficial for evaluating whether this drug could be effective as a form of prophylaxis.
I hate to say it, but sometimes the Trumpicon Valley blowhards are right, even if for the wrong reasons.
But should I be drinking Gin & Tonic?
As I’m sure you’re all thinking, chloroquine is a relative of quinine, an ingredient in tonic water famous for helping enable the colonization of Africa through its anti-malarial benefits when added to gin (or drank on its own, but gin actually improves the flavor of tonic water so it’s hard to argue against doing so). If chloroquine helps, it’s certainly plausible that quinine does as well. I’ve tested this theory myself, and I have to say the flavor profile of a quality gin combined with a mediocre tonic water is more pleasing than I remembered. I’ve also personally contracted COVID zero times since starting this practice.
What’s that? Opinion bear is telling me to drink gin and tonic to prevent COVID? I NEVER SAID THAT. There’s a plausible benefit but by no means does that outweigh the negative effects alcohol can have on your immune system. But if you’re going to be drinking anyway, it’s really the only logical way to go.
What’s that? Opinion bear says shutting down bars and depriving the people of life-saving gin and tonics is a criminally-irresponsible attack on public health by the government? Nooooo. I didn’t say that either. Stop putting words in my mouth. I’m done here.