A overview article lately revealed in BMC Medical Analysis Methodology reveals that the usual method for analysing active-control trials can result in clinically deceptive conclusions. The authors argue that the usual method has been focussing on the mistaken factor and that reversing our perspective can result in totally different, and higher, conclusions.

David Dunn 2 Aug 2023

Picture by Freepik

Randomised trials with a placebo management group are the gold commonplace for evaluating new therapies to enhance well being outcomes or to forestall illness. Nonetheless, it could be unethical to randomise sufferers to placebo if there may be already a longtime efficient technique for the therapy or prevention of the situation in query. On this case, a commonly-used design randomises sufferers to obtain both the brand new therapy or the usual therapy (referred to as active-control trials). The intention of such trials is commonly an try to indicate {that a} new therapy retains many of the effectiveness of the usual therapy whereas providing different advantages in price, simplicity, or uncomfortable side effects. We illustrate the issue with a hypothetical COVID-19 vaccine trial.

COVID-19 vaccine trial

The primary licensed COVID-19 vaccine, BNT162b2 (BioNTech/Pfizer), was discovered to cut back the incidence of COVID-19 by roughly 95%. Think about that we wished to evaluate a brand new COVID-19 vaccine after BNT162b2 had been licensed. Given the very excessive efficacy of BNT162b2, we resolve to conduct a trial evaluating the brand new vaccine versus BNT162b2. It’s a massive trial, with 10,000 individuals adopted for one yr.

Think about that the speed of COVID-19 circumstances within the trial inhabitants would have been 4% per yr within the absence of vaccination and that the brand new vaccine is 80% efficient. With these assumptions we might observe 20 COVID-19 circumstances within the BNT162b2 group and 80 COVID-19 circumstances within the experimental vaccine group (see Desk beneath).

Desk. Values in daring are recognized or straight noticed, different values are inferred.

The usual method to analysing this hypothetical trial is to calculate the ratio of the numbers of noticed circumstances within the two teams. Right here, the worth is 4, indicating that the brand new vaccine will increase the danger of getting COVID-19 by an element of 4, in contrast with BNT162b2. At face worth, this evaluation argues strongly towards approving or utilizing the brand new vaccine.

However this may imply rejecting a vaccine that we’ve got assumed is 80% efficient, a lot increased than the goal of fifty% that was set by the World Well being Group. 80% additionally occurs to be the approximate efficacy of the ChAdOx1 viral-vector vaccine which is significantly cheaper than BNT162b2 and has much less stringent chilly chain necessities, a key consideration in resource-limited settings. Certainly, ChAdOx1 saved extra lives worldwide in 2021 than every other COVID-19 vaccine. How can we resolve this paradox?

The significance of averted occasions

In our paper we argue that the interpretation of active-control trials should take into account averted occasions in addition to noticed occasions. From Desk 1 we see that BNT162b2 has averted 380 COVID-19 circumstances and that the brand new vaccine has averted 320 circumstances. The ratio of the numbers of averted circumstances is 320/380 = 0.842. The interpretation is that the brand new vaccine averts 84.2% of the COVID-19 circumstances that will have been averted by use of BNT162b2 (we name this the averted occasions ratio). This – moderately than the ratio of 4 when it comes to noticed occasions – is essentially the most related measure for coverage makers attempting to resolve which vaccine to implement.

Naturally, if all the pieces else was equal (together with price) then BNT162b2 can be the vaccine of alternative, however real-world choices are extra complicated than this.

Challenges

Thus far, we’ve got disregarded an enormous downside in estimating the numbers of averted occasions: we have to make an assumption in regards to the fee of COVID-19 within the absence of vaccination (we assumed 4% per yr in our instance). And not using a placebo group, we can not get this data straight from the trial and have to make use of exterior data as an alternative.

Our paper discusses numerous approaches to deal with this downside. None of those are totally passable however we argue that this have to be tried to attract rational inferences. Our proposed technique is particularly related when the usual therapy may be very efficient, the place it could actually result in a considerable saving in pattern dimension.