Beyond Primary Endpoints: What to Believe?

By Michael S. Lauer, M.D.
NIH Deputy Director for Extramural Research

This past year, the year of COVID-19, has sure been strange, but in one key respect it’s been business as usual. Most of our decisions about appropriate clinical care and disease prevention have eventually come from large-scale clinical trials. Large, randomized trials have shown that:

• An mRNA vaccine dramatically reduces the risk of symptomatic COVID-19 illness 14 days after the second dose in seronegative adults.
• Remdesivir shortens time to recovery in hospitalized patients with lower respiratory tract illness.
• Dexamethasone reduces 28-day mortality in hospitalized patients, but only in those with more severe illness.
• Monoclonal antibodies reduce viral load in unhospitalized patients.
• Hydroxychloroquine does not improve clinical outcome after 14 days in hospitalized patients.

You might notice that for each of these trials I wrote that a specific intervention had a specific effect (e.g. reduced mortality at 28 days or improved clinical outcome at 14 days). According to the published journal articles, these specific effects were the “primary endpoints” or “primary outcomes.” If you take a close look at each of the published papers, you will also find mention of “secondary endpoints” and, in some, you might see mention of “exploratory endpoints.” What do these terms mean? And how should we interpret them when reading publications reporting the main results of clinical trials?

Today, most post-phase 1 trials are registered ahead of time in clinicaltrials.gov. Often, you will find the clinicaltrials.gov registration number just after the abstract of the published paper; for example, the registration number for the mRNA vaccine trial is NCT04470427. The registration is what the researchers declared they were going to do before they started enrolling patients. For this trial, you can see a list of 5 primary outcome measures, including 1 efficacy measure (“Number of Participants with a First Occurrence of COVID-19 Starting 14 Days after Second Dose of mRNA-1273”). There are also 10 secondary outcome measures, including “Number of Participants with a First Occurrence of Severe COVID-19 Starting 14 Days after Second Dose of mRNA-1273 or Placebo.”

Clinicaltrials.gov defines an outcome as “a planned measurement described in the protocol that is used to determine the effect of an intervention/treatment on participants.” A primary outcome is “the planned outcome measure that is the most important for evaluating the effect of an intervention/treatment. Most clinical studies have one primary outcome measure, but some have more than one.” A secondary outcome is “a planned outcome measure that is not as important as the primary outcome for evaluating the effect of an intervention but is still of interest. Most clinical studies have more than one secondary outcome measure.”

So, the primary outcome is what’s most important, while secondary outcomes are less important but still interesting. What does that mean operationally? Usually, the primary outcome is the outcome upon which rests the statistical design of the trial. Researchers decide how many patients they will enroll based on the primary outcome. In the case of the mRNA vaccine trial, they determined their sample size of approximately 30,000 people based on their assessment of the primary outcome, namely symptomatic COVID-19 illness starting 14 days after the second dose in seronegative people. They chose “severe COVID-19” as a secondary endpoint. In the end, there were 216 people who met the primary endpoint (204 in the placebo group and 12 in the vaccine group). There were far fewer people—only 30—who met the secondary endpoint of “severe COVID-19,” and all 30 were in the placebo group.

How do we interpret these findings? First and foremost, we focus on the primary endpoint, because the trial design (including the sample size) hinged on that. We can comfortably say that the vaccine dramatically reduces the primary outcome: symptomatic COVID-19 illness beginning 14 days after the second dose in seronegative people. Can we say that the vaccine reduces the secondary outcome, severe COVID-19? Technically, no, since the trial design was not based on the secondary outcome. Nonetheless, the remarkably consistency between the primary and secondary outcomes makes me feel comfortable saying that yes, in all likelihood, the vaccine does prevent severe COVID-19.

The mRNA vaccine trial is relatively straightforward since the primary outcome results were strong (amazingly strong!) and the subgroup analyses along with the secondary outcomes were all consistent with each other. Should the researchers have omitted reporting the secondary outcomes, since they were less important and not what the trial was designed to answer? I would say, “No, definitely report the secondary outcomes.” But be explicit that they were secondary outcomes and be circumspect in describing the results, exactly as happened here.  The whole picture of the trial as designed—primary outcomes and secondary outcomes—should be explained.

More often, results are not so straightforward. The treatment may not affect the primary outcome, yet seems to work for a secondary outcome. How do we interpret that? As a rule, we would consider the secondary outcome to be “hypothesis-generating” for conducting a new trial in which that outcome is now the primary outcome. However, in some cases the secondary findings are strong enough for practice recommendations. Such scenarios need to be considered on a case-by-case basis.

One last item: what about “exploratory endpoints” or “exploratory outcomes?” They are exactly what they say they are—exploratory. We must interpret such findings with great caution and at best say that more research is needed. The mRNA vaccine team noted briefly that “preliminary data” suggested benefit for asymptomatic infection, which they described as exploratory. We don’t know whether the vaccine prevents asymptomatic infection, and this trial cannot give us the answer. What’s more, while prevention of symptomatic illness is clearly important and relatively easy to understand, the implications for asymptomatic infection are much less clear. Well … lot’s more to think about!