Threats to Validity in Trials

Despite the strengths of randomized trials, there are several issues that can lead to biases in assessment. As mentioned, blinding is intended to reduce some of

these biases, by reducing opportunities for subjective evaluations to be influenced by knowledge of treatment. Some other sources of bias in trials are incomplete follow-up, intent-to-treat analysis, and confounding imbalances that stem from random assignment.

Incomplete Follow-up

Randomized trials are susceptible to many of the same biases that afflict other types of cohort studies. One source of bias is differential follow-up of the treatment groups. The ideal situation regarding follow-up is for there to be no subjects lost to follow-up, which prevents any bias from this source. In most trials, however, some subjects are not followed to the intended study end point. Reasons for incomplete follow-up are the same ones that occur in other cohort studies, which include subjects moving from the study area, withdrawing their consent to participate in the study, or dying from a disease that is not one of the study end points. If some study subjects are lost to follow-up for any of these reasons, the count of events will be underestimated compared with what it would have been had there been no losses to follow-up.

To deal with this potential source of bias, investigators may analyze the data under the assumption that the experience of those who were lost to follow-up is similar to that of those who remained in the study. This assumption, however, is not always reasonable. For example, subjects with worsening symptoms may be more inclined to drop out of the study than those with a better prognosis. In that case, the risk of the outcome in each treatment group would be underestimated if it were based on the experience of those with complete follow-up. Alternatively, those with the worst prognosis may be less likely to drop out of a study if they believe that they will receive better care by remaining in it. In that case, the study will overestimate the risks of the study outcome, because those dropping out are at lower risk than those remaining in the study. If follow-up is incomplete and is related to both the study intervention and the study outcome, the result is differential loss to follow-up between study groups, a type of selection bias. Differential loss to follow-up can lead to study results that are biased in either direction.