 # Prediction of Treatment Effect: Surrogate Threshold Effect (STE)

The key motivation for validating a surrogate endpoint is the ability to predict the effect of treatment on the true endpoint based on the observed effect of treatment on the surrogate endpoint. Suppose that we have fitted the mixed- effects model (4.1) to data from a meta-analysis of N trials. Suppose further that a new trial is considered for which data are available on the surrogate endpoint but not on the true endpoint. It is essential to explore the quality of the prediction of the effect of Z on T in the new trial i = 0, based on the information contained in the trials i = 1,..., N used in the evaluation process, and the estimate of the effect of Z on S in the new trial. We can fit the following linear model to the surrogate outcomes S0j: We are interested in an estimate of the effect в + b0 of Z on T, given the effect of Z on S. To this end, one can observe that (в + bo|mSo, a0), where ms0 and a0 are, respectively, the surrogate-specific random intercept and treatment effect in the new trial, follows a normal distribution with mean linear in ps0, ps, a0, and a, and variance Here, Var(b0) denotes the unconditional variance of the trial-specific random effect of Z on T. The smaller the conditional variance given by (4.18), the better the precision of the prediction. Denote \$ to group the fixed-effects parameters and variance components related to the mixed-effects model (4.1), with \$ denoting the corresponding estimates. Fitting the linear model (4.17) to data on the surrogate endpoint from the new trial provides estimates for ms0 and a0. The prediction variance can be written as: where f |Var(ps0, a0)} and f {Var(\$)} are functions of the asymptotic variance-covariance matrices of (ps0, a0)T and \$, respectively. The third term on the right-hand side of (4.19), which is equivalent to (4.18), describes the prediction’s variability if ps0, a0, and \$ were known. The first two terms describe the contributions to the variability due to the use of the estimates of these parameters, respectively, in the new trial (estimation of ps0 and of a0) and in the meta-analysis (estimation of \$).

In reality, the parameters of models (4.1) and (4.17) all have to be estimated, in which case the prediction variance is given by the three terms on the right-hand side of (4.19). It is useful, however, to consider two theoretical situations:

• 1. No estimation error. If the parameters of models (4.1) and (4.17) were known, the prediction variance for в + b0 would only contain the last term on the right-hand side of (4.19). Thus, the variance would be reduced to (4.18) and the precision of the prediction would be driven entirely by the value Rrial. While this situation is of theoretical relevance only, as it would require an infinite number of trials and infinite sample sizes for the estimation in the metaanalysis and in the new trial, it provides important insights about the intrinsic quality of the surrogate, and shows that i?2riai measures the “potential” validity of a surrogate endpoint at the trial level.
• 2. Estimation error only in the meta-analysis. This scenario is again possible only in theory, as it would require an infinite sample size in the new trial. In that case, the parameters of the single-trial regression model (4.17) would be known and the first term on the right-hand side of (4.19) would vanish: In this case, equation (4.19) would provide the minimum variance of the prediction of в + b0 that is achievable when the size of the metaanalysis is finite. Gail et al. (2000) used this fact to point out that the use of a surrogate validated through a meta-analytic approach will always be less efficient than the direct use of the true endpoint. Even so, a surrogate can be of great use in terms of reduced sample size, shortened trial duration, or both.

Using the aforementioned considerations, Burzykowski and Buyse (2006) have proposed a useful measure of surrogacy, alternative to R2rial(r). In particular, assume that the prediction of в + b0 can be made independently of ps0. Under this assumption, the conditional mean of в + b0 is a linear function of a0, the treatment effect on the surrogate. Assume further that a0 is estimated without error. Then the conditional variance of в + b0 can be expressed as in (4.20) and it can be shown that it is approximately a quadratic function of a0.

Consider a (1 — 7)100% prediction interval for в + b0: where 27__72 is the (1 —7/2) quantile of the standard normal distribution. The limits of the interval (4.21) are functions of a0. Define the lower and upper prediction limit functions of a0 as One can then compute a value of a0 such that This value is called the surrogate threshold effect (STE). The STE is the smallest treatment effect on the surrogate necessary to be observed to predict a treatment effect on the true endpoint that is statistically significantly different from zero. The STE depends on the variance of the prediction. The larger the variance, the larger the absolute value of the STE. In practical terms, one would hope to get a value of STE that can realistically be achieved, given the range of treatment effects on surrogates observed in previous clinical trials. If the STE were too large to be achievable, the surrogate would not be useful for the purposes of predicting a treatment effect on the true endpoint. In such a case, the use of the surrogate would not be reasonable, even if the surrogate were “potentially” valid, i.e., with Rriai(r) — 1. The STE thus provides important information about the usefulness of a surrogate in practice.