Case Studies

The Age-Related Macular Degeneration Trial

The results in Section 15.3.1 indicate that the use of MI to balance an unbalanced dataset prior to fitting Model (15.1) reduces model convergence issues. In this section, this method is applied to estimate surrogacy in the ARMD trial. In particular, it will be examined whether the change in visual acuity after 24 weeks is an appropriate surrogate for the change in visual acuity after 52 weeks. As was detailed in Section 2.2.1, the ARMD trial is a multicenter study that enrolled a total of 181 patients from 36 centers. Here, the clustering variable is center (rather than trial). Centers that enrolled less than 5 patients were discarded from the analyses to avoid problems during the MI phase (recall that the imputations are conducted for each center separately, and thus a

Model

Small imbalance щ ~ N (20, 2.52)

Large imbalance щ ~ N (20, 52)

Convergence

category

Between-cluster variability 7

Number of clusters

Number of clusters

5

10

20

50

5

10

20

50

MI UN

Proper convergence

Small (.1) Large (1)

.164

.617

.941

.999

  • 1
  • 1
  • 1
  • 1

.189

.616

.946

.999

.999

.999

.999

1

Convergence but non-PD D/H matrix

Small (.1) Large (1)

.811

.365

.059

.001

  • 0
  • 0
  • 0
  • 0

.770

.343

.054

.001

.001

.001

.001

0

Divergence

Small (.1) Large (1)

.025

.018

  • 0
  • 0
  • 0
  • 0
  • 0
  • 0

.042

.041

  • 0
  • 0
  • 0
  • 0
  • 0
  • 0

MI FA0(4)

Proper convergence

Small (.1) Large (1)

.813

.954

.998

.999

  • 1
  • 1
  • 1
  • 1

.817

.950

.996

1

.999

.999

.999

1

Convergence but non-PD D/H matrix

Small (.1) Large (1)

.077

.012

.002

.001

  • 0
  • 0
  • 0
  • 0

.072

.022

.003

0

0

.001

  • 0
  • 0

Divergence

Small (.1) Large (1)

.109

.029

.002

.001

  • 0
  • 0
  • 0
  • 0

.111

.028

.001

0

.001

0

  • 0
  • 0

TABLE 15.4

Bias, efficiency, and MSE of the estimates of R%ndiv in the non-MI and MI surrogate evaluation models that properly converged as a function of balance of щ, the number of clusters (5, 10, 20, 50J, and the between-cluster variability.

Model

Balanced (equal nf)

Small imbalance hi ~ N (20, 2.52)

Large imbalance

hi ~ N (20, 52)

Measure

Between-cluster variability (7)

Number of clusters

Number of clusters

Number of clusters

5

10

20

50

5

10

20

50

5

10

20

50

Non-MI,

UN

Bias

Small (.1)

Large (1)

  • -.001
  • -.005
  • -.001
  • -.001
  • -.002
  • -.001

.001

-.001

  • -.014
  • -.009

.001

-.001

-.001 .001 -

.001

-.001

.009

-.009 -

.001

-.001

-.002 .001 -

.001

-.001

Efficiency

Small (.1)

Large (1)

.080

.072

.052

.053

.037

.037

.023

.024

.072

.071

.053

.052

.037

.037

.023

.024

.067

.073

.052

.052

.038

.037

.023

.024

MSE

Small (.1)

Large (1)

.006

.005

.003

.003

.001

.001

.001

.001

.005

.005

.003

.003

.001

.001

.001

.001

.004

.005

.003

.003

.001

.001

.001

.001

Non-MI,

FA0(4)

Bias

Small (.1)

Large (1)

  • -.001
  • -.006
  • -.009
  • -.001
  • -.002
  • -.001

.001

-.001

  • -.001
  • -.007

.001

-.001

-.001 .001 -

.001

-.001

-.001 -.007 -

.001

-.001

-.002 .001 -

.001

-.001

Efficiency

Small (.1)

Large (1)

.074

.073

.052

.053

.037

.037

.023

.024

.076

.072

.053

.053

.037

.037

.023

.026

.072

.074

.051

.052

.038

.038

.023

.027

MSE

Small (.1)

Large (1)

.006

.005

.003

.003

.001

.001

.001

.001

.006

.005

.003

.003

.001

.001

.001

.001

.005

.006

.003

.003

.001

.001

.001

.001

MI,

UN

Bias

Small (.1)

Large (1)

  • -.005
  • -.005

.001

.002

-.001

.002

.003

.001

.008

-.005

.004

.001

-.001

.004

.004

.001

Efficiency

Small (.1)

Large (1)

.089

.087

.066

.065

.047

.047

.030

.031

.095

.098

.074

.076

.057

.057

.040

.043

MSE

Small (.1)

Large (1)

.008

.008

.004

.004

.002

.002

.001

.001

.009

.010

.005

.006

.003

.003

.002

.002

MI,

FA0(4)

Bias

Small (.1)

Large (1)

  • -.001
  • -.005

.001

.002

-.001

.002

.003

.001

.003

-.005

.003

.001

-.001

.004

.004

.001

Efficiency

Small (.1)

Large (1)

.090

.097

.066

.065

.047

.047

.030

.031

.097

.099

.074

.076

.057

.057

.040

.043

MSE

Small (.1)

Large (1)

:

:

:

:

.008

.008

.004

.004

.002

.002

.001

.001

.009

.010

.006

.006

.003

.003

.002

.002

Model

Balanced (equal щ)

Small imbalance

m ~ N (20, 2.52)

Large imbalance

m ~ N (20, 52)

Measure

Between-cluster variability (7)

Number of clusters

Number of clusters

Number of clusters

5

10

20

50

5

10

20

50

5

10

20

50

Non-

Mi,

UN

Bias

Small (.1)

Large (1)

  • -.152
  • -.168
  • -.097
  • -.075
  • -.041
  • -.031
  • -.019
  • -.013
  • -.168
  • -.170
  • -.093
  • -.074

-.036 - -.031 -

  • -.016
  • -.012
  • -.174
  • -.152
  • -.094
  • -.074

-.034 - -.029 -

  • -.002
  • -.009

Efficiency

Small (.1)

Large (1)

.254

.248

.224

.219

.182

.158

.111

.095

.246

.239

.229

.222

.184

.157

.112

.096

.280

.248

.232

.221

.187

.157

.114

.095

MSE

Small (.1)

Large (1)

.087

.089

.059

.053

.035

.026

.013

.009

.088

.086

.061

.055

.035

.026

.013

.009

.108

.084

.063

.054

.036

.025

.013

.009

Non-

Mi,

FA0(4)

Bias

Small (.1)

Large (1)

  • -.139
  • -.099
  • -.090
  • -.077
  • -.071
  • -.060
  • -.035
  • -.024
  • -.157
  • -.109
  • -.088
  • -.073

-.061 - -.063 -

  • -.032
  • -.024
  • -.151
  • -.076
  • -.082
  • -.071

-.057 - -.058 -

  • -.018
  • -.020

Efficiency

Small (.1)

Large (1)

.267

.272

.233

.232

.203

.178

.121

.101

.269

.269

.238

.236

.197

.179

.122

.100

.273

.266

.237

.234

.201

.177

.125

.101

MSE

Small (.1)

Large (1)

.090

.084

.062

.060

.046

.035

.016

.011

.096

.084

.064

.061

.043

.036

.016

.011

.097

.076

.063

.060

.044

.035

.016

.011

MI,

UN

Bias

Small (.1)

Large (1)

  • -.186
  • -.165
  • -.080
  • -.071

-.034 - -.031 -

  • -.016
  • -.012
  • -.193
  • -.163
  • -.087
  • -.072

-.036 - -.032 -

  • -.022
  • -.018

Efficiency

Small (.1)

Large (1)

.255

.244

.226

.221

.172

.155

.105

.095

.257

.254

.229

.222

.174

.160

.128

.108

MSE

Small (.1)

Large (1)

.099

.087

.057

.054

.031

.025

.011

.009

.103

.091

.060

.054

.031

.027

.017

.012

MI,

FA0(4)

Bias

Small (.1)

Large (1)

  • -.258
  • -.193
  • -.088
  • -.071

-.034 - -.031 -

  • -.016
  • -.012
  • -.266
  • -.193
  • -.094
  • -.072

-.036 - -.032 -

  • -.022
  • -.018

Efficiency

Small (.1)

Large (1)

.251

.253

.228

.221

.172

.155

.105

.095

.251

.261

.233

.222

.174

.160

.128

.108

MSE

Small (.1)

Large (1)

:

:

:

:

.130

.101

.060

.054

.031

.025

.011

.009

.134

.106

.063

.054

.031

.027

.017

.011

TABLE 15.6

Convergence rates for the Age-Related Macular Degeneration Trial using MI UN (unstructured) and MI FA0(4) (factor analytic) to restore “balance” in cluster sizes.

mi un :

MI FA0(4)

Proper convergence

0.701

0.944

Convergence but non-PD D/H matrix

0.299

0.015

Divergence

0

0.041

sufficient number of observations should be available in a center). The dataset that was analyzed contained 119 patients from 17 centers. The center that had the largest sample size included 18 patients, of whom 9 patients received placebo and 9 patients received the experimental treatment. The same procedure that was described in Section 15.3 to obtain “balanced” datasets (using MI) was employed here. Thus, in all center-by-treatment groups that had less than 9 patients, data were imputed to achieve balance. The imputations were conducted for each of the centers separately, using change in visual acuity after 24 weeks as S, change in visual acuity after 52 weeks as T, as well as treatment Z in the imputation model. A total of 1000 imputations were conducted. For each of the imputed datasets, Model (15.1) was fitted. Both the FA0(4) and UN covariance parameterizations for D were used.

Results

When Model (15.1) was fitted to the non-imputed data of the case study, convergence issues occurred. In particular, the models that used the UN parameterization for the D matrix did not converge and the models that used the FA0(4) parameterization converged to a non-PD D/H matrix.

Table 15.6 shows the convergence rates that were obtained when the MI- based approaches were used. Overall convergence was high and equaled 100% and 96.9% in the MI UN and MI FA0(4) scenarios, respectively. The use of the MI FA0(4) strategy led to higher rates of proper convergence compared to the MI UN strategy (94.4% versus 70.1%, respectively).

The mean i?2rial of the properly converged results and their CI95% for the MI UN and FA0(4) models equaled 0.573 [0.078; 0.941] and 0.597 [0.069; 0.985], respectively. The mean R?ndiv and their CI95% were 0.453 [0.192; 0.673] and 0.431 [0.079; 0.696] for the MI UN and FA0(4) models, respectively.

To establish a frame of reference against which these estimates can be compared, the non-imputed ARMD data were analyzed using a two-stage approach (a full bivariate weighted fixed-effect model was used; for details see Chapter 4). This analysis yielded Rrial = 0.729 with CI95% = [0.487; 0.972] and R2ndiv = 0.512 with CI95% = [0.384; 0.639].

The results indicate that there was an acceptable level of agreement between the trial- and individual-level surrogacy estimates that were obtained in the MI-based and non-MI-based approaches. Both analyses lead to the conclusion that, at the level of the trial (center), the treatment effect on T = visual acuity after 52 weeks can be predicted with moderate accuracy based on the treatment effect on S = visual acuity after 24 weeks (moderate R?2rial estimates). At the level of the individual patients, the accuracy by which T = visual acuity after 52 weeks can be predicted based on S = visual acuity after 24 weeks is moderate as well (moderate i?2ndiv estimates).

The 95% confidence intervals (CI95%) of the MI-based Rrial and R?2ndiv were wide, but it should be kept in mind that the number of clusters and patients were relatively small. In addition, there were large imbalances in the cluster sizes in the ARMD dataset. For example, 7 out of the 17 centers that were available for analysis had only 5 patients and thus the ratio of the available data relative to the data that had to be imputed in these centers was small (available: 5 patients, to be imputed: 13 patients). In the next section, another case study is considered where the number of clusters and the number of patients is higher.

 
Source
< Prev   CONTENTS   Source   Next >