MARKOV MODEL EXAMPLE: ADVANCED PROSTATE CANCER

Figure 4.3 depicts a simplified model of the treatment of hormone-naive advanced prostate cancer with abiraterone acetate (AA). This model and its attendant data are drawn from Ramamurthy et al.’s study of the costs and projected benefits of AA versus docetaxel in the treatment of this malignancy, to which the reader is referred for the complete model and cost-effectiveness analysis [5]. For this chapter, the model and data are simplified in favor of didactic value.

In Figure 4.3, states are represented for Stable on AA (denoted AA-Stable in the figure), AA with the side effect of fatigue (AA-Fatigue), progressive disease (PD), and Death (Dead). For clarity, arrows from states to themselves have not been drawn. The figure thus depicts the principal transitions in the model. The largest state-to-state transition is from progressive disease to death. Table 4.8 contains the initial monthly cycle transition probability matrix for a 68-year-old man (the median age in the clinical trials on which the model is based). This table is calculated from baseline estimates given by Ramamurthy et al.

Note that from AA-Stable, the most likely result after one month is to stay in AA-Stable, but there is a little greater than 2% chance of transitioning to PD. Additionally, there is a 0.3% chance of developing fatigue within the first six months of treatment, which would decrease quality of life for up to a year. A time-dependent general population risk of death must also be added to the model. At 68 years of age, the annual risk of death is 1.83%, rising over time according to the Gompertz exponential function. At 84, the risk of death is 10%. Table 4.9 shows the experience over 12 monthly cycles of 10,000 men aged

FIGURE 4.3 Principal transitions in prostate cancer model. Transitions to same state (e.g., Well-Well) are not shown.

TABLE 4.8

Transition Probability Matrix for the Prostate Cancer Model

68, treated with AA, according to the Markov model with the rising general death rate. In one month, 254 men have transitioned out of the AA-Stable state, with the greatest fraction to PD. The number of men in AA-Fatigue rises over the first six months of treatment, and then decreases as no more are added to this state and patients recover from treatment-induced fatigue. The number of men in PD rises each month, but as fewer men are on AA treatment with each passing month, the increase lessens with each month. In fact, after 32 monthly

Expected State Membership of Markov Cohort Over 12 Monthly Cycles

TABLE 4.9

cycles (not shown), the number of men in the PD state peaks, then begins to decline as death due to progressive cancer and the general population mortality takes a greater toll.

Over an expected lifetime, the Markov model yields a membership of each state as shown in Figure 4.4. The AA-Stable cohort declines steadily as men transition to one of the other model states, and back transitions from AA- Fatigue are never significant. AA-Fatigue rises over the first six months of the model, then declines to zero by Month 18. PD rises steadily to Month 32, and then falls slowly as patients die from their disease. The Dead state shows a typical sigmoid function, as initially few die, then the rate increases as more men reach PD, and then the rate slows as fewer men remain alive.

Baseline results from this model are presented in Table 4.10. Averaged over a cohort, the patient with advanced prostate cancer treated with AA can expect to live 32.9 months with stable disease. The overall 2% risk of fatigue, averaged across the entire cohort, adds 0.2 months per man in AA-Fatigue. A further 19.5 months are expected in PD, for an overall life expectancy of

52.6 months. Of course, no single patient has precisely this experience.

Sensitivity analysis (see Chapter 13) can be conducted on Markov transition probabilities, and modern software easily supports this. An Appendix to this chapter provides R code developed by the authors that can be used to replicate the model over the initial five years, and can be adapted to change parameters for sensitivity analysis or extend the model time [10].

Ramamurthy et al.’s more complete Markov formulation incorporates quality adjustments and a cost model, and is truncated to five years based on the

FIGURE 4.4 Natural history of prostate cancer example.

TABLE 4.10

Expected Results of Abiraterone Treatment Model

clinical trial results. Later chapters in this book will illustrate how costs and structural interventions can modify Markov and other stochastic models to generate sophisticated analyses of pharmacoeconomic problems.

Software Implementation

The modeler has a choice of different types of software to run a Markov model. TreeAge Pro is a specialized program for decision analysis with a full graphical user interface, where models are set up via “point-and-click” options. This interface makes it easy to work with, but the proprietary software limits transparency. It also may be challenging to set up especially complex models via a graphical interface, and model specifications are limited to the options available in the software [11]. Alternatively, the modeler could use a general purpose programming language to implement the model. Here we focus on R (R Foundation for Statistical Computing), but many other programming languages could be used, including MATLAB (The Math Works Inc.), Python, and SAS

(SAS Software). R is a freely available, open source statistical programming language, with a large library of user-contributed packages written for many types of analyses. If models are implemented with clear and well-documented code, using open source programming languages can improve reproducibility and transparency, making it easier for modelers to share and fully understand one another’s work [11-13]. Modelers can also borrow functionality from other packages, giving greater flexibility than what is available from specialized software. These advantages have led to an increase in the popularity of R for health economic evaluations [13]. One drawback of R is the learning curve for users unfamiliar with the language, especially for those with limited programming experience. Also, due to the nature of the user-contributed code libraries, there may be syntax differences between packages, and the amount of detail in the documentation and example code can vary, making some libraries easier to use than others.

Several R libraries contain useful implementations of Markov models, and more may be added in the future. The heemod (Health Economics Evaluation Modelling) package is particularly convenient, as it was specifically designed for cost-effectiveness analysis. We used this library to run our example (code available online at https://github.com/BethHandorf/Pharmacoeconomics-Mar kovModels). Other libraries for discrete-time Markov models are CTMCPack and markovchain. For a more thorough discussion on using R software for health economic decision modeling, see Jalal et al. [13].