Critical Appraisal of Outcome Studies of Autism and AS
In contrast to natural history studies, there is a much broader literature on the outcome of individuals with autism. The findings suggest that, for most individuals, autism is a lifelong disorder with significant deficits in social functioning and communication limiting adaptive functioning as adults.26,27 The number and severity of autistic symptoms seems to decrease over time, but most adults with autism remain dependent on care givers for many years. The two most important predictors of outcome are IQ and language: those with higher IQ scores and better language abilities have a better outcome.
Although these findings are consistent, the follow-up and outcome literature is limited by a number of methodologic issues in sampling, measurement, and analysis that affect both the internal and external validity of the findings. With respect to sampling, in most of outcome/follow-up studies, samples were enrolled at very different points in the trajectory of the disorder. An important methodologic criterion for outcome studies is that the sample should represent an “inception” cohort in which subjects are sampled at the same early stage of the disorder. For example, many outcome studies sampled individuals with autism at 10-12 years of age (or as adults, see earlier discussion) even though the disorder can be said to onset at 2-3 years of age. Such late sampling raises the possibility that a substantial number of individuals with autism (or AS) may have been lost to the sampling frame. This includes those who might make a rapid recovery and no longer meet criteria for the diagnosis or those who lose their autism diagnosis but perhaps retain the diagnosis of intellectual disability or some other developmental disorder. It is also important to highlight the necessity of obtaining a representative group of participants by sampling a consecutive series of cases rather than a convenience sample. The representativeness of a convenience sample is always problematic because it is likely that volunteers are different on a number of variables from study participants who are sampled consecutively.
Regarding measurement, one important assumption to consider is whether the trajectories on a variable of a study group are homogeneous or heterogeneous over time. This assumption will determine the method of statistical analysis used to estimate change over time. It is also essential that reliable and valid outcome and predictor measures are used. The concept of “development equivalence” is important in this context.28 That is, instruments should measure the same construct at several different points in time. It may be, for example, that the communication construct measured by a specific instrument is different when applied to preschoolers compared to when it is applied to late adolescents. In preschoolers, the instrument may be measuring largely grammar and vocabulary skills, whereas in adolescence it may measure more pragmatic (i.e., more social) aspects of communication. Using the same measure over both time periods may make it difficult to interpret changes in the development of the construct over time.
It is also extremely important to have a prospective measurement of predictor and outcome variables. Retrospective measurement of predictors (as was done in many of the AS-autism outcome studies) leads to potential measurement bias; that is, the current status of the outcome may influence the measurement of the predictor variable. Another important issue is to be sure that if a study is assessing different outcomes, these should be “independent” of each other from a measurement perspective. For example, the Vineland Adaptive Behavior Scales (VABS) of communication and socialization are highly correlated, and doing separate analyses on both variables and arriving at similar results may reflect the fact that these measures share a lot of variance and cannot be considered independent constructs. The same applies to the independence of multiple predictor variables. The two most important predictor variables in the outcome literature in autism are language and IQ.26 Thus, for example, it may be that language and IQ in AS are very highly correlated and their predictive ability is in fact related to their shared variance rather than to anything independent and unique. The lesson to be drawn from these considerations is that a very careful measurement model must be developed that includes both predictor and outcome measures prior to the analysis of the influence of predictors on outcomes.
A final important methodological issue is the method of analysis. Longitudinal studies can address several different questions: the prevalence of an outcome, the time to an outcome (such as relapse or recovery), or change over time in an outcome. Which question the investigator is most interested in is extremely important to determine because it will set the best longitudinal analytic technique to use (whether it is simple linear regression, survival analysis, or growth curves, for example). To measure change, it is critically important to have more than two data points (i.e., baseline and one follow-up). Trajectory analysis requires at least three data points, and the more data points the better, particularly if one is interested in estimating the shape of a curve.