Systematic and Random Error
There are two types of measurement errors: systematic and random. Bias and error reduces or destroys measurement validity. Bias is the distance between the true response and mean of the obtained responses. A good measurement tool should try to measure a variable as close to its true value. But it is almost impossible to measure the true value, even with biological values, because there is always some amount of error. An evaluator needs to plan, plot test, and implement methods to reduce the amount of error—both random and systematic error. Random error randomly affects measurement of the variable. But the positive interpretation about random error is that it does not have any consistent effects across the sample. In other words, the sum of all random error will be 0. Thus it does not change the conclusion of the study, but adds random (non-systematic) “noise” to the data and its values.
Systematic error is the major bias in measurement: values tend to be consistently positive or negative. A biased measure may lead an evaluation to make a different conclusion on a set of data for an impact rate than it would make if it had the true measure. Of course, evaluators do not usually know the true value, although it may sometimes be defined in the literature.
Measurement bias will always occur when not enough thought went into instrument development, if all measures and methods are not pilot tested, or the instrument was not used with care and attention to detail by assessors or participants. The assessment of the validity of an instrument is more difficult than the study of its reliability. Using the same instrument at two different times or with two observers (reliability) at the same time is relatively easy. Establishing validity requires an evaluation to obtain or to develop multiple measures of the dependent-impact-outcome variables and associated rates to determine which is the most accurate.
The assessment of salt in the diet provides an excellent example of potential, large measurement error. Because high salt intake is positively associated with high blood pressure (hypertension), a hypertensive control program may need instruments and methods to measure salt intake. A relatively accurate measure, not subject to self-report errors and biases, is an assessment of the sodium excreted in the urine. There is high day-to-day variability, however, in an individual’s salt consumption because of the lag time between episodes of unusually high salt ingestion and a body’s achievement of sodium balance. Several studies have estimated that an evaluator might need seven consecutive days of 24-hour urine samples to estimate regular salt consumption. Difficulties will arise in obtaining these samples. People do not want to carry urine sample bottles to work or to other activities because it is embarrassing and inconvenient. Multiple containers are needed to collect urine. They must be sterile, and urine must be collected at regular intervals. People will forget to provide every sample. The cost per individual would also be very expensive.
Multiple studies have been conducted to assess whether an overnight urine sample, testing for sodium and creatinine, can obtain similar information and replace the tedious and expensive 24-hour urine samples. Other investigators have used self-report measures of dietary consumption. In both cases, however, the seven consecutive days of 24-hour urine samples probably provides the criterion against which all measures of salt intake are assessed—“the gold standard”—because it was the more accurate and more valid measure of the variable desired.
