# The Validity of Findings

Assuming, however, that the instruments and data are valid, we can ask whether the findings and conclusions derived from the data are valid. Asian Americans get higher average scores on the math part of the SATs (scholastic aptitude tests) than do other ethnic groups in the United States—581 versus 515 for all ethnic groups combined (College Board 2008). Suppose that the SAT math test is a valid instrument for measuring the general math ability of 18 year olds in the United States. Is it valid to conclude that ‘‘Asians are better at math’’ than other people are? No, it isn’t. That conclusion can only be reached by invoking an unfounded, racist assumption about the influence of certain genes—particularly genes responsible for epicanthic eye folds—on the ability of people to do math.

Reliability

Reliability refers to whether or not you get the same answer by using an instrument to measure something more than once. If you insert a thermometer into boiling water at sea level, it should register 212 Fahrenheit each and every time. “Instruments” can be things like thermometers and scales, or they can be questions that you ask people.

Like all other kinds of instruments, some questions are more reliable for retrieving information than others. If you ask 10 people ‘‘Do the ancestors take revenge on people who don’t worship them?’’ don’t expect to get the same answer from everyone. ‘‘How many brothers and sisters do you have?’’ is a pretty reliable instrument (you almost always get the same response when you ask a person that question a second time as you get the first time), but ‘‘How much is your parents’ house worth?’’ is much less reliable. And ‘‘How old were you when you were toilet trained?’’ is just futile.

Precision

Precision is about the number of decimal points in a measurement. When you stand on an old-fashioned scale, the spring is compressed. As the spring compresses, it moves a pointer to a number that signifies how much weight is being put on the scale. Let’s say that you really, truly weigh 156.625 pounds, to the nearest thousandth of a pound.

If you have a predigital bathroom scale like mine, there are five little marks between each pound reading; that is, the scale registers weight in fifths of a pound. In terms of precision, then, your scale is somewhat limited. The best it could possibly do would be to announce that you weigh ‘‘somewhere between 156.6 and 156.8 pounds, and closer to the former figure than to the latter.’’ In this case, you might not be too concerned about the error introduced by lack of precision.

Whether you care or not depends on the needs you have for the data. If you are concerned about losing weight, then you’re probably not going to worry too much about the fact that your scale is only precise to the nearest fifth of a pound. But if you’re measuring the weights of pharmaceuticals, and someone’s life depends on your getting the precise amounts into a compound, that’s another matter.

Accuracy

Finally, accuracy. Assume that you are satisfied with the level of precision of the scale. What if the spring was not calibrated correctly (there was an error at the factory where the scale was built, or last week your overweight house guest bent the spring a little too much) and the scale was off? Now we have the following interesting situation: The data from this instrument are valid (it has already been determined that the scale is measuring weight—exactly what you think it’s measuring); they are reliable (you get the same answer every time you step on it); and they are precise enough for your purposes. But they are not accurate. What next?

You could see if the scale were always inaccurate in the same way. You could stand on it 10 times in a row, without eating or doing exercise in between. That way, you’d be measuring the same thing 10 different times with the same instrument. If the reading was always the same, then the instrument would at least be reliable, even though it wasn’t accurate. Suppose it turned out that your scale was always incorrectly lower by 5 pounds. This is called systematic bias. Then, a simple correction formula would be all you’d need to feel confident that the data from the instrument were pretty close to the truth. The formula would be:

true weight = your scale weight + 5 pounds

The scale might be off in more complicated ways, however. It might be that for every 10 pounds of weight put on the scale, an additional half-pound correction has to be made. Then the recalibration formula would be

true weight = (your scale weight) + (scale weight/10)(.5) or