STATISTICAL HYPOTHESIS TESTING VERSUS ESTIMATION
Often, a P value is used to determine the presence or absence of statistical significance. Statistical significance is a term that appears laden with meaning, although it tells nothing more than whether the P value is less than some arbitrary- value, almost always .05. The term statistically significant and the statement “P < .05” (or whatever level is taken as the threshold for statistical significance) are equivalent. Neither is a good description of the information in the data.
Statistical hypothesis testing is a term used to describe the process of deciding whether to reject or not to reject a specific hypothesis, usually the null hypothesis. Statistical hypothesis testing is predicated on statistical significance as determined from the P value. Typically, if an analysis gives a result that is statistically significant, the null hypothesis is rejected as false. If a result is not statistically significant, it means that the null hypothesis cannot be rejected. It does not mean that the null hypothesis is correct. No data analysis can determine definitively whether the null hypothesis or any hypothesis is true or false. Nevertheless, it is unfortunately often the case that a statistical significance test is interpreted to mean that the null hypothesis is false or true according to whether the statistical test of the relation between exposure and disease is or is not statistically significant. In practice, a statistical test, accompanied by its declaration of "significant” or “not significant,” is often mistakenly used as a forced decision on the truth of the null hypothesis.
A declaration of statistical significance offers less information than the P value, because the P value is a number, whereas statistical significance is just a dichotomous description. There is no reason that the numeric P value must be degraded into this less-informative dichotomy. Even the more quantitative P value has a problem, however, because it confounds two important aspects of the data, the strength of the relation between exposure and disease and the precision with which that relation is measured. To have a clear interpretation of data, it is important to be able to separate the information on strength of relation and precision, which is the job that estimation does for us.