Chapter 1.9 Determining Significance

The term significance when related to research has a very specific role.  Significance refers to the level of certainty in the results of a study.  We can say that our subjects differed by an average of ten points with 100% certainty because we personally witnessed this difference.  To say that the population will differ is another story.  To do this, we must determine how valid our results are based on a statistical degree of error.  If we find, through the use of inferential statistics, that the grades of those with and without work experience are different me must state the estimated error involved in this inference.  While the standard acceptable error is 5%, it can be as high as 20% or as low as 0.1%.

The amount of error to be accepted in any study must be determined prior to beginning the study.  In other words, if we want to be 95% confident in our results, we set the significance level at .05 (or 5%).  If we want to be 99% confident, our significance level is set at .01.  We can then state that there is a difference in the population means at the 95% significance level or at the 99% significance level if our statistics support this statement.  If our statistics estimate that there is 10% error and we said we would accept only 5%, the results of our study would be stated as ‘not significant.’  When determining significance, we are saying that a difference exists within our acceptable level of error and we must therefore reject the null hypothesis.  When results are found to be not significant, the only option available is to accept the null hypothesis.