Chapter 10.4 Methods

The methods section is often the most precisely written part of a research report. Since replication and analyzing methods is so important, a good deal of time should be spent analyzing this section. As a consumer of research, it is imperative for you to understand the foundation of each study and be able to critically analyze how the data that will lead to the results section was derived.

When reading the methods section you should look for information regarding the subjects and the manner in which the subjects were selected. You should be able to discuss the pitfalls of not using randomization, or of various types of randomization. You should be able to understand the strengths and weaknesses of the type of design used and how the researchers used control groups or groups that were not equivalent. The use of standardized procedures is also important, as we ideally want every group to experience the same environment except for the variable(s) being measured. If confounding variables are not controlled for, you should be able to discuss how this lack of control might impact the results of the study.

Issues related to internal and external validity should be carefully addressed as well. Look for how the researchers addressed subject maturation in longitudinal or longer term studies, how they handled extreme scores and the tendency for scores to regress toward the mean, and how they dealt with differences in mortality or drop out rates between groups. And finally, you should be aware of what assessment procedures were used, whether these instruments are valid and reliable, and whether or not they were used correctly.