Chapter 1.12 Putting it All Together

Toward the beginning of this chapter we asked the question, ‘do college students with work experience earn better grades than those without work experience.’  Knowing the steps involved in doing research and now having a basic understanding of the process, we could design our experiment and with fictional results could determine our conclusions and how to report our findings to the world.  To do this, lets start with our theory and progress through each of the ten steps.

Step 1: Determining a Theory.

Theories are developed through our interaction with our environment.  For our particular theory, we observed that older college students tend to perform better on classroom tests than younger students.  As we attempt to explain why, we developed our theory that real world work experience creates a motivation in students that allows them to perform better than students without this motivation.  Our theory, therefore, states that prior work experience will result in higher grades.

Step 2: Defining Variables.

Every experiment has an independent and a dependent variable.  The independent variable (IV) is what we start with; it refers to the separation of our groups.  In our case, we want to look at prior work experience so the presence or absence of this would constitute our experimental groups.  We may place those students who have been in the work force for more than one year in group 1 and those with less than one year in group 2.

Our dependent variable is our outcome measure so in our case we are looking for a difference in class grades.  To operationally define the variable grades, we might use the final course average as our outcome measure.  If the independent and dependent variable(s) are difficult to determine, you can always complete the following statement to help narrow them down: The goal of this study is to determine what effect _________ (IV) has on _________ (DV).  For us, the goal is to determine what effect one year or more of prior work experience has on course average.

Step 3: Determining Hypothesis.

When we plug our variables into our original theory we get our research hypothesis.  Simply stated, Students with one or more years of prior work experience will receive higher final course averages than students with less than one year of prior work experience.  Since statistical analysis often tests the null hypothesis or the idea that there is no difference between groups, our null hypothesis could be stated as: Final course averages of students with one or more years of prior work experience will not differ from final course averages of students with less than one year of prior work experience.

Step 4: Standardization.

To make sure that each subject, no matter which group they belong to, receives the same treatment, we must standardize our research.  In our case, we are looking at final course averages so we must make sure that each student receives the same instruction, the same textbook, and the same opportunities to succeed.  While this may be difficult in the real world, our goal is to get as close as possible to the ideal.

Therefore, we may choose to gather subjects from a general psychology class since this is a class required of most students and will not be affected by college major.  We may further decide to research only those students who have a specific instructor to keep the instruction between the two groups as similar as possible.  Remember, our goal is to assure, at least as much as possible, that the only difference between the two groups is the independent variable.

Step 5: Selecting Subjects.

Because our population consists of all college students, it will be impossible to include everyone in the study.  Therefore we need to apply some type of random selection.  Since we want to use only those students who have the same instructor, we may ask all of this instructors students, prior to any teaching, how much work experience they have had.  Those who report a year or more become the potential subject pool for group 1 and those who have less than one year become the subject pool for group 2.   We could, at this point decide to include all of these subjects or to further reduce the subjects randomly.  To reduce the subject pool we could assign each student in each group a random number and then choose, at random, a specific number of students to become subjects in our study.  For the purpose of this example, we will randomly choose 20 students in each group to participate in our study.

Step 6: Testing Subjects.

Since we are not applying any type of treatment to our subjects, this phase in the procession can be omitted.  If we were determining if the teaching styles of different instructors played a role in grades, we would randomly assign each student to a teacher.  In that case, teaching style would become an independent variable in our study.

Step 7: Analyzing Results.

Our original question asked if final averages would be different between our two groups.  To determine this we will look at the mean of each group.  Therefore, we will add up the averages of the 20 subjects in each group and divide each of these by 20 (representing the number of subjects in each group).  If, after comparing the means of each group, we find that group 1 has a mean of 88 and group 2 has a mean of 82 then we can descriptively state that there is a six-point difference between the means of the two groups.  Based on this statistic, we would then begin to show support for our alternative hypothesis and can progress to the next step.

Step 8: Determination of Significance.

Our goal was not to describe what their averages were, but rather to make inferences about what is likely happening in the entire population.  We must therefore apply inferential statistics to our results to determine the significant or lack of significant findings.  We will set our confidence level at 95 percent and then apply statistical analysis to our results to see if the difference of six points with a sample size of 40 is significant.

Imagine that we did find a significant difference.  In this case we could say that with a 95% confidence level, students with one year of more work experience receive higher averages than those with less than one year of work experience.  Since the null hypothesis, which stated that no difference exists between the two groups, was not correct, we must reject it.  And by rejecting the null, we automatically accept our alternative hypothesis.

Step 9: Communicating Results.

When communicating the results of our study we need to do several things.  We need to make a case for why we did this research, which is often based on our literature search.  We then need to report the process we took in gathering our sample and applying the treatment.  We can then report our results and argue that there is a difference between the two groups and that this difference is significant enough to infer it will be present in the entire population.

Finally, we must evaluate our research in terms of its strengths, weaknesses, applicability, and needs for further study.  In terms of strengths, we might include the rigors of gathering subjects and the fact that we used a random sample of students.  We may argue that the statistical methods used were ideal for the study or that we considered the recommendations of previously completed studies in this area.  Weaknesses might include the small sample size, the limited pool from which our sample was gathered, or the reliance on self-reported work experience.

To discuss applicability and needs for further studies we could suggest that more studies be completed that use a broader base of subjects or different instructors.  We could recommend that other variables be investigated such as student age, type and location of college, family educational history, sex, race, or socioeconomic background.  We might even suggest that while our findings were significant they are not yet applicable until these other variables are investigated.  The sections of a research report and how to write this report in order to communicate results is the main focus of chapter 2.

Step 10: Replication.

The final step in any research is replication.  This can be done by us but is most often completed by other researchers based on their own review of the literature and the recommendations made by previous researchers.  If others compete a similar study, or look at different variables and continue to find the same results, our results become stronger.  When ten other studies agree with ours, the chances are greatly improved that our results were accurate.  If ten other studies disagree with our findings then the validity of our study will be, and most certainly should be, called into question.

By replicating studies and using previously gained knowledge to search for new answers, our profession continues to move forward.  After all, we used the ideas of other researchers to design our research, and future researchers may incorporate our findings to make recommendations in their research.  The cycle is never ending and allows for perpetual seeking of new knowledge.