Putting
it All Together
Toward
the beginning of this chapter we asked the question,
‘do college students with work experience earn
better grades than those without work experience.’
Knowing the steps involved in doing research
and now having a basic understanding of the process,
we could design our experiment and with fictional
results could determine our conclusions and how to
report our findings to the world.
To do this, lets start with our theory and
progress through each of the ten steps.
Step
1: Determining a Theory.
Theories are developed through our
interaction with our environment.
For our particular theory, we observed that
older college students tend to perform better on
classroom tests than younger students. As we attempt to explain why, we developed our theory that
real world work experience creates a motivation in
students that allows them to perform better than
students without this motivation. Our theory, therefore, states that prior work experience will
result in higher grades.
Step
2: Defining Variables.
Every experiment has an independent and a
dependent variable.
The independent variable (IV) is what we
start with; it refers to the separation of our
groups. In
our case, we want to look at prior work experience
so the presence or absence of this would constitute
our experimental groups.
We may place those students who have been in
the work force for more than one year in group 1 and
those with less than one year in group 2.
Our
dependent variable is our outcome measure so in our
case we are looking for a difference in class
grades. To
operationally define the variable grades, we might
use the final course average as our outcome measure.
If the independent and dependent variable(s)
are difficult to determine, you can always complete
the following statement to help narrow them down:
The goal of this study is to determine what effect
_________ (IV) has on _________ (DV).
For us, the goal is to determine what effect
one year or more of prior work experience has on
course average.
Step 3: Determining
Hypothesis. When we plug
our variables into our original theory we get our
research hypothesis.
Simply stated, Students with one or more
years of prior work experience will receive higher
final course averages than students with less than
one year of prior work experience.
Since statistical analysis often tests the
null hypothesis or the idea that there is no
difference between groups, our null hypothesis could
be stated as: Final course averages of students with
one or more years of prior work experience will not
differ from final course averages of students with
less than one year of prior work experience.
Step 4: Standardization. To make sure
that each subject, no matter which group they belong
to, receives the same treatment, we must standardize
our research. In
our case, we are looking at final course averages so
we must make sure that each student receives the
same instruction, the same textbook, and the same
opportunities to succeed.
While this may be difficult in the real
world, our goal is to get as close as possible to
the ideal.
Therefore,
we may choose to gather subjects from a general
psychology class since this is a class required of
most students and will not be affected by college
major. We
may further decide to research only those students
who have a specific instructor to keep the
instruction between the two groups as similar as
possible. Remember,
our goal is to assure, at least as much as possible,
that the only difference between the two groups is
the independent variable.
Step 5: Selecting
Subjects. Because our
population consists of all college students, it will
be impossible to include everyone in the study.
Therefore we need to apply some type of
random selection.
Since we want to use only those students who
have the same instructor, we may ask all of this
instructors students, prior to any teaching, how
much work experience they have had.
Those who report a year or more become the
potential subject pool for group 1 and those who
have less than one year become the subject pool for
group 2.
We could, at this
point decide to include all of these subjects or to
further reduce the subjects randomly. To reduce the subject pool we could assign each student in
each group a random number and then choose, at
random, a specific number of students to become
subjects in our study.
For the purpose of this example, we will
randomly choose 20 students in each group to
participate in our study.
Step
6: Testing Subjects.
Since we are not applying any type of
treatment to our subjects, this phase in the
procession can be omitted.
If we were determining if the teaching styles
of different instructors played a role in grades, we
would randomly assign each student to a teacher.
In that case, teaching style would become an
independent variable in our study.
Step 7: Analyzing
Results. Our original
question asked if final averages would be different
between our two groups.
To determine this we will look at the mean of
each group. Therefore,
we will add up the averages of the 20 subjects in
each group and divide each of these by 20
(representing the number of subjects in each group).
If, after comparing the means of each group,
we find that group 1 has a mean of 88 and group 2
has a mean of 82 then we can descriptively state
that there is a sixpoint difference between the
means of the two groups.
Based on this statistic, we would then begin
to show support for our alternative hypothesis and
can progress to the next step.
Step 8: Determination of
Significance.
Our goal was not to describe what their
averages were, but rather to make inferences about
what is likely happening in the entire population.
We must therefore apply inferential
statistics to our results to determine the
significant or lack of significant findings.
We will set our confidence level at 95
percent and then apply statistical analysis to our
results to see if the difference of six points with
a sample size of 40 is significant.
Imagine
that we did find a significant difference.
In this case we could say that with a 95%
confidence level, students with one year of more
work experience receive higher averages than those
with less than one year of work experience.
Since the null hypothesis, which stated that
no difference exists between the two groups, was not
correct, we must reject it.
And by rejecting the null, we automatically
accept our alternative hypothesis.
Step 9: Communicating
Results. When
communicating the results of our study we need to do
several things.
We need to make a case for why we did this
research, which is often based on our literature
search. We
then need to report the process we took in gathering
our sample and applying the treatment. We can then report our results and argue that there is a
difference between the two groups and that this
difference is significant enough to infer it will be
present in the entire population.
Finally,
we must evaluate our research in terms of its
strengths, weaknesses, applicability, and needs for
further study.
In terms of strengths, we might include the
rigors of gathering subjects and the fact that we
used a random sample of students.
We may argue that the statistical methods
used were ideal for the study or that we considered
the recommendations of previously completed studies
in this area. Weaknesses
might include the small sample size, the limited
pool from which our sample was gathered, or the
reliance on selfreported work experience.
To
discuss applicability and needs for further studies
we could suggest that more studies be completed that
use a broader base of subjects or different
instructors. We could recommend that other variables be investigated such
as student age, type and location of college, family
educational history, sex, race, or socioeconomic
background. We
might even suggest that while our findings were
significant they are not yet applicable until these
other variables are investigated.
The sections of a research report and how to
write this report in order to communicate results is
the main focus of chapter 2.
Step 10: Replication. The final
step in any research is replication.
This can be done by us but is most often
completed by other researchers based on their own
review of the literature and the recommendations
made by previous researchers.
If others compete a similar study, or look at
different variables and continue to find the same
results, our results become stronger.
When ten other studies agree with ours, the
chances are greatly improved that our results were
accurate. If
ten other studies disagree with our findings then
the validity of our study will be, and most
certainly should be, called into question.
By
replicating studies and using previously gained
knowledge to search for new answers, our profession
continues to move forward.
After all, we used the ideas of other
researchers to design our research, and future
researchers may incorporate our findings to make
recommendations in their research.
The cycle is never ending and allows for
perpetual seeking of new knowledge.
