METHODOLOGY: WHAT IT IS AND WHY IT IS SO IMPORTANT
Copyright American Psychological Association
CHAPTER 1
METHODOLOGY: WHAT
IT IS AND WHY IT IS SO
IMPORTANT
Alan E. Kazdin
Scientific knowledge is very special. This knowledge
is based on the accumulation of empirical evidence
and obtained through systematic and careful observation of the phenomenon of interest. At a very
general level, the ways in which the observations
are obtained constitute the methods of science. Yet,
these methods can be considered at multiple levels,
including the principles and tenets they are designed
to reflect, a way of thinking and problem solving,
and concrete practices that scientists use when actually conducting an investigation. This book draws
on each of these levels because they work together
and make for good science and scientific research.
The purpose of this introductory chapter is to
convey what methodology is, why it is needed, and
the key tenets that guide what we do as scientists.
These foci may seem obviousafter all, everyone
knows what methodology is and why it is needed.
Perhaps so, but the answers are not all so obvious.
It is useful to give the rationale for what we do and
why because it provides the common base we as
psychologists and social scientists share with all
of the sciences. Also, that base underpins all of the
chapters that follow. Let us begin.
SCIENTIFIC METHODOLOGY
AND ITS COMPONENTS
Methodology in science refers to the diverse principles, procedures, and practices that govern empirical research. It is useful to distinguish five major
components to convey the scope of the topics and to
organize the subject matter.
1. Research design: This component refers to the
experimental arrangement or plan used to examine
the question or hypotheses of interest. It includes
fundamental issues related to who the participants
will be, how they will be assigned (e.g., randomly),
and the comparisons (various groups) included
in the study. Many different arrangements exist,
including those in which some experimental
manipulation is made (true experiments) or
groups are formed (observational study), by which
to evaluate differences in characteristics of interest.
2. Assessment: This component pertains to the
measurement strategies (e.g., self-report, neuroimaging) and the measures that will be used to
provide the data. There are many different types
of measures and multiple measures within each
type. Key issues related to assessment, such as
reliability and validity of the measures, are
pivotal to research.
3. Data evaluation and interpretation: This component encompasses all of the methods that will be
used to handle the datato characterize the
sample, to describe performance on the
measures, and to draw inferences related to the
hypotheses. Statistical significance testing is
dominant and the most familiar method used to
develop and evaluate data but, as later chapters
show, other methods are also used.
Methodological Issues and Strategies in Clinical Research, Fourth Edition, A. E. Kazdin (Editor)
Copyright ? 2016 by the American Psychological Association. All rights reserved.
3
Alan E. Kazdin
Copyright American Psychological Association
4. Ethical issues and scientific integrity: This multifaceted component includes a variety of responsibilities that the investigator has in the conduct
of the study and can encompass all of the other
components (e.g., design, data analyses, publication of findings). Ethical issues include multiple
responsibilities to participants (e.g., their rights
and protections) and adherence to the professional standards of ones discipline (e.g., ethical
codes). Scientific integrity includes responsibilities to the scientific community and the public
at large (e.g., transparency, accurately reporting
findings) and is also part of professional standards
and ethical codes. Before a study begins, proposals are usually required (e.g., by universities,
agencies) that discuss not only specifics of the
project (e.g., research design, assessment) but
also ethical issues and assurances that participant
rights are protected (e.g., scrutiny of the procedures for any untoward effects, informed consent,
protection of privacy).
5. Communication of research findings:
Communication of our work is key to building
the knowledge base, stimulating responses to our
work, and promoting and fostering new theory
and findings as we ourselves or others follow up
on the study we have described. Findings can be
communicated to other professionals through
many different venues (e.g., journal articles on
empirical studies, review articles, conference
symposium presentations, poster sessions).
Communication also includes the media
(dissemination of information to the public via
TV, radio, and the web). Communication of
findings has its own responsibilities and challenges, as discussed later.
I have divided methodology into these components in part to convey the breadth and depth of
the topic. There are books, courses, and journals
devoted specifically to each of these components.
As one example, psychological assessment is an
enormous topic encompassing models of scale
development, validation, the vast range of assessment modalities, and sources of artifact and bias
that can greatly affect data obtained from a measure.
Similarly, data analyses and the vast array of statisti4
cal models and analyses have their own courses and
journals. This book covers all five components and
does so in a way that underscores their integration
and interrelation. There are always more topics and
components of methodology one could add. For
example, the historical roots of science and science
and social policy are legitimate topics that could be
covered as well. Yet, in developing an appreciation
for methodology and the skills involved in many of
the key facets of actually conducting research, the
five will suffice.
WHY DO WE NEED SCIENCE AND
ITS METHODS AT ALL?
Rationale
I have already mentioned the components of
scientific methods, but now let us step back a bit.
Why do we even need methodology in general
and its components? Four reasons can make the
case for why we need science and the methodology
of science. First, we need consistent methods for
acquiring knowledge. There are many sciences,
and it would be valuable, if not essential, to have
principles and practices that are consistent across
them all. We would not want the criteria for what
counts as knowledge to vary as a function of quite
different ways of going about obtaining that knowledge. This consistency is more important than
ever today, because much of research on a given
topic involves the collaboration of scientists from
many different fields and many different countries
to address a set of questions for a given project.
Scientists from many different areas must speak the
same methodology language, share the same underlying values about how to obtain knowledge, and
agree on procedures and practices (e.g., statistical
evaluation, reporting data that do and do not support a particular hypothesis). Consistency is also
critical within any given scientific discipline. For
a given science (e.g., psychology), we would want
consistency throughout the world in the standards
for obtaining scientific knowledgethe accumulation of knowledge from all individuals in a given
field requires this level of consistency. Science says,
essentially, these are our goals (e.g., describe; understand; explain; intervene when needed, possible,
Methodology: What It Is and Why It Is so Important
Copyright American Psychological Association
and desirable) and these are our means (use of theory,
methodology, guiding concepts, replication of
results). Science is hardly a game because so many
of its tasks and topics are so seriousindeed, a
matter of life and death (e.g., suicide, risky behavior,
cigarette smoking). Yet there are rules and there are
enormous benefits to be gained by all sciences and
scientists. Think of the chaos if methods varied
across countries or professions; we simply could not
accumulate an agreed-on body of knowledge.
Second, methodology is needed to identify,
detect, isolate, and reveal many of the extremely
complex relations that exist in the world. Science
uses special controlled arrangements and special
methods (e.g., equipment, measures) to isolate
influences that are otherwise difficult, if not impossible, to detect from casual observation in everyday
life. Consider a brief sample of findings from the
natural and social sciences conveying the complexities of our world that the methods of science were
needed to reveal. Consider the guiding question in
the examples and the answers that scientific method
provided:
What is near the boundary of our universe? Well,
for starters, a galaxy (a system of millions of stars
held by gravitational attraction) has been identified that is more than 13 billion light-years away
(e.g., Maartens, 2013).
How did dinosaurs become extinct?
Approximately 66 million years ago (give or take
300,000 years), a huge asteroid (15 kilometers,
or more than 16,400 yards, wide) crashed into
the earth (near Yucatan, Mexico) and led to the
extinction of more than half of all species on the
planet, including the dinosaurs. The material
blasted into the atmosphere led to a chain of
events that resulted in a global winter (e.g.,
Brusatte et al., 2014).
Are male and female interactions and behaviors
influenced by a womans menstrual cycle? Where
a woman is in her menstrual cycle apparently
has an effect on her behavior (e.g., selection of
clothing, gait when walking, and the type of man
that seems attractive) and how men respond to
it. All of this occurs outside of consciousness but
conveys dynamically changing interactions influ-
enced in part by ovulation cycles (e.g., Haselton
& Gildersleeve, 2011).
When prisoners come before a parole board, are
there any unexpected influences on the decision of whether they can be released before
their prison sentence is complete? Surprisingly,
the point during the day at which a given prisoner sees the parole board is relevant to the
outcome. An evaluation of multiple parole
decisions revealed that the likelihood of being
granted parole is much higher in the morning
and immediately after a lunch break than at
other times (Danziger, Levav, & Avnaim-Pesso,
2011). Indeed, as hunger (or fatigue) increases
and as lunch time approaches, the chances of
being paroled decrease, but they bounce up again
right after the lunch break. The same raters were
involved, and the result cannot be explained by
severity of the crimes or types of prisoners.
Do early harsh environments for children (e.g.,
exposure to violence, enduring stress, corporal
punishment) have any long-term effects? Yes,
they can lead to many untoward outcomes,
including poor academic performance (e.g., poor
grades, dropping out of school) and mental illness (e.g., posttraumatic stress disorder, depression, anxiety). Also, the outcomes can include
enduring impairment of the immune system
(ability to ward off infection and inflammation)
and are likely the reason why many such children have premature deaths from serious disease
much later in adulthood (e.g., Krug, Dahlberg,
Mercy, Zwi, & Lozano, 2002).
The findings in these examples required very
special observation procedures under special
arrangements, measures, assessments, and methods
of data evaluation. The conclusions I list are not
discernible by everyday observation. If you said, I
knew all along based on my casual observations that
there was a galaxy at the boundaries of our universe;
whats the big deal? or Of course prisoners who
are seen after the parole boards lunch break are
more likely to be granted parole, you are among
a very elite group. The rest of us needed careful
research and scientific methods to grasp these
phenomena!
5
Alan E. Kazdin
Copyright American Psychological Association
Third, whether the relations are complex or not,
for many questions of interest extensive information
(a lot of data) are needed to draw conclusions. How
to obtain that information (assessment, sampling)
requires very special procedures to yield trustworthy
results. For example, how many individuals experience some form of psychiatric disorder? To answer
this question, one needs a large sample, a representative sample, and special procedures (e.g., use of
measures known to provide consistent information
and to reflect the phenomenon of interest). As it
turns out, approximately 25% of the U.S. population at a given point in time meet criteria for one or
more psychiatric disorders (Kessler & Wang, 2008).
Approximately 50% experience a disorder at some
point in their lifetime. This kind of information
cannot be obtained from casual observation or individual experience. Large data sets and systematically
collected data are needed to address many questions, and science is needed to provide the information in a trustworthy, consistent, transparent, and
replicable way.
Finally, we need science to help surmount the
limitations of our usual ways of perceiving the
environment and reaching conclusions. Along with
these limitations in our perceptions, there are many
sources of subjectivity and bias that interfere with
obtaining more objective knowledgethat is, information that is as free as possible from subjectivity
and bias. How we perceive and think is wonderfully
adaptive for handling everyday life and the enormous challenges presented to us (e.g., staying out
of danger, finding mates and partners, rearing children, adapting to harsh and changing environments,
meeting the biological needs of ourselves and our
familyit is endless). Evolution spanning millions
of years has sculpted, carved, sanded, and refined
these skills. Yet those very adaptive features can
actually interfere, limit, and distort the information
presented to us and do so by omission (our perception omits many facets of experience that we do not
detect well) and by commission (we actively distort
information on a routine basis). Scientific methodology has emerged in part to surmount the limitations
of more casual observation.
That said, a few limitations are worth noting.
Science does not get rid of these limitations. Rather,
6
methodological practices are designed to help manage and overcome them.
Brief Illustrations of Our Limitations
in Accruing Knowledge
Senses and their limits. The limitations of
our sensesincluding vision, hearing, and smell
serve as a familiar example to convey how very
selective we are in the facets of reality that we
can detect. We consider what we see, hear, and
smell to represent reality, that is, how things are.
But this reality is very selective. For example, we
see only a small portion of the electromagnetic
spectrum and refer to that as the visible spectrum.
Probably a better term would be the human visual
spectrum. We cannot see infrared, or ultraviolet,
for example. Other animals (e.g., birds, bees and
many other insects) see part of the spectrum we do
not see, which helps with their adaptation (e.g.,
identifying sex-dependent markings of potential
mates that are only visible in ultraviolet light).
The same is true for sounds and smells; many
nonhuman animals have senses that evaluate different parts of the world from those we can experience. Many animals can hear sounds that we do
not hear (e.g., dogs, elephants, pigeons) and have
a sensitivity to smell that vastly exceeds our own
(e.g., bears, sharks, moths, bees). More generally,
many nonhuman animals trump our vision, hearing, and smell or have differences that are not
better (more sensitive) or worse but just different
(e.g., seeing different parts of the electromagnetic
spectrum).
These examples are intended to make one point:
As humans, we see one part of the world, and that
picture is quite selective. The picture we have of
what is omits piles of things that are. So one reason for science is to overcome some of the physical
limitations of our normal processing of information. Much of what we want to know about and see
cannot be discerned with our ordinary capacities
(our senses). In fact, much of what we have learned
about the universe and also about interpersonal
interaction and attraction comes from what is not
obvious, detected, or detectable by means of usual
sensory perception.
Methodology: What It Is and Why It Is so Important
Copyright American Psychological Association
Cognitive heuristics. Leaving aside physical
limitations in seeing, smelling, and hearing the
world, more persuasive arguments for the need
for science come from many areas of cognitive
psychology. These arguments are more persuasive
in the sense that when we look at experience well
within the capacities of our senses, we may still
have enormous limitations in how we process
that information. You already know the everyday
expression seeing is believing; psychological
research has provided considerable support for
the additional claim believing is seeing. We
process the world in special ways, and various
cognitive processes have been well studied. These
processes can and often do systematically distort
and lead us to make claims and inferences that
do not reflect reality, as revealed by less biased or
unbiased means.
Several characteristics of normal human functioning, referred to as cognitive heuristics, reflect
how we organize and process information. These
processes are out of our awareness and serve as
mental shortcuts or guides to help us negotiate
many aspects of everyday experience (Kahneman,
2011; Pohl, 2012). These guides help us categorize,
make decisions, and solve problems. The heuristics
emerge as bias when we attempt to draw accurate
relations based only on our own thoughts, impressions, and experience. There are several cognitive
heuristics, but let me convey a sample to make concrete what I am talking about.
The confirmatory bias reflects the role of our
preconceptions or beliefs and how they influence
the facets of reality we see, grasp, and identify.
Specifically, we select, seek out, and remember
evidence in the world that is consistent with and
supports our view. That is, we do not consider and
weigh all experience or the extent to which some
things are or are not true on the basis of the realities we encounter. Rather, we unwittingly pluck out
features of reality that support (confirm) our view.
This is particularly pernicious in stereotypes, as one
case in point. For example, experimental manipulation of ethnic characteristics (e.g., skin tone among
African Americans, ethnicity of victims in a crime)
leads to different evaluations of crime and sentencing
practices (e.g., Eberhardt, Davies, Purdie-Vaughns,
& Johnson, 2006). Objective facts about the material
presented can be carefully controlled in research
to allow demonstration of ethnic biases in how
participants react to stereotypes and biases they
would not otherwise express. More generally, if
we believe that one ethnic group behaves in this or
that way or that people from one country or region
have a particular characteristic, we will see evidence
that supports itthe supportive evidence is more
salient in our mind and memory and is constructed
rather than recording the incoming data objectively.
Counterevidence does not register as salient or, if
and when it does, is dismissed as an exception.
Cognitive heuristics are not the only set of influences that guide our perception. Our motivation and
mood states can directly influence how and what
we perceive of reality (Dunning & Balcetis, 2013).
Both biological states (e.g., hunger, thirst) and psychological states (e.g., mood) can directly guide
how reality is perceived. This is sometimes referred
to as motivated perception or wishful perceiving.
For example, when a person feels threatened or
angry, he or she is likely to see another as holding
a weapon rather than a neutral object (Baumann &
DeSteno, 2010). That is, the reality we perceive is
influenced by us as a filter, and our changing biological and psychological states have an impact on
what we see, hear, and recall. Obviously, motivated
perception can have life-and-death consequences
because the person perceiving (e.g., civilian, police
officer) feels threatened and acts accordingly. We
are not likely to be empathic when we hear a person shot someone else when in fact there was no
danger. The in fact may not have been so relevant
because the perception of the individual who fired
was guided by perceived threat. My comments are
not about blame or justification; rather, they are
intended to convey that reality is filtered and that
filter can be biased and influenced in ways quite different from the actual facts or events.
Memory. Other examples illustrate how our
normal processing of information influences and
distorts and, again, why we need assistance from
methodology to help surmount these influences.
Memory refers to the ability to recall information
and events, although there are different kinds of
7
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- design methodology a walkthrough
- design methodology ie edu
- design methodology i
- asic design methodology oregon state university
- system design methodology and tools cecs
- design methodology
- methodology what it is and why it is so important
- overview of design methodology
- an introduction to design thinking process guide
- the design method
Related searches
- why english is so important
- why it is important to businesses
- why traveling is so important
- why college is so important
- it is very important synonyms
- 3 reasons why college is so expensive
- why travel is so important
- why writing is so important
- why science is so important
- why learning is so important
- why it is important to be responsible
- why water is so important for us