Introduction
It has been standard wisdom among academic professionals for several
decades that educational standards in post-secondary institutions are
declining: grade inflation, fewer written assignments, and less
demanding syllabi are aspects of the typical complaint. Students, too,
have come in for questioning. Their preparation for college work on a
number of levels has been found wanting.2
For example, see Hibbing and Theiss-Morse (1996).
At the post-secondary level, however, a gap remains in the literature of
our discipline: what about the curriculum? Most of the attention to this
question has been directed at the technicalities of teaching theory and
empirical knowledge (
Cushman 1993;
Jordan and Sanchez 1994;
Luna
and McKenzie 1997;
Opheim and Stouffer
1997). However, the substance of the curriculum itself—the
textbooks that most of us use at least in our introductory-level
courses—have not received a full examination. From the past 15 years,
several articles have been published in this journal on various aspects
of textbooks as teaching tools, including their prejudices, their
effectiveness, or their ideological content (
Goldstein 1985; Sanchez 1996); there has also been a
puzzling lacuna.
There is a quiet but oft-repeated suspicion in the profession that
textbooks are not what they used to be—they do not have as much
substantive or theoretical content, they are increasingly video-driven,
and less space is dedicated to argument and information. We found
nothing from the most recent 15-year period of PS publications
that remotely addresses this question.3
While Stroup and Garriot (1997)
argue that the essential format of textbooks has
changed little since 1922, they say nothing about the character
of their contents.
Given common perceptions—correct or
not—about the deterioration of student ability (and there are articles
in recent issues of this journal about that topic), it is remarkable
that textbooks have not been subjected to a parallel scrutiny within our
profession.
There are many reasons to consider the change in quality of textbooks
over time: they are a core part of our instructional resources; writing
them is frequently considered a part of our “research” activity; and,
they are a primary way of introducing students (including potential
members of our profession) to the discipline of political science. While
there are many ways to measure change, we limit our focus in this
preliminary study to the accessibility of written materials, to the
increase or decrease in the readability of introductory texts over the
past several decades. Readability is one measure of accessibility,
making it a key factor in communicating ideas and ultimately in teaching
political science.
Analytical Strategy and Method
To determine readability levels, we used the Flesch-Kincaid Grade Level
Index. This index has been utilized in a variety of research fields, and
a large number of studies have confirmed its validity (Tefki 1987). The index is based on the average
number of syllables per word and the average number of words per
sentence.4
The formula for the
Flesch-Kincaid Grade Level score is: (.39 × ASL) + (11.8 × ASW)
-15.59 where: ASL = average sentence length (number of words
divided by the number of sentences) and ASW = average number of
syllables per word (the number of syllables divided by the
number of words).
The score provides a grade-school
level. For example, a score of 8.0 suggests that a median student in the
eighth grade would be able to understand the text. The index stretches
from below grade level four, which is considered very easy, to very
difficult reading at the post-graduate level.
Finding and scanning old textbooks is a time-consuming process: we limit
ourselves to a preliminary case study of the exceptionally durable and
wellknown introductory American government text by James MacGregor Burns
and Jack Walter Peltason, Government by the People. For
approximately 50 years, this text has been among the best selling in the
country. Government by the People was ideal for our purposes
because we could determine whether the same text has become easier or
harder to read across a significant span of time. We chose an American
government text, because those are the most frequently used texts in
political science and therefore the most likely to have multiple
exemplars of multiple editions.5
Goldstein (1985) counts 47 American
government texts, but he gives no information on
editions.
We sampled the texts in a fashion consistent with
Berland et al. (2001), who analyzed
reading grade levels of medical information web sites. Three passages of
text, exactly two pages each, from the beginning, middle, and end of
relevant chapters were sampled, then scanned and converted to Microsoft
Word documents. The Flesch-Kincaid Index is available in the “Spelling
and Grammar” options of Microsoft Word.
We determined readability levels for sample passages from the 1957, 1966,
1969, 1972, 1990, and 1995 editions (editions 3 through 16).6
Though sampling every edition of the text
since publication would have been preferable, obtaining all
editions proved a difficult task. Given the several editions we
did obtain, and given the time period the editions cover, we are
confident that our results reflect the general trend in
readability for this particular text.
We took samples
from three chapters across each of the editions, first selecting the
introductory chapter. Though not devoted to specific sub-field content,
the function of this chapter is to familiarize students with the various
topics of political science; the reading accessibility of its content is
clearly important. In addition, given its placement in the book and on
most syllabi, the introduction is arguably the chapter most widely read
by undergraduates.
The other two chapters we sampled concerned civil liberties and public
opinion. These two chapters offered needed variance on the important
dimension of sub-field development. We expected that as a sub-field
matures, its language is likely to become increasingly complex. In the
1950s, public opinion was in its infancy, whereas the study of
constitutional rights and liberties was well established within the
discipline. To the extent that readability mirrors sub-field
development, we would anticipate reading levels for civil liberties and
public opinion to differ initially, with civil liberties generating more
diffi-cult content, and then to converge as the field of public opinion
matured.
Findings
Figure 1 shows the average readability level
across editions. The trend is clear, striking, and apparently contrary
to the common charge of declining standards. Since the publication of
the third edition in the late 1950s, the reading level of Government
by the People has become increasingly higher. Difficulty has
increased by approximately a grade level, rising from just below an 11th
grade to nearly a 12th grade level. The magnitude of this change is not
trivial. The readability of chapter two in Harry Potter and the
Sorcerer's Stone is at a high 10th grade level and that of
specific articles in The New York Times is at a 12th grade
level (Biskupic 2001; Neilan 2001). Thus, the change in readability of
Government by the People over the past several decades is
approximately the same as the difference in readability between The
New York Times and the initial volume of the Harry
Potter series.
Figure 2 presents the data disaggregated by
chapter. Across all editions, the readability of the civil liberties
chapter has remained at grade level 12. By contrast, the readability of
the introductory and public opinion chapters exhibits significant
change. As expected, the readability of the public opinion chapter
starts substantially below that of the civil liberties chapter but
ascends rapidly during the 1970s to approximately 12th grade level by
the mid 1990s. The introductory chapter takes a similar path, although
its ascent is much steeper, moving to a 12th grade level by the 7th
edition in 1969.
To this point, the data appear to undermine the familiar and admittedly
attractive critique of declining texts and permissive pedagogy. Rather
than “dumbing down,” Government by the People has increased in
reading diffi-culty. We cannot, however, dismiss the charges of the
critics quite so easily. Though reading difficulty has increased,
communication extends beyond words and sentences. Perhaps an alternative
form of communication gives rise to the now pervasive “dumbing down”
charge. Pictures, after all, are worth a thousand words, and inspection
of our introductory texts suggests publishers have accepted the wisdom
of this old adage. Modern texts are filled with splashy photos,
graphics, tables of all sorts, and boxed inserts. This type of
presentation, more than the written word, may in fact trigger the
critics' lament of market-driven (i.e., dumbeddown) texts that are
attempting to appeal to wider, and hence less academically seasoned,
audiences.
In Figure 3, we present documented changes in
the number of graphics across the chapters. Anything not part of the
main text was included in the count, namely figures, tables, photos, and
boxes. Clearly, the early 1970s represented a significant change in the
form of communication. While previous editions had comparatively small
number of graphics, by the mid 1980s virtually every page contained at
least a picture, table, or figure. Studies are mixed on whether these
changes are merely cosmetic or provide pedagogical substance (Jordan and Sanchez 1994); the trend is
evident.
Discussion
To our knowledge, this case study is the first attempt to examine the
readability levels of an introductory textbook in political science. In
this case, we found that the level of readability has increased by
roughly a grade level from high 10th grade to nearly 12th grade. The
change occurred over several decades and varied by sub-field. The
introductory and public opinion chapters showed the most changes while
civil liberties remained at grade level 12 for the entire period. This
evidence suggests that readability levels are tied in part to sub-field
development, but the extent of this linkage remains an open question for
future analyses.
We also saw a dramatic increase in graphics and pictures in the
introductory text. Before the 1970s, a picture or graphic appeared
approximately once every five pages; our current text offers at least
one graphic per page, sometimes more. If this change is typical, it may
be an important cause for the general criticisms of current textbooks
compared to those of the fabled past, but combined with more difficult
reading levels, the claim of “dumbing down” may be hollow.
This case study has some important limitations, even as it raises some
important questions. First, the introductory textbook market is broad,
and we have sampled only one text. Changes in readability may be
different for other texts and conceivably different for introductory
texts in the comparative, international relations, or political theory
fields. While we hesitate to draw more general conclusions, our
selection of one of the more popular and enduring introductory texts in
American politics leads us to expect that our results reflect a more
general trend in the field. Second, we studied readability only,
utilizing one measure. Alternative measures may alter our findings.
Third, our research has not explored the causes or consequences of
changes in readability. Our primary goal was simply to document changes
in textbooks, and calculating readability levels represented a step
toward realizing this end. Finally, there are potential variations in
texts that this case study explicitly did not seek to explore.
One might ask how changes in the discipline affect readability and
complexity in our textbooks, and there may also be non-disciplinary
causes for variation having to do with marketing, editorial processes,
pedagogical culture, and the like, that the present study does not
address.
Several further research possibilities exist. For example, researchers
have examined introductory texts in psychology to determine whether a
set of core concepts define a common language in the field (Zechmeister and Zechmeister 2000). A similar
study of how political science texts treat key concepts such as power,
class, representation, and con-flict and whether they have changed over
time and across introductory texts would make a substantial addition to
the topic. Comparisons could also be made between brief and full
editions, analyzing, pedagogical aids, and topic coverage (Griggs and Koenig 2001). More generally, one
could explore relative changes in text length, number of chapters, and
coverage of traditional versus non-traditional topics (Griggs, Jackson, Christopher, and Marek 1999).
The questions are many and the topic is important.
If readability has, in fact, changed in the direction this study
indicates, researchers may wish to examine whether students are more
likely to process and absorb the newer textbook content. This question
has been examined in other disciplines (Santa and
Burstyn 1977). Although ours is a preliminary study, its
results should raise awareness of the need for studying political
science textbooks generally, at a minimum encouraging a more critical
eye in choosing our textbooks. The means by which knowledge is
communicated to students through our introductory texts remains a key
factor toward effective teaching, and the surprisingly sparse
consideration it garners in our discipline deserves expansion.
It should be mentioned that the Flesch-Kincaid Grade Level Index for this
note is 12.0 (high school level).
