Discussion
In this section, the results will be discussed. An interpretation of the overall results will be
presented first. This will be followed by a detailed analysis of the data from each test and then
other related points.
Overall results
The overall results for research question A (How many new words are learned from reading a
graded reader over time?) show that the subjects were able to learn some new words from their
reading but the vast majority of the new words were not learned. Moreover, the subjects forgot
the vast majority of their words they read and learned. On average, the subjects learned (three
months retention of the unprompted meaning) one new word from one hour of reading. This is
rather a poor rate of return for the effort expended at least as far as the learning of new words is
concerned. Research question B asked whether words that appear in the text more frequently
were learned before words which appear less frequently. These data suggest that the subjects are
able to learn words from context, but they are more successful if they meet the word several
times. Research question C asked about the rate at which words are forgotten. The results
depend on the test format, but in the main, about half of the words learned in the reading are
forgotten after three months. Research question D asked whether different test types yield
different test scores, and they did.
Results from the tests
The word-form recognition test assessed the subjects' ability to learn new word forms during
their reading. The test did not require them to know the meaning of the word, but just to say if
they had recognized the form before. The mean score on the immediate post test of 15.3 of the
25 items dropped to 8.4 after three months (Table 4). This translates into a decay rate of over
45% over three months. On the immediate post test, words met more than 8 times were
recognized most of the time, but words met only four or five times were recognized only half of
the time (Table 5). However, by three months, nearly half of these gains in word-form
recognition were lost. There was also an increase in the number of test items selected in error
(Table 6). At the immediate post test 8.5% of the words said to be recognized were incorrect,
but by three months this was 28.5%. The standard deviations also increased as a percentage of
the means in a similar way. This seems to suggest that the subjects were less and less sure of
their knowledge as time passed and they were more willing to guess incorrectly.
The multiple-choice recognition test was a test of prompted meaning recognition.
Approximately 40% of the words were remembered at the immediate post test, but after three
months, this decreased to one quarter. In other words, of the 10.6 words that were learned, only
RFL 15.2 – At what rate do learners learn and retain new vocabulary from reading a graded reader?
http://nflrc.hawaii.edu/rfl
149
6.1 of them were retained, which is a decay rate of over 40%. At the immediate post test 3.6
(72%) of the 5 items at the 15-18 occurrence rate group were recognized when prompted, but
this dropped to 1.9 (38%) after three months. Words appearing eight or more times had a 54%
chance of being learned but a less than 40% chance of being retained after three months. About
one word appearing fewer than 8 times was recognized immediately after reading, dropping to
about half of that after three months (Table 6).
On the meaning (translation) test, the subjects had to write a Japanese translation for the test
item. At the immediate post test 4.6 (18.4%) of the 25 items were translated correctly, dropping
to less than one word (3.6%) after three months. This is remarkable considering the time
expended on reading. Only 42% of the words occurring more than 15 times were translated
correctly just after the reading. However, within one week this had dropped to 10%, and by 6%
after three months. Words met fewer than 15 times had little or no chance of being learned.
This suggests that meanings are lost faster than other the types of word knowledge tested here.
The rate of forgetting found here largely mirrors that found in memory research that shows that
most forgetting occurs soon after learning (Ebbinghaus, 1885 /1913; Baddeley, 1990).
The data from the three tests suggest that while some words were learned, the benefits of the
learning were soon lost especially when higher demands were placed on word knowledge (i.e.,
when meaning was not prompted). Approximately half of the word knowledge gained in the
reading was lost after three months. Importantly, the data seem to suggest that there is a critical
threshold of the number of times a word has to be met before learning can take place from this
type of reading, but it is not clear what this constitutes.
Implications for the type of test used
Research question A asked about the rate at which new items are learned. In natural reading,
learners have to be able to recognize words without being prompted (i.e., no definition is given
and there is no gloss in the margin). Thus, a new word can be said to be learned only when the
subject can connect the form of the word (its spelling) with its meaning. Thus, only the results of
the meaning (translation) test can validate whether learning, in terms of unprompted meaning
recognition which most closely resembles the knowledge needed for natural reading, has taken
place. The recognition of only the form in the word-form recognition test does not constitute
learning a new word, but may reflect the recognition of a new word form. Moreover, recognition
of the meaning when prompted on the multiple-choice recognition test is not considered to be
"learning a new word" in this sense either, but is a demonstration of simple prompted recognition
which does not occur in natural reading.
It is our contention, therefore, that a multiple-choice test is not the best way to assess learning
new words from context. Similar conclusions were found in Hulstijn (1992). These test types
might be better used as pedagogical classroom vocabulary activities than as research tools. It is,
of course, recognized that prompted word-form recognition and prompted meaning recognition
are important assessment points in the learning of a word. However, they do not show whether a
word is known in the sense of being able to make an unprompted form-meaning connection.
Having said that, much research in this area has tended to use multiple-choice tests (e.g., Day,
Omura and Hiramatsu, 1991; Dupuy and Krashen, 1993; Pitts, White and Krashen, 1989). As
RFL 15.2 – At what rate do learners learn and retain new vocabulary from reading a graded reader?
http://nflrc.hawaii.edu/rfl
150
we can see from the data in Table 4, the mean scores on the multiple-choice test are higher than
those on the meaning (translation) test. The immediate post-test showed that the multiple-choice
test scores were over two times (230%) higher than those of the meaning test scores (10.6
compared to 4.6). However, after three months, this had dramatically increased to nearly seven
times higher, or some 677% (6.1 to 0.9). This is rather worrisome for the interpretation of some
previous research in this area because this seems to imply that some of the reported vocabulary
gains from studies of extensive reading that have used multiple-choice tests may have been overestimated. The overall results here suggest that a multiple-choice test score will be double that
of an unprompted meaning test in terms of what constitutes a known word. This also implies
that the vocabulary gains found in the studies mentioned in Table 1 that used multiple-choice
tests should be halved to approximate unprompted recognition word knowledge scores.
Number of meetings needed to learn a word
As we saw in the introduction, previous estimates of the number of times it takes to learn a word
from reading varied considerably. It is clear from this research that it is very difficult to pin a
number on this age-old question. It seems much more complex than a simple single figure.
From the results of this experiment, it seems that to have a 50% chance of recognizing a word
form again three months later, learners have to meet the word at least eight times. Similar results
could be said for prompted recognition. However, for unprompted form-meaning recognition
(i.e., word learning) there is only a 10% to 15% chance that the word's meaning will be
remembered after three months even if it was met more than 18 times. If the word was met
fewer than 5 times, the chance is next to zero. This is rather disappointing because it suggests
that we do not learn a lot of new words from our reading even with a 96% coverage rate.
There are several reasons why this might be so. Firstly, the learners are presumably focused on
comprehending and enjoying the story rather than on the words themselves. The words were not
made explicit by bolding or highlighting the words in any way, as is the case in natural reading.
Because of this, the learners are not being forced to notice them and their awareness of the words
is not being raised. Some recent research has suggested the noticing of a form is an essential
step in word learning (Schmidt, 1990). The question for editors of graded reader series therefore
is whether we should highlight words in the text to ensure that certain key words are noticed. It
is still an open question whether highlighting the words would help vocabulary acquisition, but
many who believe that graded readers should look like books, not like textbooks, would object to
this proposal. However, recently some series of graded readers (e.g., Oxford University Press'
Dominoes, and Cideb's Black Cat series) have started to do this in their easier level Readers that
blend an intensive reading approach with an extensive reading one.
Secondly, the coverage rate in this experiment may have been too high. In other words, little
was learned despite the coverage rate being over 96%. It is important to remember that some of
the words were learned during the reading, which would have increased the coverage rate and
therefore it is not likely that the coverage rate would have been a problem. Moreover, the
majority of the subjects rated the book as either easy or not too difficult, thus comprehensibility
may have been high and the conditions for successful learning from context seem to have been
met.
RFL 15.2 – At what rate do learners learn and retain new vocabulary from reading a graded reader?
http://nflrc.hawaii.edu/rfl
151
Thirdly, the reason for low vocabulary rate retention may have simply been that there were too
few chances to learn the words. As we have seen, it takes much more than one meeting of a
word to learn it from reading. Moreover, even words met more than fifteen times in the text still
have only a 40% change of being learned. This seems to suggest that it would take well over 20
or even 30 meetings for most of those words to be learned. If a learner is reading at the 96 to
99% coverage rate as suggested above, and it takes twenty or more meetings with a word to learn
it as the data here seem to show, then learners will have to read several hundred or several
thousand words in order to learn one new word from their reading. This is a greater amount of
reading than is often recommended for foreign language learners reading graded readers.
(Nation and Wang [1999] for example suggest a "book a week at their own level of difficulty" as
an appropriate rate.) It should also be pointed out that the volume of text that would need to be
read to meet an unknown word increases with reading ability level. This is because rarer words
are met less frequently and thus more text has to be read to meet an unknown word the required
number of times. This also has implications for the amount of text that needs to be read.
Fourthly, the subjects may have found the learning of substitute words more difficult because
they may have already known the real English word forms prior to reading. This could
potentially mean that when they met a substitute word, they may have been confused because
they would have expected the already known word form, not a substitute. To determine if this
happened, the subjects were interviewed about the words they met. Many of the subjects said
that they were able to guess the meaning of words such as yoot (yes) and molden (dead) even
though they knew the real English word. Further investigation of this revealed that guessing and
comprehension were not slowed to any large degree because the real English word was already
known. In fact, some of the subjects reported that they assumed it to have a similar meaning to
the already known word, which aided their comprehension.
Individual variation
As one would expect from a study of this type, there is considerable individual variation in the
gain scores (see Appendices D, E, and F). This variation is also apparent in the increase in
standard deviation scores as a percentage of their mean over time. One possible reason would be
the speed at which they read. The faster readers could be assumed to have better language
ability. Therefore, the data were reanalyzed looking at the differences between subjects who
read quickly compared to those who read slowly. The subjects were broken into two groups –
those who finished under 60 minutes and those who finished over. On the word-form
recognition test's first administration both groups scored exactly the same score of 15.3. On the
multiple-choice test, the faster group scored 10.5 while the slower group scored a similar 11.0.
On the translation test, the scores were 4.8 and 3.7. Similar profiles were evident at the other
two administration times. The similarity of these figures suggests that the speed of reading does
not greatly affect the ability to guess new words from context.
Another reason may have been their ability level and this was also investigated. The authors
have known each of the subjects quite well, often for several years. Casual observations seem to
suggest that the subjects with above average proficiency for the group scored slightly higher on
some tests. However, not all those who appear to be lower ability in English did below average
on the tests, some in fact scored above average. This suggests that the variation may be a result
RFL 15.2 – At what rate do learners learn and retain new vocabulary from reading a graded reader?
http://nflrc.hawaii.edu/rfl
152
of the reading matching their preferred learning style rather than a manifestation of their ability.
This of course warrants further investigation.
Interview data
Although no comprehension test was given, the subjects reported that they had understood the
main part of the story and had high levels of comprehension. After the immediate post-test we
asked all subjects whether the story was easy to read, difficult or very difficult for them, and also
we asked what they thought of the story. The four subjects felt it was difficult and could not
enjoy it so much. These subjects got low scores, especially on the meaning (translation) test. By
contrast, those who rated the book as easy to read generally showed higher gains, but as
mentioned above, this was not always a consistent finding.
Difficulty or ease of learning
In Table 4 there is an obvious increase or bump in scores at the 8-10 occurrence rate for the
multiple-choice-choice recognition test and the meaning (translation) test. It appears that several
words in this group were easy to learn and retain. For example, in Group 8 to 10 the word jurg/s
appeared with the meaning of"year/s". In A Little Princess, jurg/s always appears after numbers
which may have contributed to a high learning rate. An analysis of the item by test data shows
that jurgs is consistently the highest rated word on both tests and at all three test times. Molden
(dead) rated almost as highly as jurg across the two tests and times. These two words account
for approximately 65% of all the correct suppliances at the 8-10 level and thus have
disproportionately been better learned than other words at that level. At the 15-18 level yoot
(yes) is also consistently well recognized on all tests and at all times. Yoot alone accounts for
approximately 58% of the total score for that level on the meaning (translation) test across all test
times.
This suggests two things. Firstly, that some words are easier to learn than others, but secondly
that the experimental data may have been compromised in some way. If we assume that words
like yes will be easier to spot than more conventional words like head, sun and face, then the
scores for these levels would have been much higher. Similarly because jurgs is easier to guess
than a conventional word because of its strong collocation with numbers, it also may have been
disproportionately easy to learn. To examine this, jurgs and yoot were removed from the data
analysis in order to more clearly see the rate of acquisition of more conventional content words.
The data were reanalyzed on the basis of a re-calculated mean score at each level based on the
mean for the remaining words. By doing so, the learning rate on the meaning (translation) test is
cut in half (less 47.5%). Thus, while yoot and jurgs can be said to be important words to some
degree, their ease of learning may have over-estimated the results for the learning of the
meanings of normal content words. This suggests that lower learning rates can be expected than
those presented here.
Also there are some words which lead the subjects to guess incorrectly. This may be because
subjects tried to connect the new form to information they already have, whether it is in English
or Japanese. The translation data suggest that some subjects misunderstood a word that sounds
