Method

Method
Overview In this study, 25 words that appeared with different occurrence frequencies were selected from the graded reader, A Little Princess, and were changed into substitute words. The subjects read the book and were tested on their recall of the words on three types of test, over three test periods.
Participants Fifteen 19 to 21 year old Japanese female subjects from a university in Western Japan were the subjects in this experiment. All the participants volunteered to take part. Almost all the subjects were members of the university's English Club. Twelve of them were English majors but all the subjects were at the lower-intermediate level or above. This was determined by their class assignments and ongoing teacher evaluations.
Materials In order that learning can take place, there should be a good balance of known and unknown words. If the text is too difficult, successful guessing will be hard to achieve. When constructing research of this nature, several points have to be kept in mind for successful
RFL 15.2 – At what rate do learners learn and retain new vocabulary from reading a graded reader?
http://nflrc.hawaii.edu/rfl
135 guessing to occur. Firstly, it is well known that until the learner reads at a very high level of text comprehension and text coverage (i.e., one unknown word in 50 or so) little new vocabulary can be guessed from context (Liu and Nation, 1985; Laufer and Sim, 1985; Bensoussan and Laufer, 1984; Hu and Nation, 2000). The optimal rate seems to be between 96 to 99% coverage of known words. Secondly, learners need to meet an unknown word many times before it is learned (Nagy, Herman and Anderson, 1985; Nagy, Anderson and Herman, 1987; Shu, Anderson and Zhang, 1995; Nation 2001: 237). This rate appears to be about ten to fifteen times or more but depends on the word itself and many other factors. One way to achieve the desired coverage rate would be to use a text the subjects would normally meet in their level studies. Several words could then be selected from this text and tested after reading. This presents two problems. Firstly, we would have to know that the words were indeed unknown prior to reading which would mean a pre-treatment intervention test of the chosen items. However, doing this would highlight the selected words for the subjects, which might compromise the study. The second problem is that we cannot be sure that the non-test items (i.e., the surrounding co-text) were all known. If we selected say 40 items to test, there may be another 40 items the subjects did not know. Eighty unknown items in a text would make the text rather lexically dense in terms of unknown words, thus lowering the known/unknown coverage rate. These two points mean that the text would have to be screened to determine that it met the appropriate rate of 96-99% before reading. The preferred alternative framework we selected was to use a graded reader which would be very easy for the subjects, but introduce some test items into the easier text. For example, a 400 headword graded reader should be easy for intermediate subjects to read and should present no great problems lexically. In this way, we could be reasonably assured that the surrounding cotext for the test items would be known and we can thus look to see what rate of acquisition takes place based solely on the test items. Twelve graded readers were selected as candidates for this study. Each book was scanned and converted into digital text and then analyzed by computer for the frequency of occurrence of its words. The aim of this was to find a text with a suitable range of occurrence rates which would be easy to read for level subjects. In the end, A Little Princess was selected as it met the criteria (see below). A Little Princess text is one of Oxford University Press's graded readers Level 1, and has 400 different headwords. We then had to decide how to present the test items within the surrounding easy text. One choice was to use synonyms for the test items. For example, curtains could have been changed to drapes and so on. Unfortunately, this raised two potential problems. Firstly, the synonym might already be known, and we would have to introduce a pre-test of the items to ascertain this, which again may prejudice the test. Moreover, it would have caused us problems if some of the subjects knew some words but not others. The second problem with using synonyms is that not all words have synonyms. For example, there are no obvious synonyms for our test items year or snow. We were not able to select different test items because A Little Princess was the only one of the twelve graded readers that were analyzed that had a decent spread of test items according to our occurrence rate criteria (see below). Furthermore, the use of synonyms would
RFL 15.2 – At what rate do learners learn and retain new vocabulary from reading a graded reader?
http://nflrc.hawaii.edu/rfl
136 often require a change at the phrase or sentence level to accommodate the relevant collocational and colligational changes, which may compromise the comprehension of the text. To solve this problem it was decided that we should change the spelling of the test items. These words are henceforth called substitute words. It is important to note that these are not nonsense words as they are sometimes referred to in the literature. A nonsense word would be one coined for learning that did not previously exist, for example the word for an imaginary six-legged horse, or a yellow and black striped tomato. The use of substitute words refers to the change in spelling of an already known, very common concept. There is no reason why (apart from largely unknown historical reasons) the symbol in English for the large glowing yellow object in our sky should be called sun or blund or smalt or indeed any other letter combination. As words are symbols of meanings, a change in the symbol (its spelling), provided it conforms to normal spelling conventions, has face validity. The main advantage of a change in spelling is that it allows us to ensure that the words would not be known before reading (although the word's concept would be). A further reason for changing the spelling of the words was to ensure that the words would not be met after the reading in the subjects' normal studies. If they had met the words later, it would have affected their recall on the delayed post-tests. The substitute words were constructed to look like plausible English words and take on English spelling conventions. For example, we changed house into windle, yes into yoot, name into
parrow, week into prink and so on. These words had already been tested for plausibility by native speakers (Waring, 1999). The substitute words were also checked by five second language learners, who were not part of the experiment, to ensure that that they could pronounce them fairly well so it would not slow their reading. Implausible words, and words difficult to pronounce, were discarded. The substitute words were not highlighted in the text by making them bold, colored or underlined in any way, and were left unmarked for natural reading. We ensured that the words fitted smoothly into the text, which on occasions meant making some words plural by using the unmarked "add an s" rule. No definitions or glosses were given at any time. In order to get reasonably reliable data, we needed to test at least 25 words which the subjects would have to guess from context. And, in order to answer research question B, we needed to select words of differing frequencies of occurrence. However, we also needed to decide what types of words we should select. Nouns and adjectives were chosen because they are generally easier to guess than adverbs. Verbs were not selected because they appear with their inflections which can make it difficult to decide whether the word is "known" and to determine how frequently the word has occurred. After looking at the occurrences of words in A Little Princess, we made six groups of words. These were a group of 5 words appearing one time, a group that included five words appearing 4 to 5 times, and so on for an 8 to 10 group, a 13 to 14 group, the 15 to 20 group and the 21 to 31 group. The sum of the number of occurrences of all the words in the 6 categories that need to be learned was 480. Unfortunately, this made for only 91.9% coverage as A Little Princess has only 5872 words (618 types). This meant that the chance of guessing would be low and would be
RFL 15.2 – At what rate do learners learn and retain new vocabulary from reading a graded reader?
http://nflrc.hawaii.edu/rfl
137 potentially too difficult for the learners and may affect the study as it did not meet the criteria of 96-99% set earlier. Therefore, we deleted the group of five test items which appeared 21 to 31 times in the text, and we were left with 221 occurrences for the 25 items in the remaining five groups with a "known words" coverage of 96.2% of the running words and a 96.0% coverage by types. In calculating this figure of 96.2%, we worked on the assumption that all the other words in the book would be known as it was at a reading level far below what learners of their ability should have been capable of. Clearly, this would not be true for all learners, and for all words, but was the best assumption we could make without having the learners read and underline every word they did not know before reading the text. It must be noted of course, that as the subjects read the book, many of the words will be recognized and learned as they read, thus the coverage rate will increase as they progress through the book. Following the discussion above, we could safely assume then that the learners would be reading at about an i+1 level, (i.e., at the 96% to 99% recommended by Hu and Nation (2000) above for successful guessing from context).