part2

Several researchers have commented on the inadequacies of the body of research (not only those cited above) purporting to show positive benefits for vocabulary acquisition (among other issues) within foreign language reading. For example, Coady (1997:226), referring to oft-cited research, says that "there appears to be a serious methodological problem with these studies." Nation (1999:124) says that many studies "generally lacked careful control of the research design." Horst, Cobb and Meara (1998) also point out that some of the incidental learning from exposure experiments are "methodologically flawed" (1998: 210). Waring (in preparation), in a metaanalysis of some 28 studies of reading in a foreign language, also found that many of these studies lacked careful control.
Issues in study design This section will briefly cover some of the research design issues that are common in the body of research into vocabulary gains from reading addressed in this study.
RFL 15.2 – At what rate do learners learn and retain new vocabulary from reading a graded reader?
http://nflrc.hawaii.edu/rfl
133
Lack of retention data. It is quite rare in research into vocabulary gains from reading to ask how long these gains will last. The vast majority of the studies on learning from exposure are crosssectional tests whereby data are gathered only after the reading. To our knowledge, only one second language study has systematically attempted to gather data on how much learning was retained over time (Yamazaki, 1996). One problem with this is that new words will be fresh in the mind for an immediate post test, thus these scores will be higher than if the test were taken some time later. The real test of whether a word has been learned is whether the meaning of a word is retained over time. Following this, we could suggest that cross-sectional studies do not measure the subjects' real amount of learning from reading and therefore we should be wary of claims made only on immediate post test data because the gains would probably have been overestimated.
Depth of word knowledge. Further, it is rather safe to assume that broadly there are stages, levels or degrees of word knowledge. These could range from knowing only that you have seen or heard the word-form without being able to recall the meaning, to a full understanding of the word and its various nuances and use in a variety of contexts both receptively and productively. In most learning from context research (and most vocabulary research in general), only one test is given which means we can only look at one type of word knowledge gains. This is a shame because it limits us to a one-dimensional picture of what is happening as a result of the treatment. The problem for researchers attempting to create a meta-analysis of the results of many studies is compounded when different researchers use different test types.
Issues in instrumentation. Another problem with many previous studies concerns the types of test used to assess gains in vocabulary from reading in a foreign language. The level of difficulty of the test will have a significant effect on the amount of learning that can be demonstrated and therefore it will affect the gains that can be shown (Waring, 1999; Nation 2001). A test that allows the subject to demonstrate even a small amount of information about the word such as a simple word-form recognition test "have you seen this word before?" will be easier than one that demands that the subject demonstrate more detailed knowledge such as shades of meaning, or the differences from similar words. This implies that researchers who are collecting "learning from context" data should be aware that the type of test that is selected will have a great bearing on the apparent results. Most studies looking at gains from context when reading extensively have used a multiplechoice test (e.g., Day, Omura and Hiramatsu, 1991; Dupuy and Krashen, 1993; Pitts, White and Krashen, 1989). The complications associated with multiple-choice tests as accurate measures of word knowledge are legion and well documented elsewhere (e.g., Anderson and Freebody, 1981; Meara and Buxton, 1987; Wesche and Paribakht, 1996; Waring, 1999). This type of test is not necessarily the most suitable for assessing how much vocabulary has been learned for several reasons. Firstly, random guessing will affect gain scores. Secondly, multiple-choice tests only assess prompted meaning recognition, not the unprompted meaning recognition one needs for normal reading. Thirdly, multiple-choice tests are notoriously difficult to construct reliably. Therefore, it seems wise when conducting this type of research to use several different tests to determine what types of word knowledge are learned from reading.
RFL 15.2 – At what rate do learners learn and retain new vocabulary from reading a graded reader?
http://nflrc.hawaii.edu/rfl
134
The effect of frequency of occurrence on incidental word learning. Few studies (Horst, Cobb and Meara, 1998 is an exception) have looked at what types of words are learned in the reading. Usually a single figure is given that reflects the total number of words learned irregardless of whether the words learned words appeared frequently or not. This would be valuable data because if studies controlled for occurrence rate, we would be able to determine how many times a word needs to be met in the reading for it to be learned in fluent reading.
Research questions From the above, we can see that there is a need to discover how much vocabulary is learned from reading in a foreign language to answer the following research questions: A. How many new words are learned from reading a graded reader and retained over time? B. Are words that appear frequently in the text more likely to be learned than words which appear less frequently? C. At what rate are the words forgotten (i.e., how many of the words known at a previous test time were not known later)? D. Do different test formats yield different gain scores? A study was undertaken to answer these and other questions.