Chapter 3

CHAPTER 3 Methodology
3.1. Participants
The participants in this study were the students of the 12th grade in an Islamic
public senior high school in a rural area in West Sumatra, Indonesia. In addition to
English, the students also study Arabic as a foreign language, the latter reflecting the
Islamic curriculum in the school.
In total 98 students participated in the study, 29 males and 69 females, and have
learned English for about 5 years. They belong to two different programs of study, Social
Sciences (N=49, 14 males and 35 females) and Natural Sciences (N=49, 15 males and 34
females). The students were grouped into the programs based on their own interest and
their average grades for the first and second semesters of the 10th grade. Students in the
Social Science typically have high marks in economics, sociology, history and geography,
while Natural Science students are required to have high marks in math, physics,
chemistry and biology.
English was not a factor in the students’ program selection. However, English is
one of three subjects examined in a national final exam in the last semester of the 12th
grade for all students. It is also a requirement for university entrance for both programs.
Therefore, English is an important part of the curriculum for both Social and Natural
Science programs, and is taught in the two programs for four and six lesson hours (one
lesson hour is approximately 45 minutes) per week in the 11th and 12th grades,
respectively.
19
The English proficiency level for the students is based on the grades obtained in
the end of semester test. The test is done at school level, but the education department at
regional level provided the test papers. A group of teachers, selected as schools’
representatives in the regency, developed the test. The exams typically assess reading,
writing, listening, speaking and grammar proficiency. Based on test performance in the
last semester at the 11th grade, there were different results of the test across the two
groups, the students from the Natural Science program performed better than the students
from Social Science (the averages are 72.67 and 66.30 respectively). In addition to this
measure, the participants’ English proficiency level was also assessed by a Vocabulary
Levels Test (Nation, 2001) which was administered as part of the data collection.
3.2. The Testing Instruments
Nation's Vocabulary Levels Test (VLT) was used to measure the students'
receptive vocabulary. This test was chosen because it is commonly used by other studies
and it is easy to administer and score. Since the participants were senior high school
students, the receptive vocabulary levels test used tested the participants’ knowledge of
vocabulary items from the 2000, 3000 and 5000 most frequently occurring words. These
are seen as words that all learners need to know to read basic texts and that should be
concentrated on in class (Nation, 2001). Each level had ten items containing six words
and three meanings. The participants must choose the right word to go with each meaning.
The following are the instructions and a sample item (see appendix B, Part A).
20
This is a vocabulary test. You must choose the right word to go with each meaning. Write
the number of the word next to its meaning.
1 business
2 clock ____ part of a house
3 horse ____ animal with four legs
4 pencil ____ something used for writing
5 show
6 wall
The learners’ morphological knowledge was tested by the Morphological
Awareness Test presented in Chang et al. (2005). This test consisted of two parts: a
Morpheme Identification Awareness test and a Morphological Structural Awareness test,
which are discussed below. The purpose of the first test was to verify the participants’
ability to distinguish homophones (words that have same pronunciation and possibly
same and/or different spelling), while the aim of the second test was to check if the
participants could demonstrate their ability to combine morphemes in a productive
manner.
The Morpheme Identification Test consisted of one example and five test items.
All the items have two pictures and the subjects were asked to look at the pictures and
then answer the question by choosing the correct picture containing the meaning of the
target word. The following is the instruction and an example (Chang et al., 2005, p. 430-
431).
Instruction
There is one example item and five test items. All the items have two pictures. Look at
the pictures and then answer the question by choosing the correct picture containing the
meaning of the target word.
Example:
There are two pictures for the example item, which means “the letter T” and a “teacup,”
respectively.
21
A. (1) (2)
Which contains the meaning of the “tea” in “tealeaf”?
Only five items were used because of the lack of time and resources to get the
appropriate pictures best presenting the homophones.
The Morphological Structural Test consisted of twenty scenarios illustrating
objects that are regularly encountered in daily life, as well as some which are rarer. As
was the case with previous test, the items were adapted from Chang et al. (2005). The
participants were asked to come up with words to describe the objects or concepts
presented by each scenario. Fourteen of the scenarios required responses involving
morpheme compounding, whereas the remaining six items involved syntactic
manipulations. One example of the compounding items was this: Early in the morning,
we can see the sun coming up. This is called a sunrise. At night, we might also see the
moon coming up. What could we call this? The correct response for this item is moonrise.
An example for manipulating grammar was this: John is stotting. Yesterday he did this.
What did he do yesterday? The correct response for this item is stotted.
However, there was an important change made to the two tests in this present
research. The scenarios for each item in the present study were in written form, not
presented orally as in the previous study (Chang et al., 2005). There are two reasons
behind this change; firstly, practicality in administering the test and validity of results.
This is because teachers at the school conducted the test; neither the researcher herself
nor English native speakers. It was practical for the teachers to administer the test by
simply delivering the test papers to the students; moreover, an oral test given by a non
22
native speaker may skew the result. Secondly, the participants were EFL senior high
school students, not L1 children; therefore, their exposure to English is mainly through
reading, not oral interaction.
Finally, in order to gain a better understanding of the participants’ perceptions of
the tests and their vocabulary learning, specific questions were included at the end of
each test (VLT, Morpheme Identification and Morphological Structure Awareness test)
and more general questions were given after completion of all three tasks. The questions
were created by the researcher to investigate the participants’ perception of their English
vocabulary knowledge, as well as their reactions to the morphological awareness tasks.
These questions were asked to provide a cross-check of what the test results show and to
ascertain the participants’ ideas about morphological awareness:
1. Which do you find easier? Morpheme Identification Test (Part 1) or
Morphological Structure Test (Part 2)?
For all three tests, instructions and examples were written in Bahasa Indonesia,
but the content of the tests was in English. On the other hand, all survey questions were
in Bahasa Indonesia in order to ensure that the participants would understand them.
Participants were instructed to respond in Bahasa Indonesia as well, so that they could
express their ideas and thoughts more easily.
3.3. Procedure
Before administering the tests, the researcher contacted one of the English
teachers at the school and explained the purpose of this study. The teacher then contacted
the principal to ask for his approval. After getting his approval, the tests were administered
to students at the beginning of their first semester in the 12th grade. The participants
23
completed the tests individually. The project received ethical clearance from the
University of Queensland Medical Research Ethics Committee (MREC) and Behavioural
& Social Sciences Ethical Review Committee (BSSERC).
The test consisted of three parts. Part 1 was Nation’s Levels Test (VLT - 90
items). Part 2 was the Morpheme Identification Test (5 items) and Part 3, the
Morphological Structural Awareness Test (20 items). The participants also completed a
10-question survey about their perceptions about vocabulary learning and morphological
awareness. The test was administered over two days to minimize fatigue. The first day of
testing consisted of the VLT, and two questions asking about the participants’ perception
of the test and their judgement on their vocabulary level. The second day of testing
included the morphological awareness test and questions on their view about the concept
of morphology and their beliefs on their vocabulary learning in general. The participants
received instruction for each part only on the day the particular test was done and they
were allowed to complete the tests in two hours on each test day.
3.4. Data Analysis
To answer research question 1, which investigates the vocabulary size of the
participants and compares the difference between groups, the results of the VLT were
summarized by mean frequency and standard deviation across the three different levels
(2000, 3000 and 5000). The scores obtained were added to get the total scores of the
three levels. In order to highlight the differences in the vocabulary knowledge that was
employed by each group of participants, the results of all the participants in total and the
separate results of each group (Social Science and Natural Science) were compared. Also,
24
a one-way ANOVA was performed to see if there is a statistical difference between the
vocabulary size of students from the Social and Natural Science programs.
The results of the morphological awareness test were also analysed for the mean
and standard deviation for the two parts (Morpheme Identification and Morphological
Structure) and the group as the whole. Independent group t-tests were carried out to see if
the groups mean differences were significant. The data were then analysed to emphasize
the difference between the two parts in light of research question 2. In order to ensure
reliability, all tests were scored twice, once by the researcher and once by a colleague.
In light of research question 3, for each participant, the correlations between both
the VLT total score and Morpheme Identification and the VLT total score and
Morphological Structure were analysed. These correlations highlight the relationship
between the vocabulary size and the morphological knowledge of each participant.
Moreover, the answers to the perception survey were analysed to investigate the
participants’ perception of the Morphological Awareness test as well as their interest in
applying these strategies for their future English vocabulary learning.