Implementing Keyword and Question Generation Approaches in Teaching EFL Summary Writing

Summary writing has been considered an important aspect of academic writing. However, writing summaries can be a challenging task for the majority of English as a Foreign Language (EFL) learners. Research into teaching summary writing has focused on different processes to teach EFL learners. The present study adopted two methods – keyword and question generation – to guide Taiwanese university students in writing summaries in English. To decrease the students’ apprehension resulting from the difficulties in writing summaries, portfolios were used as a vehicle to help the students collect and reflect on the articles they read and the summaries they wrote. This paper investigated how much keyword methods and question generation helped Taiwanese EFL university students improve their English summary writing. The data showed that, with the help of keywords and question generation, the two approaches helped the majority of the participants increase their English reading and summary writing abilities.


Overview
In recent years, summary writing has become an essential type of writing in which students read articles and summarize main ideas in their own words.However, Johns (1998) argues that traditional approaches to summary writing often ignore the processes involved and frequently overestimate the abilities of the students, assuming they can easily do what their teachers have taught them.The impetus for conducting the research reported here came from my second-year university students' poor performance on summary writing in their compulsory English Reading and Writing course in Taiwan.The majority of the students had difficulties in grasping the main points in an English article and paraphrasing them in short summaries, even though summary skills had been taught for several hours in class.
To improve and strengthen the students' summary writing skills and enhance their motivation, portfolio collection was adopted.In the last two decades, there has been a proliferation of research concerned with the usage of portfolio collection in L1 primary and secondary schools (Anderson & Bachor, 1998;Koretz, 1998;Clemmons et al., 1993;De Fina, 1992), and the research focus has usually been on the quality of information portfolios provide, i.e., the reliability and validity of the information (Hamp-Lyons, 1996).Recently, there has been a shift in attention to exploring students' perceptions and the benefits of using portfolios in English as a Second Language and EFL writing classrooms (Romova & Andrew, 2011;Aydin, 2010).Portfolios have proven beneficial to language learners' reading and writing abilities in L1 and EFL contexts.Therefore, in the present study, portfolios serve as a vehicle to help EFL learners in Taiwan collect and reflect upon their summary writing.It is hoped that the findings of the present study can provide teachers, researchers, or educators with evidence of the benefits of implementing the keyword method and question generation to enhance university students' summary writing ability in an EFL context.

Implementation of Keyword Method and Question Generation
Quality summary writing requires comprehendingthe text, selecting important information, combining ideas, and paraphrasing in one's own words.Additionally, writing a summary can be influenced by the genre students read.Campbell (1990), investigating how NS and NNS university students in the USA used information from reading texts in their own academic writing, suggests that process-oriented writing might develop students' awareness of academic texts and help them process summary writing.Chen and Su (2012) adopt a genre-based approach to teaching Taiwanese university students writing summaries of short novels, emphasizing the understanding of context in narrative essays.These two studies focus on the implementation of two different approaches to teaching students to address specific types of reading genres.In contrast, the present study does not aim to develop summary writing approaches to reading specific articles in English, but rather to help EFL learners learn to write appropriate summaries from a variety of articles.
Research in L1 settings has highlighted a number of individual strategies that have been shown to have a significant impact on reading comprehension: (a) activating prior knowledge, (b) using mental imagery, (c) using graphic organizers, (d) establishing text structure, (e) monitoring comprehension, (f) answering questions, (g) generating questions, (h) practicing mnemonic skills, and (i) summarizing (Grabe, 2004, p. 51;Trabasso& Bouchard, 2002).These nine reading strategies are equally applicable to encouraging comprehension as part of foreign language learning.In particular, the strategies of question generation and summarization demand not only cognitive, but also metacognitive strategies (De Fina, 1992).Silva and Matsuda (2002, p. 255) note that summarizing ideas from an article facilitates learners' metacognitive thinking and development in learning a language, and the six journalist's questions, the so-called '5W1H' (who, what, when, where, why, and how), are a common approach to developing written text.In addition to question generation, the keyword method, which has been frequently applied to learning vocabulary, is closely related to developing mnemonic skills in second and foreign language learning (Sagarra & Alba, 2006;Fritz et al., 2006;Wang, Thomas, & Quellette, 1992;Pressley et al., 1980).Specifically, Sagarra and Alba, and Fritz et al. found that good keyword mnemonic skills can have a beneficial influence on vocabulary retrieval in reading and listening and on vocabulary acquisition.However, how muchkeywords benefit reading comprehension when related to the productive skill of writing, as required for portfolio collection, remains unknown.Hence, it is worth exploring how much question generation and the keyword method facilitate reading comprehension in the context of summary writing.

Research Questions
Because learning English in Taiwan is compulsory from primary school to the tertiary level, using effective approaches to learning English reading and writing in academic contexts thus becomes essential to Taiwanese students.It is accordingly worth investigating how far the keyword method and question generation helped Taiwanese university students write summaries in English through portfolio collection.The two research questions in the present study were: 1) Did the students think keywords and question generation helped them improve their English summary writing ability?If so, in what ways?
2) In which aspect(s) did the students think their English reading ability improved after completing the reading portfolio collection?

Participants
The participants in the study were 111 Taiwanese second-year university students specializing in foreign languages, education, or business from a private university in Southern Taiwan.The participants were divided into two groups (one at a lower level of study and the other at a higher level) according to their English scores on the College Student English Proficiency Test (CSEPT) (Note 1) the previous year.The CSEPT scores for the participants with lower English ability (N = 57) ranged from 140 to 167; comparing their CSEPT scores to their CEFR level, these students were considered as being at the A2 Way Stage Level (The Language Training and Testing Center, 2009).The scores for the participants at the higher level (N = 54) ranged from 190 to 240, which put them at the B1 Threshold Level.As non-English majors, all the students were required to take (and pass) the English Reading and Writing course as part of their curriculum.

Research Design and Procedure
Before implementing the keyword method and question generation in this portfolio collection, the participants were asked to write a summary assignment from articles they chose (related to the topics in the course book).The assignment served as a pre-test.After the pre-test, the students were asked to choose and read eight English articles published in magazines, newspapers, or the Internet; the articles they chose had to be relevant to the topics in the course book.There was no restriction on article length, but it was suggested each should be at least an A4 page.After reading, they had to complete a worksheet for each article.The purpose of using the worksheet was to assess students' reading comprehension and to improve the summary writing skills that they had learned in class.The writing worksheet, which was designed by the researcher, consisted of three parts -three keywords, five out of six journalist's questions, and a summary.The first part required students to find three keywords from each article they read.Next, students needed to generate five comprehension questions based on the six journalist's questions from each article.In particular, the first and second parts were designed to systematically guide the students to produce the desired type of writing.At the last stage, the participants were asked to write a summary of approximately 100 words.

Data Collection and Analysis
The summaries were assessed by using a writing scale that was designed by the researcher on the basis of a text analysis of the participants' writing samples in class.The rating involved four equally weighted subscales: (1) content relevance, (2) coherence, (3) appropriate use of vocabulary, and (4) grammar and spelling (Appendix 1).Before the rating scale was used to rate the participants' portfolio summaries, it had been piloted by rating the summary assignments from another 42 students with a similar background to the participants in the present study.To ensure reliable intra-rater reliability, the researcher, as the scorer, first rated the summaries from the 42 students using the rating scale in Appendix 1 and then rated the same summaries again two weeks later.The scorer reliability coefficient obtained was .81,which was considered reliable for a classroom assessment.After the participants completed the eight pieces of summary writing in their portfolio collection, a t test was used to compare the participants' scores in the pre-test summary assignment and the average scores in the portfolio summary writing to see if there was any significant difference between the two sets of scores.
Semi-structured interviews were conducted in Chinese with the participants after they had completed their portfolios, and the interview data were translated by the researcher and checked by another teacher who speaks both English and Chinese.The reason for using interviews was that, if handled well, they can yield rich insights into respondents' experiences, opinions, and feelings toward predetermined questions.The advantage of using semi-structured interviews lies in the fact that the participants have the freedom to express their opinions in depth when necessary, while the event as a whole stays under the control of the interviewer or the researcher.The interview questions were the following: (1) Now that you have completed your portfolio collection, do you think your English reading ability has improved?And if so, in what ways?(2) Do you think the keywords and question generation have helped you improve your English summary writing ability?The worksheet and the interview questions were piloted with another ten students, and no problems were reported.The interviews were tape-recorded and the data transcribed, classified into similar categories, then analyzed.

Results and Discussion
Sixty-eight out of 111 participants (61.3%) agreed to be interviewed.Among the 68 interviewees, Participants 1 to 32 (32 students) were from the higher English ability group, while Participants 33 to 68 (36 students) were from the lower ability group.
In general, the students reported that using portfolios benefited their English ability in terms of vocabulary size, comprehension of context and grammar, reading speed, and general knowledge.Through extensive reading, 43 out of 68 interviewees (63.2%) from both groups reported that their vocabulary had increased and that they had learned to guess the meaning of new vocabulary from context.Specifically, 24 out of 36 participants (66.7%) from the lower English group, compared with 15 out of 32 (46.9%) from the higher group, said that highlighting keywords in articles had helped them read faster because they found it easier to grasp the main idea in each article.One participant said that he had learned how to read effectively and efficiently: I find that I read faster than before.Now I pay more attention to important and recurring words or phrases.Although I still encounter new vocabulary, I know how to get the main ideas quickly.It is not like our traditional exams where you have to understand every single word before writing down correct answers.(Participant 46, trans.)Twenty-two participants (61.1%) from the low English group, compared with 14 participants (43.8%) from the high English group, claimed that they found it easier to make connections between keywords and new vocabulary in context, when asked to provide three keywords in their worksheet.It is known (see Section 2 above) that the keyword method can be beneficial to memorizing vocabulary, owing to its close relation to mnemonic skills (Sagarra& Alba, 2006;Fritz et al., 2006;Wang et al., 1992;Pressley et al., 1980).In addition, twelve out of 68 (17.6%) from both groups mentioned that finding the keywords and generating 5W1H questions helped the participants to stay focused on the topic they read.They said that they tended to remember information that they were interested in and ignore other information, so when they needed to write a summary, they had to go back again and again to collect relevant information.By using the two methods, they knew what to look for in an article.For example: When I read English, I usually don't pay attention to the content, unless I am taking a test.So when I was asked to write a summary based on the article, it was hard, because I had to go back to check what I read several times and find the key points.But when I wrote down the keywords and 5W1Hquestions, it was easier for me to retrieve the key points in the article.(Participant 4,trans.)When it came to writing summaries, the participants from both groups reported that they had also benefited from writing keywords and generating 5W1H questions.All but two interviewees (94.4%) from the lower English group, compared with 25 out of 32 (78.1%) from the higher English group, said that writing 5W1Hquestions helped them narrow down important and relevant information, which made it easier for them to organize the summary in a logical and systematicway; this was the intended effect for both groups.Because summarizing an article demands the ability to condense the whole article to a concise level in one's own words, using the two approaches not only allowed them to look for main points while reading, but also helped them engage in logical thinking and outline the structure of the summary.For example: I usually don't know how to write a proper summary, but after I wrote down 5W1Hquestions for the articles, it became easier for me to find the main ideas and important information, so that I had a clear idea of how to organize and write summaries.(Participant 59, trans.) Writing summaries used to be a difficult task.I didn't know where to start, because every piece of information seemed to be important.Generating the 5W1Hquestions helped me find key information in the articles, and I knew what to write in the summaries.(Participant 28, trans.) The extent to which keywords and question generation helped improve the students' English summary writing was examined by comparing the scores for students' summary assignments.The t test showed that there was a statistically significant increase in mean scores in pre-test summary assignment and in portfolio summary assessment from both groups, p<.0005 (two-tailed) (Table 1).The means that increased in the two scores were 13.04 (Pre-test assignment (Higher Group) -Portfolio (Higher Group) and 10.89 (Pre-test assignment (Lower Group) -Portfolio (Lower Group)) with a 95% confidence interval.The eta squared statistics (Pre-test assignment (Higher Group) -Portfolio (Higher Group) = .48;Pre-test assignment (Lower Group) -Portfolio (Lower Group) = .49)indicated a large effect size.Graham and Dolores (2007), in their study of teaching American adolescents with poor writing ability, discovered that strategy instruction and word processing had a great influence on improving writing skills.The interview data and statistical results in the present study showed that, through systematic instruction or guidance, students in both groups improved their writing ability markedly.In addition to the main findings, the participants also reported that they had learned new terms and broadened their content knowledge from the different topics of the articles.However, nineteen out of the 68 students (27.9%) stated that the new information had caused them to reflect on language problems they had encountered during previous reading and writing experiences.In particular, 13 out of the 19 (68.4%) from the lower English group reported that they had reflected on grammar usage in, and the organization of, different articles.This had helped them, they said, discover both the correct grammatical rules with which they had previously made mistakes and how to structure a piece of writing.As two of the students said: When I read those articles, I discovered that I have some misunderstanding of the grammar I learned before.As a result, I know which part of grammar I need to improve and strengthen.(Participant 52, trans.) By reading the articles, I learned to write by imitating the structure of the articles, grammar, or phrases, which I did not know how to use before.I think my writing skills were improving step by step.(Participant 66, trans.)Silva and Matsuda (2002) note that the quality and content of one's writing is likely to be influenced by one's understanding of written texts and of the intentions of the author(s) concerned.Aydin (2010) also suggests that portfolio-keeping in EFL writing is beneficial to the improvement of vocabulary and grammar knowledge, reading, research, and writing skills.In the present case, by applying the methods of keyword identification and question generation in the writing of eight summaries for the portfolio assessment, the participants in both groups found it easier and more efficient to search for the main ideas of articles, to infer new vocabulary, and to strengthen their organizational skills in summary writing.These results agree with the findings of several portfolio assessment studies in other academic fields.For example, Ogan-Bekiroglu and Gunay's (2008) research on secondary school physics students' portfolio collection of academic essays and experimental reports show that their students' writing skills were markedly enhanced in both genres.

Conclusion
Incorporating keyword identification and question generation into summary writing in portfolio collection were considered practical and helpful approaches for students at the two levels.Portfolio collection with the two specific writing approaches offered multiple opportunities for the students to practice reading and writing English articles at their own pace.Importantly, the majority of the students developed confidence and a greater willingness to participate in learning English through portfolio collection.However, although portfolio collection allows students to take control of their learning process, it does not mean that teachers are no longer necessary.In such a situation, having a teacher may not be crucial for every learner, but a teacher is necessary for those who have not successfully developed autonomy or independent attitudes toward learning English.As Lamb (2008, p. 272) suggests, teachers need to 'support the development of learners' autonomy according to their individual needs, through an encouragement to reflect on their metacognitive knowledge'.To conclude, the data suggest that Taiwanese students' English reading abilities, writing abilities, and knowledge can be markedly improved through the keyword and question generation methods, particularly for higher education students at the A2 Way Stage and B1 Threshold Levels of English (CEFR), when a specific pedagogical purpose is closely linked to the learning objectives.

Table 1 .
Paired samples test result of the pre-test summary assignment and portfolio summaries