gi-zen liu, ph.d. [email protected] foreign languages & literature department national...
TRANSCRIPT
Development and evaluation of
an online writing tutorial for English paraphrasing and
citation:DWright
Gi-Zen Liu, Ph.D.
Foreign Languages & Literature DepartmentNational Cheng Kung University
Outline of the presentation
Design background A brief introduction of DWright
Purpose of the study Methodology
Procedure & instrument Participants & their background
Results Content effectiveness, system ease of use,
user attitudes Discussion, findings & conclusion
Design background (1-1)
Web-based instruction (WBI): A force of transformation towards
traditional pedagogy & language learners Information & communication
technology (ICT): An effect of knowledge exchange &
development English publication:
A trend to inspire more refined scientific and social scientific research
Design background (1-2)
Online writing tools & plagiarism detectors: Inventions to help NNSE improve their writing
skills and avoid unintentional plagiarism Downfalls of these tools:
Writing tools: help improve writing quality/skills but not suffice to teach citation skills
Plagiarism detectors: help detect suspicious texts but not suffice to educate the users
Purpose of the study
Evaluate DWright design on the following aspects: Usefulness (Content effectiveness)
Material & task design and efficacy Usability (Ease of use)
Function, interface & layout design User attitudes
Methodology
Procedure & instruction: DWright introduction session Q&A session DWright trial (to finish all tasks) Survey questionnaire
Quantitative results Semi-structured interview
Qualitative results Participants:
Gender: 14 males & 16 females Education: 21 students & 9 instructors
Positive ideas and advantages mentioned
regarding DWright reading tasks
Total number of references
1. The reading content is clear and precise. 61
2. The learning materials are a great collection of information with a good use of lists, tables and links so as to organize the reading content with appropriate learning procedure.
50
3. The reading content helps achieve the learning goals. 35
4. The reading content is detailed and enriched in description. 31
5. The reading content is valuable and educational. 28
6. The reading content covers a number of practical examples 11
7. The reading content and tasks are comprehensive. 8
8. The reading materials are like e-books and thus convenient. 1
Total= 225
Results: A high agreement on content effectiveness
Results: (examples of DWright reading tasks)
Drawbacks mentioned regarding DWright reading tasks
Total number of references
1. Some of the reading tasks are too long. 22
2. Some picture or tables are not easy to understand. 18
3. Some reading tasks don't have (enough) examples. 6
4. Some reading content may be less important from the students' point of view. 5
5. Some reading task contains a long table. 4
6. Miscellaneous: (a) Some reading tasks demand high motivation; (b) There are too many examples; (c) There is no English version. 3
Total= 58
Results:
Positive ideas and advantages mentioned
regarding DWright paraphrasing practices
Total number of references
1. The paraphrasing practices are practical and have long-term use of value. 29
2. The paraphrasing practices are well-designed and great learning process. 24
3. The paraphrasing practices provide effective and clear feedback and sample examples. 23
4. The paraphrasing practices provide great writing environment to practice paraphrase. 15
5. The paraphrasing practices can improve one's writing/paraphrasing skill. 15
Total= 106
Results: A high agreement on content effectiveness
Results:(examples of DWright paraphrasing practices)
DWright paraphrasing
Results: (examples of DWright paraphrasing practices)
Feedbacks
Drawbacks mentioned regarding DWright paraphrasing
practices
Total number of references
1.The feedback report is difficult to understand. 10
2.The layout or labeling is confusing. 43.The source texts for paraphrasing are mostly from magazine articles (instead of journal articles). 2
4.There is a gap between the knowledge of paraphrase and ability to paraphrase. 2
5.Student users need to be reminded that paraphrasing doesn't always have right or wrong answers 1
6.To paraphrase a whole paragraph without reading the source text at the same time is difficult. 1
Total= 20
Results: A high agreement on content effectiveness
Positive ideas and advantages mentioned
regarding DWright multiple-choice exercises
Total number of references
1. The exercises are well-designed, comprehensive and clear. 69
2. The exercises are learning-oriented and help achieve the goals step-by-step. 27
3. The feedback answers of multiple-choice exercises help identify textual behaviors of plagiarism and citation formatting. 20
4. The materials are appropriate and can enhance reading ability. 18
5. The exercises are comprehensive. 10
6. The contrastive boxes in the exercises are helpful for reading and memorizing the content. 6
Total= 150
Results: A high agreement on content effectiveness
Results: (examples of DWright multiple-choice exercises)
DWright multiple-choice exercises and feedbacks
Drawbacks mentioned regarding DWright multiple-choice
exercises
Total number of references
1. Some reading prompts/passages are too long. 142. The amount of exercises are not enough. 5
3. The easiness/difficulty of the exercises should be modified. 4
4. The users don't have any review before the exercises and thus are required to memorize the textual behavior of plagiarism and various citation formats.
4
5. The users need to scroll the bar to click "Send" and submit the answer. 3
6. The reading prompts don't cover enough journal articles. 27. After finish all the exercises, the system will list the links categorized by different textual behaviors. The lists are in English, instead of in Chinese.
1
Total= 33
Results: A high agreement on content effectiveness
Results: perceived usability of DWright function
Positive ideas and advantages mentioned
regarding DWright function
Total number of references
1. The function to practice paraphrasing enhances writing ability, and the function of 2-2 is especially practical. 53
2. Clear, simple, easy to use 24
3. smooth 19
4. Practical, user-friendly and helpful to achieve learning goals 11
5. Of e-profile: Useful to self-monitor learning progress and comprehension, and to evaluate the improvement of paraphrasing skill
10
6. Of multiple-choice exercise: Helpful to distinguish textual plagiarism, enhance reading ability, and increase the amount of vocabulary
9
7. Of the links provided: Convenient to refer to the online sources 5
8. Overall, DWright system is distinct and has its market. 3
Total= 134
Results: perceived usability of DWright function
Drawbacks mentioned regarding DWright function
Total number of references
1. The scroll bar in the answer option part of multiple-choice exercises is annoying. 14
2. When clicking "Previous page" or "FQA page" to review reading content, you cannot go back to the same question of the exercise. 6
3. It's not smooth enough 34. The links are not enough . 25. The feedback report of patchwriting detection should provide more description for each section, and be more straightforward. 2
6. It takes time to find the information of citation format; there should be a faster and more convenient way to find information in 2-3.
1
7. There is no English version. 1Total= 29
Results: perceived usability of DWright interface
Positive ideas and advantages mentioned
regarding DWright interface
Total number of references
1. The layout is simple and clear. 49
2. The color labels for the text increases learning effectiveness. 47
3. The layout and bars can guide the users. 26
4. The layout is satisfactory. 24
5. The tables are helpful for learning and comprehension. 19
6. The color scheme and overall design is beautiful, fresh, uncluttered and smooth. 16
7. The use of the picture reduces cognitive load. 14
8. The layout helps increase reading comprehension and efficiency. 14
9. The wording is unambiguous and easy to understand. 6
Total= 215
Results: perceived usability of DWright interface
Drawbacks mentioned regarding DWright interface
Total number of references
1. The color labels are a bit complicated. 292. The content or the table is too long and can exceed the user's cognitive load. 19
3. The option box is too narrow to show all three options, which is the reason why there is an extra scroll bar. The scroll bar for the option box is especially irritating.
19
4. The feedback report of patchwriting detection is not straightforward enough. In addition with unfamiliarity for the textual behaviors of plagiarism, the users cannot understand the report.
15
5. The layout does not take into consideration the student learning behaviors 9
6. The wording to categorize different activities is confusing. 87. The labeling in the feedback report of multiple-choice exercises should be in the feedback page, instead of the FQA page. 5
8. The labeling is not clear enough; there are no descriptions for it. 59. The 1-4 picture is placed before the reading content and can confuse the users at the beginning. 5
10. There are not English version nor numbering for ) 411. The font size is not big enough. 212 The 1-4 picture has fixed size and cannot display the whole picture in the page. 2
13. The background picture can disturb reading. 2Total= 124
Results: DWright user attitudes (positive)
Positive attitudeTotal
number of references
1. DWright not only is practical and educational, but also meets its user's needs. 33
2. DWright is highly satisfactory. 23
3. The teaching/reading content is detailed and enriched with proper learning procedures. 16
4. Learning goals are achieved in each learning unit. 17
5. DWright provide clear instruction and guide. 10
6. DWright is smooth and stable. 6
7. DWright can enhance paraphrasing and writing skill. 5
8. DWright is unique/distinct from the other writing tutorials. 2
Total = 112
Results: DWright user attitudes (negative)
Negative attitudeTotal
number of references
1. There is room for improvement regarding the interface. 9
2. Without motivation, users will experience more frustration. 2
3. The prompting passages of the multiple-choice exercises cover insufficient materials/examples from journal articles. 2
4. The report can be confusing at the first; the instruction for feedback report is insufficient. 2
5. It will take some time to know about all the functions. 2
6. 3-1 and 3-2 exercises are not easy. 1
Total = 18
Results: DWright user suggestions
Overall suggestionsTotal
number of references
1. Non-native speakers of English need higher motivation to use this tutorial independently. This tutorial can be used in a blended and complementary tool in classes. It can also be in school policy to require the students to learn how to avoid plagiarism and paraphrase successfully through this tutorial.
6
2. Modify some part of the interface. 53. Promote the tutorial with better marketing strategies, since it's a practical and useful tutorial. 2
4. Combine other online search engines or resources. 25. Provide more instructions or reminders to assist the users. 26. Develop other possible functions for persistent use of the tutorial. 2
7. Provide a convertible interface for Chinese and English languages. 1
8. Add materials from journal articles for exercises. 19. Provide time tracker in e-record to monitor test/answering progress. 1
10. Reading content can provide more animation or interaction. 111. Provide more time for DWright users to familiarize themselves with the tutorial function. 1
Total = 24
Discussion: perceived usefulness
The following factors regarding perceived usefulness could influence user attitudes: Content clarity & enrichment Step-by-step reading and practicing tasks
to achieve learning goals Practical functions Useful feedbacks after practices/exercises
Discussion: perceived usability
The following factors regarding perceived usability could influence user attitudes: Effective & user-friendly paraphrasing
functions (e.g. analyzing the texts) A clear & simple layout Various color labels
Findings & Conclusion
Perceived usefulness of DWright had more impact on user attitudes than system usability did.
DWright met user needs by extending user’s knowledge to avoid plagiarism and by enhancing their paraphrasing and writing skills.
Program designers, content experts and subject teachers should support great communication to maximize the effectiveness of content usefulness to help achieve user goals.
A simple, easy-to-use and user-friendly design of DWright should provide an effective use and a pleasant e-learning experience and ensure learning efficacy.
This is the end of the presentation.
Thank you very much.