instructional design project - university of...
TRANSCRIPT
Instructional Design Project
LLDT Collective
Laura Lucas
Lauren Butcher
Dawn Cancelliere
Tom Hammond
Instructional Design
The University of Akron
Dr. Cheryl Ward
April 25, 2012
Instructional Design Project 2
Table of Contents
Introduction .............................................................................................................................. 3
Analysis Phase ......................................................................................................................... 3
Design Phase ............................................................................................................................ 6
Task Analysis .............................................................................................................. 6
Learning Theory .......................................................................................................... 6
Sequencing .................................................................................................................. 7
Instructional Objectives ............................................................................................... 8
Module 1: Learning in the 21st Century ....................................................................... 8
Module 3: Evaluating Websites & Resources.............................................................. 9
Development Phase ................................................................................................................ 10
Organization & Presentation for both Modules ......................................................... 10
A Closer Look at Module 1: Learning in the 21st Century ......................................... 12
Module 1: Added and Created Media/Informational Resources with Rationale ........ 13
A Closer Look at Module 3: Evaluating Websites & Resources ............................... 14
Module 3: Added and Created Media/Informational Resources with Rationale ........ 15
Extra Module that was Developed: Help and Tutorials for Online Learners ............. 16
Implementation Phase ............................................................................................................ 16
Implementation Action Plan ...................................................................................... 17
Alignment with the CLER Model.............................................................................. 18
Timeline .................................................................................................................... 20
Evaluation Phase .................................................................................................................... 20
Evaluation Method .................................................................................................... 20
Validity ..................................................................................................................... 22
Assessing Knowledge................................................................................................ 22
Evaluation Standards ................................................................................................. 23
Evaluation Results ..................................................................................................... 23
Conclusion ............................................................................................................................. 27
References .............................................................................................................................. 28
Instructional Design Project 3
Introduction
The instructional design problem presented to the LLDT Collective was to move a current course,
Educational Technology, from a face-to-face format to a fully online course. The team was responsible
for creating materials for two modules: Learning in the 21st Century and Evaluating Websites and
Resources. The team completed the ADDIE model in order to ensure that course objectives were met, the
needs of the learners were met, the materials aligned with course objectives and evaluations, and that
proper implementation and evaluation phases were conducted.
Analysis Phase
Dr. Kline is a professor at the University of Akron in the department of Curriculum and
Instructional Strategies. She teaches a course titled Educational Technology. Recently, the course was
updated to address the need to have pre-service teachers prepared for 21st century teaching and learning.
The course is currently designed and delivered as a face-to-face course and contains web enhanced
learning components. Although it is organized in modules, the modules need revised so that students are
able to obtain the knowledge, concepts, skills, and abilities to complete projects through a program that is
entirely online. According to ISTE’s National Educational Technology Standards for Teachers (NETS-
T), teachers should be able to "Facilitate and inspire student learning and creativity, design and develop
digital-age learning experiences and assessments, model digital-age work and learning, promote and
model digital citizenship and responsibility, and engage in professional growth and leadership" (NETS
Project, 2007, p. 9). What better way to teach students how much technology can benefit themselves and
their careers than by creating a fully online class where students are required to do all of the above?
The learners are undergraduate students. This course is open to students prior to entering the
College of Education. Some of the students enrolled in this course may be out of high school, whereas
others may be adult learners who are going back to college to make a career change. The course is
designed to accommodate diversity amongst its students. The LLDT Collective will ensure that the
recommended online materials will accommodate this diversity in regards to characteristics, lifestyles,
Instructional Design Project 4
and histories. Online courses appeal to students because of their flexible nature with time and travel
constraints. It is important to keep in mind that students who enroll in online courses often have busy
lives outside of academia with families, jobs, etc.
“One of the key steps of the design process is defining the content needed to address the
instructional need or problem. This content is then used to identify the objectives, design the instructional
strategies, develop test items, and create the instruction” (Morrison, Ross, & Kemp, 2007, p. 94). The
problem is that the current face-to-face instruction needs modified in order to move to an online course.
The problem presented to the LLDT Collective is specific to developing two learning modules.
The first module concerns "Learning in the 21st Century." The instructor would like to see
learners investigate skills that they can use when teaching in the 21st century classroom. Overall, she
would like them to understand what 21st century teaching and learning is all about. There were not
specific skills listed in the course syllabus, but the course objectives stated that students will develop the
basic understanding of using existing and emergent educational technologies in achieving curricular
goals. This includes classroom management, curriculum design, and instructional strategies. The team
believes that this objective is not concise enough to be measurable using the ADDIE process (Analysis,
Design, Development, Implementation, and Evaluation). To this end, the team clarified the objective,
stating that students will demonstrate an understanding of 21st century teaching and learning skills.
The second module concerns "Evaluating Websites & Resources." Throughout this module, the
instructor requires that learners become discriminating consumers when using websites and resources.
Students need to learn how to evaluate information and make good choices when locating and using
information on the web (for example, how to check the validity of a website). This is important because
they, in turn, need to be able to teach their future students to be discriminating consumers as well. There
was no clear objective pertaining to this module. Dr. Kline expressed her expectations in terms of what
she would like her students to be able to do at the end of this module: 1) Students will differentiate
between reliable web resources and unreliable web resources, and 2) Students will create techniques used
to teach others how to evaluate web resources.
Instructional Design Project 5
After interviewing the subject-matter expert (SME), Dr. Kline, the team decided that the first
need was to rewrite objectives for the modules to be more specific and thus more measurable. The team
also decided that students need access to course materials, projects, and assessments online. Virtual
learning communities must be formed to promote interaction and strengthening of skills through
teamwork, discussion, and debate. Course module assignments must be online experiences to encourage
self-directed learning. The materials must be accessible to students of varied technological
skills/competencies, disabilities, and diversity.
After completing a topic analysis, the team decided that it would be beneficial to add online
learning components to the current topics in both modules to enhance student learning. The topics
covered in the "Learning in the 21st Century" module include exploring 21st century learning,
understanding 21st century learners, and achieving 21
st century learning. The topic covered in the
"Evaluating Websites & Resources" module is preparing pre-service teachers to identify reliable and
unreliable websites and online resources. This is currently accomplished in the course using WebQuests.
A completed procedural analysis allowed the LLDT Collective to determine the steps needed to
create the online learning components for the current topics in the modules. These steps included:
1. Evaluate current course materials and content (topic analysis).
2. Decide which topics would benefit from added learning components.
3. Develop learning components for each module.
4. In a fully online course, provide methodologies for the completion of assignments
virtually instead of traditional classroom instruction.
5. Evaluate new learning components in regards to alignment with current materials and
topics.
As the team moved through the ADDIE model, the LLDT Collective strove for alignment with
the College of Education's theme of being Educators as Decision Makers, and developed strategies to
include components of knowledge, technology, diversity, and ethics.
Instructional Design Project 6
Design Phase
Task Analysis
The LLDT Collective identified the following instructional problem: the Educational Technology
course needs to transfer from face-to-face instruction to a completely online format. The team was asked
to address this problem in reference to two out of the six learning modules that the course covers:
“Learning in the 21st Century” and “Evaluating Websites & Resources”. After completing a topic
analysis, the team decided to add online learning resources and assessments to the current modules that
were created by Dr. Kline. The topics covered in “Learning in the 21st Century” include exploring,
understanding, and achieving 21st century learning, while the topics covered in “Evaluating Websites &
Resources” include evaluating websites for validity and reliability.
Learning Theory
When designing the instructional plan for the Educational Technology course, the LLDT
Collective focused on behavioral and cognitive learning theories. One tenet of the behavioral learning
theory is the use of instructional objectives to measure mastery of content. Morrison et al. (2007)
asserted, “Such objectives clearly identify the overt responses reflecting mastery of the content, the level
of achievement indicating acceptable performance and the conditions under which the evaluation takes
place” (p. 348). Cognitive learning theory (or more specifically, constructivist theory) is also represented
because students will be active participants in their learning. Through online experiences such as
discussion forums, blogs, video presentations, etc., students will analyze and synthesize information while
building upon their prior knowledge. Morrison et al. (2007) explained that “social interactions can
facilitate learners’ progression to more advanced ways of thinking and problem solving” (p. 355). As
self-directed learners, students will work independently and collaboratively with their classmates to learn
new concepts and to solve problems.
Instructional Design Project 7
Sequencing
Through the needs assessment and learner analysis, the LLDT Collective determined that students
in the Educational Technology course will have a variety of technological skills. Before entering the
course, however, students will have completed a computer skills assessment that prepares them for the
remainder of the online course. This assessment is the first step in learning related instructional
sequencing, the identifiable prerequisite. As students become acquainted with the learning management
system (Springboard), the team would like to appeal to students’ interests by providing videos, articles,
and thought-provoking online discussions about real-world issues. These types of online components are
the most familiar and easiest to navigate and learn about. Eventually, content resources will gradually
become more difficult as students continue to develop technological competencies. The less difficult
material will be taught before the most difficult, and learners’ competencies will be assessed before
moving onto a new topic.
Instructional Objectives
After analyzing the instructional problem, needs assessment, learner analysis, task analysis, and
learning theories, the LLDT Collective determined the overarching learning goal of the two modules:
Learning in the 21st Century and Evaluating Websites & Resources
Students will develop basic understanding of using existing and emergent educational
technologies in achieving curricular goals including classroom management, curriculum
design, and instructional strategies.
In order to reach this goal, the following instructional objectives will be put in-place for the
modules:
Learning in the 21st Century
Using the “Learning in the 21st Century” online resources, examine the characteristics of
21st century learners meeting the criteria of the Springboard discussion board and
VoiceThread response rubrics.
Instructional Design Project 8
Evaluating Websites & Resources
Based on the “Evaluating Websites & Resources” online resources, differentiate between
reliable and unreliable websites meeting the criteria of the Wikispace website evaluation
rubric.
Module 1: Learning in the 21st Century
Objective Using the “Learning in the 21st Century” online resources, examine the
characteristics of 21st century learners meeting the criteria of the Springboard
discussion board and VoiceThread response rubrics.
Pre-instructional strategies Prior to beginning the Module 1 content, students will complete the
SurveyMonkey Pretest in order to determine their current knowledge of 21st
century teaching and learning. Morrison et al. (2007) asserted that “a pretest
is designed to heighten the student’s awareness of the content by serving as
cues to the key points” (p. 171). The short pretest contains five true/false
questions that include factual evidence and real-world scenarios about 21st
century teaching and learning. The pretest is meant to “get their wheels
turning”, so it is not for an official grade. The results of this pretest will be
shared with the instructor so that he/she can initiate a discussion board for
students if necessary.
Instructional strategies The initial presentation strategy focuses on defining the concept of 21st
century learners through videos, articles, and Internet resources on
Springboard. The organizational generative strategy will follow, whereby
learners analyze key ideas in order to identify the features that define the
concept (Morrison et al., 2007). First, students will participate in a
discussion board with the following prompt:
Now that you're aware of what 21st century teaching and learning skills are
all about, think about how you would ensure that your students learn these
skills. Many schools lack the tools and funding needed to integrate
technology into their classrooms. As a result, we, as educators, need to
become extremely resourceful in our efforts to make sure our students obtain
these skills. How would you do this in a school that lacks technology?
Put your creative thinking caps on! Assume that you teach in a school that
has only three computers with internet access per classroom. Share some
ideas about how you'd make sure that your students obtain the 21st century
skills that they need for the future.
Next, students will use VoiceThread to post a response to the following:
Reflect on the articles that you read and the videos you watched in module 1:
Learning in the 21st Century. Think about what you learned and why 21st
century skills are so important in education today. You will use this
Instructional Design Project 9
VoiceThread to demonstrate your understanding of the content (see link to
VoiceThread activity for further instructions).
Finally, students will participate in the following discussion board once they
have viewed all the postings on the VoiceThread:
To be prepared for this discussion, you must have first listened to (or read)
each of your peers’ VoiceThread summaries about 21st century learning.
Your task for this discussion board is to choose two of your peers'
VoiceThread summaries that really stood out to you or gave you a new
perspective. You are to create discussion posts about these two particular
VoiceThread summaries here. These should be very reflective posts that
explain why you chose to respond to each particular summary. So, instead of
one "initial post" in this discussion, you will create two separate "initial
posts" that discuss the VoiceThread summaries of two of your peers.
You might discuss things that your peer mentioned that you agree/disagree
with, and then state the reasons why you feel that way. You could also
discuss something they said that struck you as interesting or maybe
something you never thought of before, and then state why. Maybe you even
have a personal experience that relates to what they discussed in their
summary. Be reflective!
Module 3: Evaluating Websites & Resources
Objective Based on the “Evaluating Websites & Resources” online resources,
differentiate between reliable and unreliable websites meeting the
“Wikispace Website Evaluation Activity Rubric” criteria.
Pre-instructional strategies Prior to beginning Module 3 content, students will view a Voki introduction
that provides the learning objective for this module. One reason for initially
presenting the learning objective is that “Learners were better able to
interpret and understand the implications of the learning directions and
questions because they were presented in simpler sentences” (Morrison et al.,
2007, p. 172). Additionally, students will complete the “Website Evaluation
Pretest”. This true/false quiz has 9 questions that will activate any prior
knowledge about the characteristics of valid and reliable websites.
Instructional strategies The initial presentation strategy teaches principles and rules for determining
reliable websites through articles and Internet resources on Springboard. The
elaboration generative strategy will be used when students evaluate two
websites and reflect on their reliability on the Wikispace Website Evaluation
Activity. The following is the task for this assignment (which is found on the
wiki):
There are two different links listed below (Website #1 and Website #2). One
of the links will direct you to a website that can be described as a "reliable"
source of information based on what you've learned about evaluating
websites. The other link will direct you to a website that can be described as
"unreliable". The question is: which one is the reliable website?
Instructional Design Project 10
1) Click on each link below and view the two different websites. As you take
a look at each website, think about everything you've learned about website
evaluation.
2) After you've viewed each website, your assignment is to use the criteria
listed in the 5 W's of website evaluation to evaluate each website. So, revisit
Website #1, and provide a detailed evaluation of Website #1 based on each
of the 5 W's. Pay attention to the questions listed under each of the 5 W's and
try to answer them all in your evaluation! Simply stating that the website is/is
not a reliable source of information does not showcase critical thinking
about what you've learned! When you're finished with your evaluation of
Website #1, use the same evaluation process to provide a descriptive
evaluation of Website #2!
3) When you are finished evaluating each site, decide which website can be
considered the “reliable” site. Was it Website #1 or Website #2? Why do you
feel that way?
4) Make sure that you reference the rubric before you begin! Good luck!
Development Phase
The LLDT Collective took many factors into consideration while developing instruction for both
learning modules. The team reflected on what would be appropriate and effective for a diverse group of
learners with varying degrees of technological skills and learning styles. Thoroughly reviewing the team’s
established goals, strategies, and objectives helped the team stay focused on solving the instructional
problem: moving instruction from a face-to-face format to one that is fully online. The team determined
that the objectives it created support the resolution of the instructional problem and that the learning
resources it developed are conducive to an online learning environment.
Organization & Presentation for both Modules
As a part of this phase, a detailed storyboard was developed. This helped to keep the team
organized in terms of how it decided to reconstruct its modules. It provided a comparison of what the
original modules looked like and had to offer versus the team’s reconstructed modules (what the team
replaced, added, and created in its new modules). A module “key” that describes the types of learning
resources used is displayed at the bottom of the map. This helped the team stay mindful of using a variety
of learning resources to accommodate different learning styles. It also helped the team organize its
Instructional Design Project 11
materials appropriately. The storyboard can be viewed on the LLDT Collective Wikispace presentation:
http://instructionaldesignspring11.wikispaces.com/LLDT+INCORPORATED.
The LLDT Collective cautiously organized the learning resources in a sequence that supported
student ease of use. The team suggested that the students view the resources in the order in which they
are presented because they are structured to introduce materials that are less difficult (videos) before those
that are more challenging (reading materials that the students have to process on their own). Furthermore,
each module begins with a pretest that is designed to stimulate interest and provide students with the "big
idea" before they dive into the content. Similarly, both modules end with one or more assessment
activities that are designed to showcase knowledge and skills that have been acquired, determining if the
objectives have been met.
According to Morrison et al. (2007), research suggests that "a mix of abstract (text that is not
represented by a single mental image) and concrete (text that allows the learner to easily invoke mental
images) information increases both the learner's interest and recall of the information" (p. 200). For this
reason, the team designed modules to include reading materials that provide students with both concrete
and abstract text; some of the required readings are very straightforward and comprehensible while others
are more conceptual. Additionally, supplying students with concrete references such as visual aids,
simple definitions, stories of personal experiences, and videos helped to reduce what Morrison et al.
(2007) referred to as the "extraneous cognitive load" that they have to exert (p. 207).
This course is an undergraduate course that does not require prerequisites. Therefore, assuming
that all students have familiarity with the content in each module would have led to poor design.
Consequently, the LLDT Collective developed learning resources, exercises, and interactions that are all
interrelated and expand upon the main idea. This will decrease what Morrison et al. (2007) referred to as
"step size" or large jumps between ideas (p. 201). Making precise references back to what students
learned when providing them with instructions for the assessment activities was another strategy the team
used to control the step size (p. 201). All activities that require directions for completion offer extremely
detailed instructions that are stated simply so that confusion about the requirements is minimized.
Instructional Design Project 12
The team’s systematic approach also aspired to reduce what Morrison et al. (2007) referred to as
pacing. "Pacing is a function of the number of examples, problems, interactions, or exercises presented
with an idea" (p. 202). Each module was designed to control the pace and give students enough time and
support needed to grasp and understand the main idea. Every resource had been developed with the
intention to illustrate the objectives in an informing and engaging fashion.
Everything that was developed provided specific, detailed, and step-by-step instructions for how
to either navigate content or complete assignments. Since the students are not in a face-to-face
environment where they can instantly receive feedback, this was extremely important! Numerous
revisions concerning content, activities, instructions, rubrics, website design, etc. were made throughout
the entire project to include precise information for students. The instructions appeared to be more
redundant than they probably typically would. However, for new online students, it is important to be
repetitive and offer this type of support. Every single word that was typed and re-typed was taken into
consideration. This attention to detail is crucial to the success of the team’s module reconstruction.
A Closer Look at Module 1: Learning in the 21st Century
Objective Using the “Learning in the 21st Century” online resources, examine the
characteristics of 21st century learners meeting the criteria of the Discussion
board and VoiceThread response rubrics.
Pre-instructional Strategy This module opens with a pretest that was developed using SurveyMonkey.
As previously stated, the pretest is designed to stimulate interest and provide
students with the "big idea" before they dive into the module. Since the
results are sent directly to the instructor (Dr. Kline), she is instantly provided
with data about her students’ prior knowledge and can gauge their learning
needs. The instructor will also post answers to the quiz after the students
complete all of the activities in Module 1, so that they are provided with
feedback about their initial responses.
Note: after the team is finished with this project, Dr. Kline’s email address
will be added to the account so that she will receive the results.
Initial Presentation As stated above, this module is structured to introduce materials and
activities that are less difficult before those that are more challenging. All of
the learning resources have descriptive titles which provide the students with
cues as to what they’ll be learning about as they progress through the
module.
Instructional Design Project 13
Generative Strategy The students will be actively engaged in the materials as they participate in
the pretest, watch the videos, study definitions and graphics, and read the
articles. As students partake in the discussion posts, the instructor will
involve herself in the discussion and play an active role in providing
feedback. Morrison et al. (2007) suggested that “feedback should include
directions to the learners that they should compare their responses to that of
the expert to identify differences” (p. 204). Therefore, after students
complete their VoiceThread summaries, the instructor will create her own
summary based on her expertise of the content so that students can compare
their responses to hers.
Module 1: Added and Created Media/Informational Resources with Rationale
Checklist of
Activities for
Module 1
This was created to serve as visual representation of all learning resources and
assignments for Module 1. It serves as a checklist that is viewed and printed
before the students begin their work. It is designed to help them stay on track and
ensure that they’re completing all necessary tasks. SurveyMonkey
Pretest This seven-question pretest was developed using SurveyMonkey. Bits and pieces
of information were taken from all of the resources throughout the module and
integrated into the pretest (rationale stated under the pre-instructional strategy).
Since this survey tool does not offer an option for immediate feedback, the team
configured it so that students are provided with data that shows how their answers
rank (percentage) in comparison with others who have taken the pretest. The
questions were not developed with the intention of being too difficult. Rather, they
are meant to “get the students’ wheels turning” about the content that lies ahead.
The team used SurveyMonkey because it wanted to expose the students to a useful
way of collecting data. 21
st Century Skills
Diploma News
Story
This YouTube video of a local news story was added because it puts the
importance of learning 21st century skills into a meaningful, real-world perspective.
Students may hear about the importance of these skills in different classes, but
seeing it on the news and hearing that employers see a major lack of these skills
during interviews will be a real eye opener. This will stimulate interest for the
following materials in the module. A Vision of 21
st
Century Teachers This YouTube video was added to expand upon the message presented in the video
that precedes it. The first video is about 21st century students, while the video the
team added is about 21st century teachers. The videos are a perfect match because
both follow the same visual layout. This will provide students with a glimpse of
what 21st century students and teachers look like; not just students.
A Definition of 21st
Century Skills This online Wikipedia resource was added to help students understand what 21
st
century skills actually are. It presents them with factual, straightforward
information about 21st century skills. The definition also illustrates a few graphics
for the students to study and view which will aid their comprehension of this
material. This also offers an alternative to the more typical video or article.
Instructional Design Project 14
Discussion Topic
#1 (Becoming
Resourceful)
This topic prompts students to use what they have learned and demonstrates their
resourcefulness as educators in a rapidly changing world (see storyboard for
prompt). This was something that the SME stated that she wanted the students to
be able to explain. Students will be expected to share ideas for promoting 21st
century skills in classrooms that may not have the funding or tools to support
technology integration. This discussion also fosters personal reflection, peer
interaction, and idea-sharing. An explanation of how to create a post/reply is
offered right in the discussion board and a rubric is provided as well. Discussion Board
Rubric This rubric is located in the Content area of Springboard.
VoiceThread
Summary
Assignment
This tool was developed as a method to assess knowledge that students gained as a
result of this learning module. They will be expected to provide an overall
summary of what they have learned, including critical questions presented to them
in the VoiceThread activity description. This will give the instructor insight as to
whether or not the objective has been met. Detailed instructions for the task and
how to create the finished product are embedded in the activity and a rubric is also
provided. The team wanted to showcase VoiceThread as a purposeful tool that can
be used by anyone.
VoiceThread
Rubric This rubric is located in the Content area of Springboard. It is not as detailed as
the other rubrics, because the criteria is straightforward and specifically laid-out
for the students. Discussion Topic
#2 (VoiceThread
Response)
This discussion serves as a response to the VoiceThread assignment (see link in
Springboard for complete prompt). It is intended to initiate a reflective discussion
between students and their peers. Students will have the opportunity to comment
on their peers' VoiceThread submissions. The goal is for students to think about
the perspectives of others and reflect upon concepts or ideas that gave them a new
perspective on 21st century learning. An explanation of how to create a post/reply
is offered in the discussion board area and a rubric is provided as well.
Note: The LLDT Collective restructured the SME’s module to exclude the PowerPoint presentations. The
PowerPoint slides correspond with a book that is no longer a requirement. The SME stated that the book
is very expensive and will become a suggested reading material for her students. The team determined
that removing the slides and replacing them with more direct and engaging learning resources will better
suit the objective.
A Closer Look at Module 3: Evaluating Websites & Resources
Objective Based on the “Evaluating Internet Information” online resources,
differentiate between reliable and unreliable websites meeting the criteria of
the Wikispace website evaluation rubric.
Pre-instructional Strategy Students will be introduced to this module by a speaking avatar created on
www.voki.com. The avatar will describe the objective, content, and
assessment activity. This will provide learners with simple dialogue about
what they are going to learn, why it is important, and what is expected of
them as an outcome. They will also complete a pretest created on
www.quizz.biz.
Instructional Design Project 15
Initial Presentation The initial presentation teaches students how to evaluate websites for
validity. Many materials were developed and collected to help students
achieve the objective. Resources are organized in a sequential manner that
will help students learn the process of website evaluation. All of the learning
resources have descriptive titles which provide students with cues as to what
they will be learning as they progress through the module.
Generative Strategy The generative strategy is used when the students demonstrate the knowledge
and skills they have learned and put them to-the-test in the module’s class
Wikispace assignment. They are challenged to look at two websites that
contain similar content, use specific evaluation criteria to evaluate both sites,
then determine which site can be described as reliable and which one cannot.
The instructor posts the correct answer at the end of the discussion board
after all students have completed the module. The instructor will also be
actively involved in the discussion.
Module 3: Added and Created Media/Informational Resources with Rationale
Checklist of
Activities for
Module 3
This was created to serve as a visual representation of all learning resources and
assignments for Module 3. It serves as a checklist that is viewed and printed before
the students begin their work. It is designed to help them stay on track and ensure
that they are completing all necessary tasks.
Voki
Introduction This speaking avatar was developed on www.voki.com (see pre-instructional
strategy). This is a unique way to begin a new module; a good idea for capturing the
attention of students! Website
Evaluation
Pretest
This website evaluation pretest was created using www.quizz.biz. The questions
were not developed with the intention to be difficult. Rather, they are meant to “get
the students’ wheels turning” about the content that lies ahead.
This pretest is designed provide students with immediate feedback about what they
already know and/or what they need to learn more about in terms of website and web
resource evaluation. Quizz.biz was chosen for a specific reason. It is a free site for
anyone to use, and one does not have to register or pay to use it. Saved quizzes can
be accessed at any time. This is a good tool for anyone to use!
Evaluating
Websites Using
the 5 W's
The team added this YouTube video because it demonstrates a technique that students
can use when evaluating websites and resources. It provides step-by-step instructions
for them to follow, and describes what to look for when determining if a website
should be considered reliable/valid.
The 5 W's of
Website
Evaluation
This PDF was also added because it uses the same language as the above mentioned
YouTube video. It illustrates short, simple sentences that reiterate what the video
teaches students about evaluation techniques. With these connected learning
resources, students will be exposed to both an auditory and visual representation of
the content. Also, they will use this in the Wikispace assessment activity at the end of
the module.
Instructional Design Project 16
The ABC’s of
Website
Evaluation
This informational resource was added to serve as an extension of the knowledge
gained in the preceding resources. More detailed information is available through
this site. This site provides students with explanations that describe why website “A”
and website “B” are reliable/unreliable. Links to the sites are provided so that
students can see what the author is describing as reliable/unreliable information; it is
an interactive experience.
Wikispace
Website
Evaluation
Activity
This wiki was created to serve as an assessment tool. It is designed to test the
students’ website evaluation skills; they will use what they have learned throughout
this module and apply it in this evaluation activity. This assessment activity provides
the instructor with evidence that supports whether or not the objective has been met.
Also, it exposes the students to a "wiki page" and gives them a new way to reflect
upon their learning. Detailed instructions for the task and how to create the finished
product are included on the page. A rubric is also provided. Wikispace
Website
Evaluation
Activity Rubric
This rubric is located in the Content area of Springboard.
Note: The LLDT Collective restructured the current module to exclude the WebQuest materials as the
SME stated (in the initial interview and through a phone conversation) that she would like students to
focus on website evaluation.
Extra Module that was Developed: Help and Tutorials for Online Learners
As the development phase was being edited, the LLDT Collective brought attention to the fact
that the existence of a module which offered assistance to online learners was necessary. Since there is a
possibility that students who enroll in this class may have limited experience in an online learning
environment, a module was developed to address this need. Resources to help students with Springboard,
as well as the Wikispace and VoiceThread activities, were added to this module (see storyboard for a list
of added resources). This module is titled “Help and Tutorials for Online Learners.”
Implementation Phase
Now that the content has been analyzed, designed, and developed, the next phase of the project is
where the "rubber meets the road", the implementation. One cannot assume that presenting the team's
content to a select audience will automatically be understood and integrated; the content must be prepared
and presented in such a way as to "sell" it to the client, to make it something that they need and find value
in.
Instructional Design Project 17
The innovation of the content proposed by the LLDT Collective must have perceived value. In
other words, when the team suggests a new way of doing something, how does the client trust this new
idea of bringing about a better way to learn the content? The innovative aspect of the team's proposed
content focuses upon online discussions through the Springboard classroom management system.
Students read articles, watch videos, complete quizzes, and then come together to the discussion board
"watering hole" to share what they have learned. During implementation, the team wants students to
realize the worthwhile learning that takes place through these online discussions and the value of learning
communities that are formed.
Successful implementation also requires that the team's goals align with that of the client, Dr.
Kline. Her goals and values with content presentation must be compatible with the team's offering. In
this particular case, Dr. Kline wants to transition her semi-online class to being completely online. The
team’s goals are compatible with hers. The real issue is to ensure that the modes of delivery that the team
chose are compatible with how readily they are embraced by the real clients, the students. The LLDT
Collective needs to ensure that the team’s content matches their values, needs, and experiences as well.
Through the design and development phases, the team established that the content supports proper pacing
and sequencing. Content difficulty starts with simpler concepts which gradually increase in difficulty and
complexity. Pacing is carefully set so that plenty of time and examples are presented before moving to
the next concept. This idea of complexity is important for content acceptance and has been addressed by
the team.
Implementation Action Plan
To implement the content, the LLDT Collective decided upon a twofold approach using different
audiences. The first audience consists of students enrolled in Timothy Sisson's Educational Technology
class. After reviewing the team's content, he realized that it "would be a valuable learning experience for
them to not only develop this content knowledge, but also to think about the learning process and what
that means through online delivery." The team also sought professional involvement during the
Instructional Design Project 18
Implementation Phase. To this end, the following educational professionals agreed to implement and
evaluate the content as well:
Betty Rogge An instructor who currently teaches Educational Technology
Denise Uitto Another instructor involved with Educational Technology teaching
Lori Brinker Coordinator of the Computer Based Training program (CBT)
Patsy Malavite Computer based Applications instructor for the CBT program
Jackie Ashbaugh Received her Master's in Instructional Technology two years ago
The involvement of these two groups (students and educational professionals) during
implementation provided valuable feedback from two perspectives. Although both will complete the
content as students, the professionals may identify strengths and weaknesses in the content that students
may not and vice versa. Thus, the team meets the goal of successful implementation by piloting the
content in these small groups before its adoption into future classes. It also assures observable results in
that the educational professionals listed above, in addition to Mr. Sisson, can attest to the content's
viability and can encourage others to adopt it as well.
Alignment with the CLER Model
To execute a successful implementation, it is important to specify "whom and how to
communicate information about an instructional product" (Morrison et al., 2007, p. 393). The team chose
the standard CLER model for this purpose, developed by H. S. Bhola (1982, 1988-1989). CLER involves
identifying the project's various constituents (configuration), how these participants interact with each
other (linkages), how the environment must be configured for effective implementation, and the proper
selection and application of educational resources.
With this particular project, the designer team consists of the four members of the LLDT
Collective as mentioned above. Each team member is specialized for each phase of the ADDIE model,
though all members collaborated together during each phase. The team received instructional supervision
and guidance through Dr. Ward. The client system consists of Dr. Kline, Mr. Sisson, Ms. Rogge, the
Instructional Design Project 19
students in both of their Educational Technology classes, and the four educational professionals who also
implemented the material. The linkages between these groups of people revolve around the Instructional
Technology and Educational Technology programs, with Dr. Ward managing the former and Dr. Kline
the latter. LLDT Collective formed a working relationship with Dr. Ward for overall project direction,
while the team worked with Dr. Kline to assess her needs and goals for the actual content. The team's
relationship with Betty Rogge is two-fold. Working under Dr. Kline as a senior lecturer, she offered first-
hand experience with the class' current content offerings. She also served to implement the content as a
student, providing valuable feedback as she would evaluate her own content.
The environment for the implementation phase is online, requiring computer resources with an
Internet connection. Mr. Sisson and Ms. Rogge provided these resources to their classes via a computer
lab, while the educational professionals obtained these resources on their own through office computers,
home computers, and access to computer labs at Wayne College. LLDT Collective organized and posted
all of this content into a sample Springboard classroom, making it centralized and easy to access by
clients, students, educational professionals, and the LLDT design team. Technical support to ensure
proper delivery through Springboard was provided by Chris Dreher, the designated Springboard support
specialist at Wayne College. There were several influence resources (motivations) working in tandem
during this project. From the students' aspect, Mr. Sisson and Ms. Rogge guided their students through
the team's materials and activities during their scheduled class sessions. The educational professionals
completed the materials as a form of evaluation to assess their viability to possibly be adopted into their
own classes and programs.
Financial backing was not needed to implement this project with these pilot groups, as the
facility, technical equipment, and support personnel were available. Should the project be adopted into
the mainstream program, accommodations may be needed to properly compensate for increased use of
these resources. Transportation to/from the educational setting is also a given, as the students are already
at the University for Mr. Sisson's regular class content and at Wayne College for Ms. Rogge’s. The
educational professionals can complete the content while at Wayne College for their regular duties or at-
Instructional Design Project 20
home using their own Internet connection to access everything located on Springboard. Other resource
accommodations such as housing and food are not necessary as students are already present at the
University for Mr. Sisson's class and Wayne College for Ms. Rogge’s class; the same goes for the
educational professionals. The team’s goal for this project implementation was to be minimally invasive
and incur the least possible cost. For these reasons, the team chose to implement the content using a
centrally located resource (Springboard) during times when students and professionals were already on-
campus to access the content. Because all of the materials are online, there were no costs for packaging,
duplication, warehousing, and shipping of traditional paper-based materials and video media.
Timeline
From a scheduling standpoint, the team worked with Mr. Sisson to determine the best class
session that he could task his students with completing the project's assignments. Mr. Sisson was flexible
in this respect, agreeing upon Wednesday evening, April 4th, as the assigned class session. The
educational professionals’ involvement during implementation was even more flexible. Because the
materials were accessible online, they had an entire week to complete the activities at their leisure from
wherever they had access to a computer with an Internet connection. During this week, representatives
from the team directed the project's implementation, acting as instructors to guide the participants through
the activities and discussions that ensued. The team did not require much training on managing a class
through Springboard, as they had ample experience from taking classes involving this system in the past.
Still, Chris Dreher provided support for any higher-level questions that the team had about managing the
class during the implementation phase.
Evaluation Phase
Evaluation Method
The LLDT Collective performed an extensive evaluation of the content materials and delivery
methods. Because the team enhanced the content of two modules within an existing class, formative
Instructional Design Project 21
evaluation methods were chosen, which allows evaluation of instructional processes and outcomes during
development and small group trials. A summative evaluation was not appropriate because the subset of
learning modules did not run the full course of the class; the team's proposed content would occupy two
weeks (one week for each module) of learning during a 16-week class. Similarly, a confirmative
evaluation was not appropriate for this study because this was the first time the materials were used, thus
long-term benefits could not yet be assessed.
The purpose of the formative evaluation is to not only realize the strengths and weaknesses of the
learning materials & methods, but more so to use these results to revise and improve the materials for
future learning sessions. The team received feedback from two different Educational Technology classes
spaced seven days apart. The first class was hosted by Tim Sisson at The University of Akron, and the
second class was hosted by Betty Rogge at The University of Akron Wayne College. The evaluations
from the first class allowed the LLDT Collective to revise the materials to be used for the second class.
The team also requested formative evaluations from six professional educators. Out of these, Jackie
Ashbaugh, Lori Brinker, Patsy Malavite, Betty Rogge, and Denise Uitto provided valuable feedback from
an instructor's viewpoint. Thus, the audience for this project comprised of two "live" classes of students
and five educational professionals.
During staging, all LLDT team members tested the content individually. Some of its members
conducted one-to-one trials with friends, family members, and their own classroom students to ensure that
the technology worked correctly with different computer configurations. Small group trials were not used
as testing was done as field trials during actual class sessions.
The team decided to use paper based evaluations to minimize technology barriers, promoting a
larger set of responses. The student evaluations were decision oriented, contained a single page of yes/no
questions with short essays to describe responses further if desired. The educational professional
evaluations were connoisseur-based, two pages in length, and requested elaboration. Evaluations were
returned to the LLDT Collective, scanned as PDFs, then distributed to team members for analysis,
reporting, and subsequent content revisions. Four rounds of revisions were made (during one-to-one
Instructional Design Project 22
trials, after each class session, and after reviewing the professional evaluations). The end result is a new,
online learning experience that can be applied to future classes with effective learning efficacy.
Validity
In order for formative evaluations to be effective, there "must be a direct relationship between
instructional objectives and assessment measures" (Morrison et al., 2007, p. 258). If the instructional
objectives are supported by the evaluation questions and responses, then the instrument has a high degree
of validity. To this end, the team included short essay responses to support each yes/no question in the
evaluation. Validity was ensured by the quantity of short essays provided, indicating feedback on a
deeper learning level. If the evaluations simply had yes/no responses, students could answer questions
without really thinking about how it applied to the learning objectives and their own learning experience.
The team received a short-answer response rate of 20%, responding deeply to 55 out of 279 questions.
Thus, one out of every five questions provided the team with valuable feedback. Educational
professionals provided short-answer essays to almost every question.
Another sign of evaluation test validity is the use of multiple data sources and multiple
instruments. As mentioned above, the LLDT Collective used two data sources (two class sessions and the
feedback of five educational professionals), allowing feedback from 36 individuals. Two evaluation
instruments were used; one for the learners and a different evaluation for the educational professionals.
The diversity of the data sources and instruments ensured minimal bias on the feedback.
Assessing Knowledge
To determine program effectiveness, the LLDT Collective assessed knowledge from two
perspectives. The first assessment was through student and educational professional evaluations that
occurred after the course materials were implemented. The second assessment involved providing
mechanisms for grading the materials on Springboard as if students would receive an actual grade for
their work. While the latter assessment did not contribute to the learners' grades, creating grading
instruments in the Springboard system enabled the team to connect them with the learning objectives.
Instructional Design Project 23
The student evaluations were objective tests that were comprised of nine yes/no questions. While
this type of test is fast and easy to complete (ensuring a high response rate), responses reveal only lower
levels of learning. To address this deficiency, the team included constructive elements to questions by
offering space for short essay responses. This way, learners could explain why they chose their yes/no
response, if desired. This type of response assesses a higher level of learning. Performance testing was
also utilized via the course assignments, the "Wikispace Website Evaluation Activity" in particular. For
this assignment, learners were evaluated on how they determined the validity of two websites.
From an attitudinal viewpoint, instructors Tim Sisson and Betty Rogge observed the learning
sessions while learners completed the team's materials. They addressed any questions that arose, taking
note of positive or negative comments and behaviors that came to surface. The team recognized attitude
related responses in the short answer responses as well, though these are more prevalent verbally and
visually during implementation. The professional educators provided attitudinal feedback data that the
team also used to revise the learning materials and methods.
Evaluation Standards
The LLDT Collective used pretesting to not only indicate learner preparedness for the learning
activities, but also to serve as a baseline performance to judge how much was learned at the end of
training. Some reported that the pretest questions were too easy and/or realized that what they learned
from the team's materials awoke what they already knew. Relative evaluation standards were not used for
these evaluations; there was no need to compare learners' performance with each other. The team chose
standards based assessments that involve real-world tasks, such as the “Wikispace Website Evaluation
Activity”. The responses received on the evaluation instruments themselves provided feedback from
these real-world tasks. Student self-testing was not utilized, although progress checklists were provided
for each learning module so that students could self-track their progress.
Instructional Design Project 24
Evaluation Results
The team summarized all of the short essay responses to evaluate the effectiveness of the learning
materials and delivery methods. From the educational professionals' perspective, the overriding theme of
responses centered upon the appropriateness of content and how well it aligned with the course
objectives. Professionals stated that the "level of learning matched higher education" and that the
"activities and quizzes matched instructional objectives". The detail and conciseness of the assignment
grading rubrics was positively commented upon as well. Teaching methods were appropriate to the
content and the time commitment of one week was sufficient to complete all materials and activities.
Given that this feedback came from experienced fellow teachers, the team ensured that it was valid.
In terms of content, both learners and educational professionals valued the readings, videos, and
diversity of activities. Students (for the most part) liked the use of discussion boards and wished that
more classes would use them to exchange ideas. The team’s liberal inclusion of learning materials were
"a good reference tool to find information and answer questions." The activities themselves were "cool,
but challenging." Some students even wanted to know if this particular class would be offered fully
online in the future and stated that they would “prefer to take this course online instead of in-class.”
Lastly, both audiences complimented the team on the organization and layout of the learning
materials on Springboard. Students commented that the content was "easy to navigate; everything was
explained clearly" and that the "website was easy to follow." Educational professionals appreciated that
the content was "very organized”. Both audiences appreciated how the learning activities were explained
well.
The following are comments made by the educational professionals:
Jackie Ashbaugh: "I think you did an outstanding job on this project."
Lori Brinker: "Great work! Articles were informative. Would meet needs of a
variety of learning styles."
Patsy Malavite: "Great work on this!!!! Impressed!!"
Betty Rogge: “It is a very good start.”
Instructional Design Project 25
Denise Uitto: “Meets different learning styles.”
Constructive criticism was provided in five major categories: organization, content, instructions,
pacing, and technical mechanisms. What the team found interesting was that the positive feedback
received above included just as much criticism below. While initially conflicting, this feedback is useful
as each learner has a diverse learning style which needs to be accommodated. The team took all of the
feedback seriously and revised the learning materials and methods to address the widest range of learning
styles as feasibly possible. Evaluation results showed more constructive criticism after the first class
piloted the modules. After numerous revisions and a second class piloting, there was much less
constructive criticism.
In terms of organization, the overriding criticism was that there were "too many steps to navigate
the required content." Learners had to jump back & forth between pages to read the instructions, find the
content, and perform the activities. While the team did its best to minimize the location of materials, it is
the overall layout of Springboard that contributes to this "organizational flow" problem. The best effort
was made to minimize this confusion with navigational tips throughout the materials, but mechanics of
the learning management system could only be simplified so much. A few learners also noticed that the
"instructions were all over the place," indicating that in the team's effort to explain the navigation of
Springboard, there was too much explanation. Learners suggested combining some of the content
together so that the initial presentation is not overwhelming. The LLDT Collective found it interesting
that the majority of learners with little to no online experience were the ones who found it easiest to
navigate Springboard. Since the majority of constructive criticism revolved around navigating
Springboard, the team added another module that includes tutorials for navigating Springboard,
VoiceThread, and Wikispace.
From a content perspective, while learners appreciated the varied reading materials and videos, a
few thought that some of the content "may be too advanced for the students’ levels." This opinion
conflicted with most of the educational professionals’ viewpoints, as they stated that the content matched
the instructional objectives quite well and was at the appropriate learning level. Learners also suggested
Instructional Design Project 26
providing multiple discussion topics to allow choice in responding to what they were most interested in.
One student noted that the pretest was repetitive, and one professional noted that it was too easy.
However, these instruments were intended as pre-assessments of learning before instruction and were not
intended to be difficult nor challenging. That was the intended design of those activities. Morrison et al.
(2007) asserted that “a pretest is designed to heighten the student’s awareness of the content by serving as
cues to the key points” (p. 171).
The instructions for using the content received some criticism, but also received plenty of
suggestions for improving its presentation and usefulness. Some of the feedback concerns revolved
around using the discussion board and navigating Springboard in general. The team provided more
supplemental resources and instructions to get them started as quickly as possible without detracting from
the assignments at-hand. While one learner stated that the rubrics "were vague", the educational
professionals stated they were very concise. Again, the difference could be how they are interpreted (the
viewpoint), which must be taken seriously. Some learners stated that overall, the directions were "hard to
follow", "hard to figure-out what [I] was supposed to do", and that the "assignment directions [were]
confusing". The team went over the instructions several times, trying to optimize them before the second
class implemented the learning materials. Again, there was far less confusion after the second class
piloted the materials. The educational professionals’ feedback provided the most value as their criticisms
were followed by suggestions to improve these areas. Patsy Malavite's contributions were especially
helpful. A LLDT team member met with Patsy after the pilot to fully understand and realize the
numerous suggestions that she made to improve the learning materials and delivery methods.
Pacing was also criticized in the evaluations, especially during the first class session. Learners
only had an hour to complete the materials that were designed for two weeks; one week for each module.
This frustrated a number of learners because they weren’t aware that the content they were trying to
complete in one class session was designed to be completed over a two week period. Looking back, the
team should have adjusted the quantity of materials to allot for the limited time. This adjustment was
made for the second class, which completed a minimal set of learning materials in the time allotted.
Instructional Design Project 27
Educational professionals agreed that each module could, in fact, be completed in a week's time, which
meets the design goals of this project.
Again, the LLDT Collective took all of the compliments and criticisms seriously and made
numerous revisions to the learning materials, instructions, and delivery methods to optimize the
experience as much as possible. Although the second class had far less constructive criticism, the team
would have liked to implement a third pilot to assess the effectiveness of these revisions had the team
received additional time. Based on the improvements and consistent positive feedback by the educational
professionals as well as the majority of students, the team feels that the team’s goals have been met, the
team’s problem has been solved, and materials are ready to be included in future classes that cover this
content.
Conclusion
Using the ADDIE model, the LLDT Collective created two online instructional modules for the
Educational Technology course. The team defined the learners, analyzed the course’s current content,
developed learning objectives, and designed relevant assignments/assessments. The team implemented
the learning modules with undergraduate students and educational professionals. Through evaluations, the
team learned of ways to improve the modules. Transferring the face-to-face Educational Technology class
to fully online has had a successful start, and hopefully the educational professionals will implement the
LLDT Collective’s instructional modules within their future courses.
Instructional Design Project 28
References
Bhola, H. S. (1982). Planning change in education and development: The CLER model in the context of a
mega model. Viewpoints in Teaching and Learning, 58, 1-35.
Bhola, H. S. (1988-1989). The CLER model of innovation diffusion, planned change, and development:
A conceptual update and applications. Knowledge in Society: The International Journal of
Knowledge Transfer, 1, 56-66.
Morrison, G. R., Ross, S. M., & Kemp, H. K. (2007). Designing Effective Instruction (5th ed.). Hoboken
NJ: John Wiley & Sons, Inc.
NETS Project (2007). National Educational Technology Standards for Teachers: Second Edition.
Washington, DC.: ISTE.