learning need · title: learning need author: szb-gb created date: 10/3/2013 8:37:04 am
TRANSCRIPT
Instructional Design Plan 1
Instructional Design Plan: Writer Self-Checks
Sara Bryan
Instructional Design Plan 2
Abstract
Writers for the MiracleLearn Corporation use company-proprietary authoring software to
write content for web-based interactive multimedia instruction (IMI). Writers script
lesson content and then enter the information into lesson storyboards stored on a secure
server. The data entry process is not difficult, but it is labor-intensive because of the
attention to detail required. MiracleLearn follows an established quality control (QC)
process that requires writers to conduct self-checks of their work before submitting
storyboards for editorial review. This paper describes the design elements for the
development of a training program that will teach writers to conduct comprehensive,
effective self-checks. The training program is intended to replicate the writers’ work
environment through an e-learning medium with self-paced and branched programmed
learning.
Instructional Design Plan 3
Table of Contents
I. Learning Need.............................................................................................................. 4
II. Proposed Solution ....................................................................................................... 4 Needs Analysis ............................................................................................................ 4
Client and Background ............................................................................................ 4 Purpose of Instructional Intervention ....................................................................... 5
Implications ............................................................................................................. 6 Key questions ........................................................................................................... 6
Processes ................................................................................................................. 7 Target population .................................................................................................... 7
Hypothetical Results ................................................................................................ 7 Recommendations .................................................................................................... 9
Task Analysis ............................................................................................................ 10 Process .................................................................................................................. 10
Techniques for Data Gathering .............................................................................. 10 Lesson Content ...................................................................................................... 12
Writer Self-Check Content Outline ......................................................................... 12 Flowchart .............................................................................................................. 14
Learner Contextual Analysis...................................................................................... 16 General Characteristics ......................................................................................... 16
Specific Characteristics ......................................................................................... 16 Impact of Learner Characteristics on the Design of the Instruction ........................ 17
Orienting Context .................................................................................................. 18 Instructional Context ............................................................................................. 20
Application Context ............................................................................................... 20 Learning Objectives .................................................................................................. 21
First Objective ....................................................................................................... 21 Second Objective ................................................................................................... 22
Third Objective ...................................................................................................... 22 Assessment Plan ........................................................................................................ 23
Objective Measures ............................................................................................... 23 Validity .................................................................................................................. 24
Learning Strategies .................................................................................................... 25 Learner Analysis .................................................................................................... 25
Learning Theories.................................................................................................. 26 Learning Strategies ................................................................................................ 26
Delivery Strategies .................................................................................................... 29 Delivery Medium ................................................................................................... 29
Delivery Strategy for Learning Objectives ............................................................. 30 Delivery Strategy for Assessments .......................................................................... 30
Delivery Strategy for Instructional Context ............................................................ 31 III. Insights .................................................................................................................... 32
IV. References ............................................................................................................... 34
Instructional Design Plan 4
I. Learning Need
Writers for the MiracleLearn Corporation use company-proprietary authoring
software to write content for web-based interactive multimedia instruction (IMI). Writers
script lesson content and then enter the information into lesson storyboards stored on a
secure server. The data entry process is not difficult, but it is labor-intensive because of
the attention to detail required.
Writers are required to proof their work before submitting it for quality control
(QC) review. The problem is that seemingly insignificant errors are often overlooked,
particularly in the fields that affect courseware navigation. An incomplete field or one
extra space, for example, will prevent one screen from advancing to the next. Such
oversights prolong the QC process and risk delays in project completion. A further
concern is the potential loss of business resulting from project delays or faulty
courseware.
II. Proposed Solution
Needs Analysis
Client and Background
The clients for this project are writers employed by the MiracleLearn Corporation.
In the past decade, MiracleLearn has secured and completed contracts encompassing
hundreds of hours of web-based IMI for military specialties. Using MiracleLearn’s
proprietary software, called InterAct, writers script lesson content and enter the
information into storyboards stored on a secure server. Lesson content is entered one
screen at a time. In addition to task headings, each screen requires detailed navigational
and content information, and scoring information for screens that contain questions to
Instructional Design Plan 5
check student learning. If any of the information is incorrect or missing, the courseware is
subject to quality defects or performance failure.
When the storyboards are complete, writers submit them to an editor who checks
storyboards for narration and text concerns, such as conversational flow, sentence
structure, spelling, and punctuation. Before submitting, it is the writer’s responsibility to
look for these and other errors that might affect courseware functionality once the lesson
is produced. Unless the writer himself detects functionality flaws at the beginning, the
errors can remain unnoticed until the lesson reaches the production stage.
The inadequacy of writer self-checks may be attributable to two factors. The first
is a lack of understanding about the QC process, in particular each writer’s own
importance as a preliminary step in the process. The second factor is a lack of writer
knowledge about how to check their work or what to look for.
Purpose of Instructional Intervention
Writers must understand the company QC process and their impact on the success
or failure of that process. Although writers are given training to use the InterAct software,
the detail required for accurate data entry leaves the door open for errors that are easily
overlooked. Compounding the problem is the iterative nature of project development. As
soon as a writer submits one lesson to an editor, he begins scripting another. When
storyboards are returned for correction, the writer must interrupt the lesson he is currently
scripting to make corrections and submit an amended storyboard for the next phase of
QC. Storyboards requiring extensive correction at any point in the process will slow
progress of both the lesson undergoing QC and the lesson currently being scripted. It is
Instructional Design Plan 6
therefore imperative to ensure writers have the skills to identify and correct common
errors that occur during the data entry process.
Implications
A continued cycle of submissions, corrections, and interruptions has a
compounded negative effect on the production schedule. Of particular concern are errors
or deficiencies that affect courseware functionality or the accuracy of narration. Once a
lesson has begun the production phase, narration is one of the most expensive errors to
fix because of the additional scheduling and man hours required to re-record and re-
digitize audio.
Key questions
Morrison, Ross, & Kemp (2007) provide guidelines for conducting needs and goal
analyses. Among the guidelines is the recommendation to conduct interviews to establish
a broad perspective of a performance gap in order to best determine whether a training
intervention is appropriate. With this guideline in mind, the following questions were
asked:
Are writers aware of the entire project QC process?
Is there an established procedure for conducting and tracking self-checks? If so,
what is that procedure?
How much time is allotted for writer scripting, including self-checks?
What is the review process within the QC department?
How much time is allotted for storyboard review by the QC department?
How much time is actually spent per lesson conducting storyboard QC?
What is the average number of errors detected during storyboard review?
Instructional Design Plan 7
What types of errors will cause product functionality failure if not corrected?
Processes
MiracleLearn is a geographically dispersed corporation, and many employees work
from home or a location other than headquarters. Writers and editors work independently
of one another and support staff. Because of the geographic spread, an informal approach
consisting of telephone conversations, instant messaging (IM) and email will be used to
assess training needs. The rationale for this approach is that it is expedient. Writers, QC,
and other project personnel work autonomously and are already comfortable responding
to direct communication from a variety of sources. The company also has an unofficial
policy endorsing the use of IM for daily communication. Project management has
historically been very supportive and non-judgmental of writers’ work; therefore, it is
anticipated that no threat will be perceived by a direct approach and employees
questioned will respond in a spirit of cooperation.
Target population
The primary target population consists of technical writers charged with two principle
tasks: first, to develop IMI lessons from outlines provided, and second to use the
company-proprietary authoring tool to create the lesson storyboard. A secondary
population is the project editors, who are tasked to check storyboards for appropriate
writing style and mechanics. Editors are not required to check for software technicalities
but would benefit from an awareness of the requirements.
Hypothetical Results
Writers are aware of and generally familiar with the steps in the project QC
process. Recent emphasis (i.e., within the past six months) by project management has
Instructional Design Plan 8
stressed the importance of “looking over” storyboards before submitting them for the
initial storyboard review by editors. However, there is no training or consistent guidance
available for new writers.
No current procedure has been established for self-checks. There is an example
online document for troubleshooting production reports, but it is of minimal value. The
document is extremely out of date, having been created at the inception of the authoring
tool, which has since undergone several revisions. The document is also stored in an area
so obscure that most writers – indeed, most project personnel – are unaware of its
existence.
Time allotted for scripting storyboards varies from five days to two weeks. The
variance is dependent upon the estimated size of the lesson, writer workload, and
scheduled delivery date. Writers felt that, for the most part, they were given adequate
time to complete first lessons within a project. Subsequent lessons, however, sometimes
proved problematic when work was interrupted to attend to corrections and revisions
from previous lessons. On occasion, writers found themselves alternating between two,
and as many as four, lessons in various stages of development or quality control. At such
times, writers tended to correct only the comments identified on a particular screen
without checking to see whether changes impacted other screens within the lesson.
The entire QC process for a project encompasses six levels. The first three levels
are pre-production reviews; the next three levels are technical and post-production
reviews. Training for this needs assessment is geared only to pre-production reviews,
levels 1 through 3. At level 1, the writer submits the storyboard for editorial review. At
level 2, the writer submits the amended storyboard for SME review. At level 3, the writer
Instructional Design Plan 9
submits an updated report for narration review and first-stage production. QC personnel
are given one to four days from the day of receipt to review and return lessons to writers.
Time allocated is determined by the number of lessons scheduled in the reviewer’s queue.
While the writers are working on only one project at a time, reviewers are typically
evaluating documents from two or three different projects simultaneously. For this
reason, reviewers require a moderately flexible window for reviewing time rather than a
specific schedule. Time actually spent reviewing each lesson is one to two hours, with an
average detection rate of approximately eight errors per each ten pages. Of that number,
approximately 20% are critical errors.
Recommendations
Training on self-checks is needed for new writers as a follow-up to their initial
training on using InterAct. Experienced writers would also benefit from the training as a
refresher and a way to learn more efficient methods of checking their own work. A six-
hour seminar is recommended that includes the following components:
QC process overview, with emphasis on the writer’s impact on the process
Running (generating) a production report, which creates a competed Word
document to review
Reviewing a production report
Distinguishing between critical and quality errors
Comparing reports to the online storyboard
Tracking corrections
Instructional Design Plan 10
Task Analysis
Process
Morrison, Ross, & Kemp (2007) explain that the topic and procedural analysis
methods are suitable for concrete and highly structured tasks, while critical incident
analysis is appropriate for observing and defining interpersonal skills and attitudes.
The creation of a storyboard is a highly detailed cognitive process; consequently, self-
checks must be equally detailed. At the same time, specific procedures are necessary to
formulate a systematic method for self-checks. Additionally, the development of a
proactive attitude toward self-checks is desirable in order to foster the necessary attention
to detail. Tasks that lead to the mastery of successful self-checks encompass all areas
review; therefore, any of the recommended analysis methods might have been
appropriate. However, the primary emphasis was on cognitive domain. Accordingly, a
topic analysis was conducted to determine instructional content for the writer self-check
training.
Techniques for Data Gathering
Data for the project was gathered from a survey of existing materials and through
subject matter expert (SME) interviews. Information from both sources was recorded as
Microsoft TM Word documents and stored in the instructional designer’s project file. A
synopsis of the findings follows for existing materials and SME interviews.
Existing materials include a guide for developing interactive courseware (ICW), a
writer’s handbook, and a project style guide (PSG). The ICW guide details the production
process and is oriented primarily for management; however, it is a useful context
resource for all multimedia team members because it also lists roles, responsibilities,
Instructional Design Plan 11
typical timelines, and best practices. All of these points of information serve to orient the
writer to the significance of his role in the courseware development process. The writer’s
handbook provides general information about writing for web-based training, writing for
instruction; and specific information related to scripting for narration, graphics, and text.
Information in the writer’s handbook provides the foundation for the knowledge base
writers must have in order to develop lessons in InterAct. The PSG is a document created
at the inception of each project to establish conventions for lesson development such as
structure, jargon, and rules for scripting narration and text, such as the use of acronyms in
narration, pronunciation and written form of military terminology, equipment
nomenclature, and approved abbreviations. It is a key document for prescribing the
format of information entered in the storyboards.
SMEs are the editors, who serve both as first-line QC personnel and the primary
content resource for project. Two editors review storyboard content for quality in
separate general areas: writing style and data entry accuracy. The style editor critiques
numerous items such as sentence structure, syntax, and word choice and order. At the
same time, storyboards are evaluated for adherence to the PSG. The data entry editor
monitors the presence and correct application of software features such as templates,
branching, and scoring information for questions related to learning checks and practice
exercises. Interviews with SMEs determined that common style errors include writing
acronyms in a way that the narrator knows how to pronounce them, punctuation or
spelling errors, and adding double instead of single spaces between sentences. Common
data entry errors include missing, incorrect, or incomplete coding information such as
branching and template designations. An additional problem cited by the SMEs was an
Instructional Design Plan 12
apparent reluctance by the writers to take the time to conduct comprehensive self-checks,
ostensibly from the belief that “QC would find the mistakes.”
Lesson Content
The proposed content will consist of three phases of online multimedia
instruction. The first and third phases are synchronous, consisting of web-ex
presentations by project managers to introduce and summarize the training. The second
phase constitutes the bulk of the training in the form of two or more asynchronous
lessons that detail procedures and techniques for conducting self-checks. Information
listed in Section IIB, Self-check solutions, will be developed in detail to show writers
exactly where and how to identify storyboard errors. Content for this section may be
expanded pending ongoing SME reviews, and will be submitted for approval by the
corporate project coordinator before beginning course development. A preliminary
outline of lesson content follows.
Writer Self-Check Content Outline
I. Lesson Introduction (synchronous training)
A. Why conduct self-checks
1. Eliminate negatives
a. Resistance to conducting self-checks
(1) Too much bother
(2) QC will catch errors anyway
b. Consequences for not checking
(1) Causes a domino effect of corrections and changes
(2) Creates even more compressed writing schedule
Instructional Design Plan 13
2. Accentuate positives
a. Smoother QC process
b. Less time used to correct QC comments = More time available
for completing subsequent assignments
B. When to conduct self-checks
1. Periodically while entering storyboard
2. At storyboard completion
3. Between QC phases
II. Self-Check Procedures (asynchronous training)
A. Commonly identified QC problems
1. Writing
a. Mechanics
(1) Abbreviations and acronyms
(2) Parallel construction
(3) Numbers: figures versus words
b. Project style guide non-compliance
(1) Narration scripted incorrectly
(2) Text does not match narration
2. Data entry
a. Missing, incomplete, or incorrect information
(1) Templates
(2) Branching
(3) Narration or matching text
Instructional Design Plan 14
(4) Question scoring
b. Extra spaces
B. Self-check solutions
1. Run spell check
a. Run Production Reports (PR)
b. Differentiate between spelling errors versus software coding
c. Use (Ctrl+F) to find extra spaces
2. Review section by section
a. Source file section
b. Navigation section
c. Content section
d. Question section
3. Read what you wrote
a. Scan entire lesson
b. Read narration aloud
III. Summary (Synchronous training)
A. Content review
B. Question/answer session between writers and QC personnel
Flowchart
A representative segment of the proposed training is provided in the flowchart
that follows. During SME interviews, it was determined that the first step for conducting
effective self-checks is to conduct a spell check exercise on a storyboard Production
Report (PR). PRs are Word documents that can be generated from InterAct once the
Instructional Design Plan 15
storyboard has been completed. The flowchart in Figure 1 details the sequence of events
for conducting the spell check.
Figure 1
Flowchart of the procedure for running a production report and conducting a spell check
Instructional Design Plan 16
Learner Contextual Analysis
General Characteristics
Target learners are multimedia content writers. Their primary task is to create
scripts for web-based lessons, most of which are created for military clients. With the
exception of age and possibly cultural differences (unknown), the learners are a generally
homogenous group for the writer self-check project. All are adults with an approximate
age range of 25 – 60 years. Most are at the lower end of the range. Experience levels with
technical/content writing range from six months to ten years. Writers at the upper end of
the age range have experience in other areas such as military service, management, and
education. Most writers have at least a baccalaureate degree; some have master’s degrees.
Project team members work in an online environment and are geographically dispersed.
As a result, many are in contact with one another only via electronic media. Demographic
characteristics other than education and experience are therefore considered to be of
minimal significance.
Specific Characteristics
Entry Competencies. Writers must have familiarity with the InterAct authoring
tool. While entry level writers can benefit, they would do better to have completed at
least one storyboard prior to beginning self-check training. Additional required skills are
presumed, given the nature of the work conducted. These skills include typing skills;
facility with computers; ability to access and work within several programs such as
InterAct, Word, and an Internet browser.
Social/Motivational Characteristics. Most writers are geographically distant from
one another, which may impact the social aspect of the learning environment. The
Instructional Design Plan 17
capability to collaborate, if desired, is somewhat inhibited by the absence of face-to-face
contact with other writers.
Writer motivation to conduct self-checks or to learn how is manifested in the
project work schedule. Writing assignments are deadline-driven. The more aggressive the
project schedule, the more pressured writers are to complete tasks in a timely manner.
SME observations have noted variance in motivation to conduct self-checks, ostensibly
as a result of project timelines. SME assumption is that less experienced writers are
focused on meeting deadlines and will expend efforts to do so at the expense of quality.
The presumed attitude is that QC personnel will find and report errors anyway, making
the writers’ own efforts unnecessary. More experienced writers, on the other hand, are
willing to take time to review their work but tend to demonstrate impatience with the
perceived imposition of additional training or job aids. Whether resistance comes from
meeting deadlines or rejecting training, the common theme throughout the experience
spectrum is a mindset that expresses the idea “I don’t have time for this; I’ve got work to
do.”
Impact of Learner Characteristics on the Design of the Instruction
Social and motivational considerations are paramount during the design of the
instruction. Conducting successful self-checks requires both metacognitive skills and a
willingness to comply. Identifying and correcting errors are cognitive processes, but
applying a strategy to those processes incorporates metacognition. Dobrovolny (2006)
conducted a study to assess how adults learn with instructional technology. Each learner
participated in self-paced, technology-based course of his or her own choosing. Results
indicated that learners relied on techniques such as metacognition, which involves
Instructional Design Plan 18
self-assessment and self-correction; reflection; and prior and authentic experiences. All
such techniques are relevant to the process of conducting self-checks of finished work.
Metacognition is seldom addressed without a corresponding mention of
motivation. In his study of self-regulated learning, Wolters (2003) stated “The specific
characteristics most often attributed to self-regulated learners concern their motivational
beliefs or attitudes, their cognitive strategy use, and their metacognitive abilities”
(Wolters, 2003, p. 189). Motivation is a prerequisite of attention to detail, which is a key
element for successful self-checks. Willingness to commit to the time and attention
required is a concern. While the aforementioned motivation to meet deadlines is laudable,
it is important to stress the value of doing the job right the first time. Consideration must
be given to promoting a more efficient and more effective method to conduct self-checks.
Given the focus on efficient techniques for conducting self-checks, focus must also be
placed on efficient techniques for delivering the training. One technique is to afford
writers the flexibility to access training at will within a specified time period.
Orienting Context
Familiarity, experience, and time are key factors of the self-check project.
Interviews with SMEs and management personnel have indicated writers are generally
well equipped with the first two factors but continually in need of the third. Specifically,
writers have familiarity with InterAct and experience writing storyboards. Time to
complete assignments is a variable factor that can be adversely affected when numerous
storyboard corrections are required after a QC check.
Meeting deadlines is an ongoing concern for an entire project team. A shortfall is
most likely to begin with the writer, whose work is the first phase of the courseware
Instructional Design Plan 19
development process. In the interest of keeping the schedule current, writers have
expressed disinclination to take time for self-checks, thereby demonstrating a preference
for submitting storyboards as soon as they are complete in order to keep things moving
forward. If a storyboard is incomplete or is returned repeatedly for corrections the writer
loses rather than gains time. The writers’ collective goal, therefore, should be finding a
way to maximize time allotments for task completion. Accordingly, the initial portion of
the training should describe value received (gained time for follow-on assignments) in
exchange for value given (conducting the checks).
One way to maximize time is to practice efficient methodology. Self-check
training is intended to establish a system that will enable writers to meet the requirement
for conducting self-checks without an added time burden. Utility of the training is
twofold: first, the opportunity to learn an efficient system for conducting self-checks; and
second, an occasion to refresh storyboard development skills. Improved storyboards will
further reduce the time needed to check and correct them.
Most corporate training initiatives are offered as enhancements to, rather than
requirements of, job performance. Writers will not be held accountable but will be
encouraged to complete the training. A generally positive response is anticipated for three
reasons. First, the training offers a solution to individual time needs. Second, the
asynchronous element offers writers some capability to control their own schedules.
Third, efforts expended individually by writers promote the success of an entire project
team.
Instructional Design Plan 20
Instructional Context
Training will be designed for and presented with the InterAct authoring system.
Writers will require access to a computer, the Internet, and InterAct. Because the writers
will train under the same conditions that they work, no orientation to the learning
environment will be necessary prior to beginning the training. Scheduling concerns are
minimal because the instruction will be designed for online access. Most of the training
will be asynchronous, allowing writers to log in at their convenience any time. To ensure
the instruction is completed within a reasonable time frame, writers will be given a two-
week period to complete the training. Two synchronous sessions, approximately one hour
each, will require coordination and scheduling. Project team members normally
participate in weekly teleconferences conducted by the project manager to share
information about progress and process updates. The synchronous instructions sessions
could be scheduled as part of, or in lieu of, two weekly meetings.
Given the flexible online environment, factors such as lighting, noise,
temperature, seating, accommodations, and transportation are non-issues for this project.
No disabilities have been identified; therefore, no provision is currently required for
adaptive access or technologies.
Application Context
Given that writers will train and work in the same online environment, work
application is inherent to the training. They will be able apply learned skills during the
process of and immediately following training.
Instructional Design Plan 21
Learning Objectives
Morrison, Ross, & Kemp (2007) describe two general types of instructional
objectives, cognitive and behavioral. Cognitive objectives state, in general terms, what an
intended learning outcome should be and to what standard it should be accomplished.
Behavioral objectives are more specific and include a third element, the condition under
which learning is demonstrated. Gebrewold & Sigwart (1997) present the requirements
succinctly by stating that useful objectives include a performance, a condition, and a
criterion. Both types of objectives, cognitive and behavioral, provide a framework for
teachers upon which instructional content is built and learning assessment is made. For
learners, objectives are signposts to follow through the instructional content to the
intended outcome. For the purpose of the Writer Self-Checks project, objectives are
presented in behavioral terms. Three objectives follow for the Writer Self-Checks project.
The first is in the affective domain; the last two are within the cognitive domain.
First Objective
After completing a course of instruction on how to conduct them, appreciate the
value of self-checks. The objective is achieved when the writer exhibits any combination
two or more of the following behaviors:
• Speaks positively about the value of self-checks.
• Acknowledges time saved in the course of the QC process as a result of conducting self-
checks.
• Runs a production report upon completion of each storyboard.
• Runs a spell check of each storyboard.
• Takes the time to review all sections of the storyboard.
Instructional Design Plan 22
• Takes the time to review produced lessons.
• Checks continuity to ensure that corrections made on one screen are made on all related
screens.
Enabling objectives:
Describe the company QC process.
Explain the writer’s impact on the QC process.
Second Objective
Upon completing a lesson storyboard, perform a comprehensive self-check prior
to submitting the script for editorial review. A margin of error is permissible such that the
submitted script will be found to contain no more than one quality deficiency per page
and no critical errors within the storyboard.
Enabling objectives:
Run a production report.
Run a spell check.
Survey all areas of the production report.
Distinguish between critical errors and quality deficiencies.
Third Objective
Given a produced lesson with graphics and audio included, troubleshoot problems
with InterAct functionality. A successful review will resolve all writer-related findings.
Enabling objectives:
Identify errors in Title, General, Branching, and Content fields.
Correct errors in Title, General, Branching, and Content fields.
Instructional Design Plan 23
Check continuity to ensure that corrections made on one screen are made on
all related screens.
Assessment Plan
Morrison, Ross & Kemp (2007) suggest measures for instructional outcomes
based on the areas of knowledge, skills and behavior, and attitudes. The first objective is
within the affective domain and will be evaluated with attitude assessments. The
remaining two objectives for the Writer Self-Checks project are at the analysis level of
the cognitive domain and would be measured appropriately by skill/behavior
assessments.
Objective Measures
The first objective is manifested primarily in the affective domain and is intended
to elicit an attitudinal change. Post-training interviews will be conducted to assess
writers’ opinions about the training and the value of conducting self-checks. Subsequent
assessments will include observation of writers’ actions in regard to conducting self-
checks. Observations will be made by project managers and editors. Interviews offer the
capability of immediate assessment while observations will evaluate change over an
extended period.
Some of the behaviors listed for assessment of the objective within the affective
domain also serve as measures of cognitive achievement. As an alternative to Mager-
style behavioral objectives, Gronlund (as cited by Morrison, Ross, & Kemp, 2007)
suggested cognitive objectives. The difference is that cognitive objectives describe
learning outcomes in more general terms than do behavioral objectives. The resulting
outcome allows for sample performances to indicate behaviors “…that allow the teacher
Instructional Design Plan 24
or instructor to infer that the learner has achieved the higher-level intent” (Morrison,
Ross, & Kemp, 2007, p. 115). Sample behaviors for the first objective that also serve as
cognitive assessment measures include running a production report and running a spell
check.
The second objective is intended to elicit a behavioral outcome that is best
measured by the direct testing approach. Writers will be given a sample storyboard
containing both quality and critical errors for the learner to identify. Writers will not only
identify all errors, but will also classify them as quality or critical.
As with the second objective, a behavioral outcome is expected for the third
objective that is best measured by direct testing. Writers will be shown a series of
example screens containing flaws caused by writer error. Associated storyboard pages
will accompany the screens. Writers will identify the flaws on the screen and then review
the storyboard page to determine the cause of the malfunction.
Validity
Guidance for measurement validity is that “A test is considered valid when it
specifically measures what was learned, as specified by the instructional objectives for a
unit or topic” (Morrison, Ross, & Kemp, 2007, p. 244). The concept is echoed on a
broader scale by Clarke, Stow, Ruebling, & Kayona (2006), who discuss considerations
for curriculum assessment. Among the considerations for developing curriculum for
school organizations, Clarke, et al. state that curriculum must be aligned with what is
taught and tested. The same principle applies to training developed for a work
environment.
Instructional Design Plan 25
The measures prescribed for all three objectives of the Writer Self-Checks Project
match the action specified and closely represent the actual behaviors expected from
writers in the performance of their jobs. The first objective describes the desired attitude
shift after completion of the training. While a questionnaire or Likert-type scale could be
used as an assessment tool, it is anticipated that interviews and observations will generate
more useful data. The behavioral objectives specified for the project prescribe learning at
the analysis level of the cognitive domain. Writers are expected to review their work for
quality and functionality. The review of sample storyboards and developed screens
emulates activities that writers will conduct after completing the training. Errors placed in
the storyboards and screens will be similar to errors commonly found during editorial
reviews.
Learning Strategies
Learner Analysis
Learners for the Writer Self-Checks Project are educated, competent, and
conscientious workers who are accustomed to operating under minimal supervision. The
writers’ education, experience, and online environment are key factors in the decision for
learning strategies. Kelly (2006) relates that important elements for successful learning
include an understanding of learners’ backgrounds and expectations, and setting an
environment conducive to learning. The writers have in common an ability to
communicate well both orally and in writing, which enables them to work autonomously
in an online environment and which offers a degree of control about when and how the
writers work. Consequently, there is an expectation among the writers to control what,
Instructional Design Plan 26
when, and how they learn. Strategies developed for this project are therefore oriented
toward replicating the work environment in a training scenario.
Learning Theories
Strategies for writer self-check objectives will be founded primarily on
constructivist theory, underscored by cognitive theory. Writer self-check skills are
procedural in nature but are founded on existing knowledge. The ability to build on
existing knowledge is inherently a metacognitive skill requiring learner self-awareness.
Similarly, a writer’s ability to understand and draw upon his or her own methods for self-
check requires self-awareness.
Underhill (2006) describes constructivist theory as largely student-centered.
Writers are charged with checking their own work prior to submitting it for quality
review; therefore a student-centered approach appears to be the most appropriate for
promoting an environment writer-centered initiative after the training.
Writer self-check training incorporates an affective domain element in concert
with cognitive procedure, metacognitive awareness, and constructivist experience. To be
fully successful, writers must exhibit a willingness to apply themselves to a self-check
effort. Dabbagh (2006) presents learning strategies for both cognitive and affective
domains, which align with both cognitive and constructivist learning theories.
Learning Strategies
Course content will be based upon recommendations presented from the project
needs analysis. Based upon this content, plans follow for the initial presentation,
generative strategies, and learning strategies for each of the three objectives.
Instructional Design Plan 27
First Objective
Initial presentation. Project managers and QC personnel will facilitate a
discussion of writer impact on successful versus marginal self-check efforts. Writers will
be encouraged to participate in the discussion to voice concerns, questions, and
reflections.
Generative strategy. Generative strategies are organization and elaboration.
Emphasis is on elaboration, because it is at this point the writers are encouraged to
present their own ideas and recommendations.
Learning strategy. Modeling and prior learning are key strategies for eliciting
affective change. Krathwohl, Bloom, and Masia (as cited by Milller, 2005) established a
taxonomy for learning classifications in the affective domain. The high end of the
taxonomy describes an orientation toward or identification with the change. It is this
outcome toward which self-check training is directed. Miller also cites several learning
theories for aptitude formation and change. Among them are social learning, which
suggests a powerful model as a learning intervention; and Krathwohl’s taxonomy, which
indicates that affective learning at any level is dependent on learning at lower levels.
Presentations and reinforcement by managers along with the writers’ own testimonials is
intended to establish the requisite modeling and learning strata.
Second Objective
Initial presentation. A guided tutorial will be presented on the importance of
conducting self-checks and the procedure for running a production report. A sample
report will be displayed that highlights each section in turn and specifies where errors
commonly occur.
Instructional Design Plan 28
Generative strategy. Generative strategies range from integration to elaboration.
Morrison, Ross, & Kemp (2007) explain that integration helps change information to a
form that is more easily remembered. Organizational strategy is a connective approach
that enables learners to relate existing ideas to new ones. Elaboration pushes learners to
create or add new ideas through such techniques as mental imagery or creating diagrams.
Learning strategy. Modeling and explaining will be given heavy emphasis for this
objective. Showing writers how to create a production report is the first element of the
integration strategy. The learning element will be reinforced by having writers create
production reports themselves. Guiding writers through the report is an organizational
element that reinforces information writers already have about what goes into the report
when lesson data is being entered into the InterAct authoring system. Mental imagery
will be promoted by highlighting each section of the production report as a visual cue of
where to focus when conducting self-checks.
Third Objective
Initial presentation. Writers will be given a review of information from the first
objective. They will then be shown a sample lesson that contains embedded errors.
Generative strategy. Generative strategies include will evolve to a closer focus on
organization and elaboration. Writers will build on skills learned or refreshed from
creating and reviewing a production report by reviewing a sample lesson that has been
produced from a storyboard. They will elaborate on their skills by identifying errors
implanted in the sample lesson, discussing additional pitfalls, and posing solutions.
Learning strategy. Training for the second objective will include comprehension
monitoring strategies. Weinstein (as cited in Dabbagh, 2006) explains that
Instructional Design Plan 29
comprehension monitoring encourages students to create and monitor their own goals.
The intent for the self-check training is to foster both metacognition and an appreciation
for the effort to conduct effective checks. Comprehension monitoring activity will begin
with at the inception of training by explaining and encouraging the value of self-checks.
Authentic learning activity and self-directed learning are key strategies for this objective.
The training environment and tools are the same ones utilized when writers develop
actual storyboards. Writers will be expected to review, identify, correct, and explain
errors and malfunctions in the sample lesson.
Delivery Strategies
Delivery Medium
All training will be delivered in an e-learning environment. Writers are
accustomed to working and conferencing via distance media, and it is not economically
feasible to gather all employees at one location. Scheduling on-site training would be
difficult for two reasons. First, the writers and QC personnel are geographically
dispersed. Second, project deadlines invariably become compressed as each project
progresses. The corporation usually has several active projects in varying stages, so the
number of writers available is often small for training at a particular time.
Given that most employees are accustomed to online working and
communicating, an online training session is also recommended that incorporates both
synchronous and asynchronous segments. The synchronous segments would be a one-
hour kickoff meeting and a one-hour summation conducted via web conferencing. The
asynchronous training would occur between the scheduled web conferences. A one-week
time frame would be established for the completion of all training components.
Instructional Design Plan 30
Clark (1999) indicated that two advantages of self-paced instruction are that it
allows learners to proceed at their own pace and it is often less costly than classroom
instruction. Therefore, the training will be primarily self-paced but bracketed with
synchronous web-ex conferences to allow monitoring, reflection, and opportunities for coaching.
Delivery Strategy for Learning Objectives
In support of the first objective to appreciate the value of self-checks, an introductory
web-ex will be conducted by project managers and quality control personnel. The
purpose is to provide preliminary information about the importance and impact of
successful self-checks. Conferencing software will be used to enable writers to see
example storyboards while points of information about them are discussed. Also during
the conference, writers will be given information about when and how to access self-
paced IMI lessons covering the second and third objectives.
All job performance tasks are completed at the writers’ workstations. Learning
objectives leading to self-check competence are directly related to job tasks and will also
be performed at writers’ workstations.
Delivery Strategy for Assessments
Assessment for the objectives consists of interviews and direct testing. After the self-
paced lessons have been completed, interviews will be conducted during a second web-ex
to assess writers’ opinions about the training and the value of conducting self-checks.
To evaluate the second and third objectives, direct testing will be used as described
by Morrison, Ross, & Kemp (2007). Writers will be given a sample storyboard
containing both quality and critical errors to identify. Writers will not only identify all
errors, but will also classify them as quality or critical. After identifying storyboard
errors, writers will be shown a series of example screens containing flaws caused by
Instructional Design Plan 31
writer error. Associated storyboard pages will accompany the screens. Writers will
identify the flaws on the screen and then review the storyboard page to determine the
cause of the malfunction.
Because this is an online interactive environment, writers will identify errors by
clicking the place where they appear in the storyboards. Prompts and follow-on activities
will be generated depending on the writer’s inputs. Writer responses will be
electronically scored and displayed immediately upon completing each activity. The
scores will be stored on the server where the IMI is accessed. Project managers will have
access to the writers’ data to determine lesson completion status and final scores.
Delivery Strategy for Instructional Context
Training will be designed for and presented with the InterAct authoring system.
Writers will require access to a computer, the Internet, and InterAct. Writers will train
under the same conditions that they work; therefore, no orientation to the learning
environment will be necessary prior to beginning the training. Scheduling concerns are
minimal because the instruction will be designed for online access. Most of the training
will be asynchronous, allowing writers to log in at their convenience any time. To ensure
the instruction is completed within a reasonable time frame, writers will be given a two-
week period to complete the training. Two synchronous sessions, approximately one hour
each, will require coordination and scheduling. Project team members normally
participate in weekly teleconferences conducted by the Project Manager to share
information about progress and process updates. The synchronous instructions sessions
could be scheduled as part of, or in lieu of, two weekly meetings.
Instructional Design Plan 32
Logging in to InterAct and accessing the topics constitutes the self-paced portion of
the training. Although writers will be working at their own pace, the training is not
without specific direction. Branched programmed learning will be developed as described by
Clark (1999). Although essentially a linear approach, branched programmed learning is
more complex in that it will direct the learner to different parts of a lesson according to
the way he responds to prompts. Writers are familiar with branched programming
because it is the same method used for most of MiracleLearn’s IMI projects.
Modeling and prior learning will be emphasized to elicit the attitude change
needed to achieve the first objective. The second and third objectives will employ
modeling and explaining; and comprehension and monitoring, respectively.
Training lessons will be presented online in the same format that writers use to
develop product lessons for clients. Writers will log in to the InterAct program just as
they do when they are working. Instead of accessing a lesson for scripting, however,
writers will access the training lessons and follow all prompts in the same way that a
client user would. Writers are effectively “trading places” with a MiracleLearn client and
experiencing an example of the kind of work the writers themselves create. From the
perspective of a user rather than a writer, the writers will see a model of what they should
be creating, and have a hands-on example for conducting self-checks.
III. Insights
Morrison, Ross, & Kemp (2007) tell us that not every performance problem needs
to be solved by instruction. In the case of writer self-checks, for example, one solution
might have been to alter project schedules to allow more time for lesson development and
self-checks. Another solution might have been simply to create a job aid in the form of a
Instructional Design Plan 33
checklist or annotated storyboard for writers to follow. A reward and punishment system
is yet another possibility.
I’m confident that training is the right solution for the need I’ve identified. As I
completed the process, I reflected on the many elements of instructional design that we
have learned or reviewed over that past ten weeks. Developing each part of this project
entailed an enormous amount of work and research, as my list of references will attest.
Far more time was spent developing each phase of a design plan than was spent
identifying the reason for the plan. That’s appropriate; learning the steps of instructional
design was the point of the course. But even so, a thought occurred to me: What if the
learning need initially identified were not the problem at all? What if, for example, the
editors needed to learn more about how InterAct works before evaluating the writers’
storyboards? Or that project managers needed to exercise more careful oversight? Or that
clearer skill standards needed to be established before hiring writers?
A thorough needs analysis would answer most such questions – if they arose. I’m
not sure they would. In the future, I think I’ll give some attention to what I now believe is
an overlooked area: need identification, the point before a front end analysis even begins.
What’s important is to ensure a problem or need is correctly identified. If that first step
hasn’t been done completely or accurately, then any design procedure that follows is
suspect and potentially a huge waste of time and resources. Solving the problem right is
point of instructional design. But first, be sure you’re solving the right problem.
Instructional Design Plan 34
IV. References
Advanced Distributed Learning. (2007). History of ADL. Retrieved September 3, 2007
from http://www.adlnet.gov/about/history.aspx
American Society for Training and Development. (2006). Professional continuing
education, knowledge management, and the workplace: A growing industry
[Electronic copy]. ASHE Higher Education Report, 2006, 32(2), 17-25. Retrieved
September 3, 2007 from the EBSCOhost database.
Clark, D. (1999). Training media dictionary. Retrieved August 27, 2007 from
http://www.nwlink.com/~donclark/hrd/media.html
Clarke, N., Stow, S., Ruebling, C., & Kayona, F. (2006) Developing standards-based
curricula and assessments. Clearing House, 79(6), 258-261. Retrieved August 14,
2007 from the EBSCOhost database.
Dabbagh, N. (2006). The instructional design knowledge base. Retrieved August 25, 2007
from Nada Dabbagh's Homepage, George Mason University, Instructional
Technology Program:
http://classweb.gmu.edu/ndabbagh/Resources/IDKB/index.htm
Dobrovolny, J. (2006). How adults learn from self-paced, technology-based corporate
training: New focus for learners, new focus for designers. Distance Education,
(27)2, 155–170. Retrieved August 10, 2007 from the EBSCOhost database.
Gebrewold, F. & Sigwart, D. F. (1997). Performance objectives: Key to better safety
instruction. Professional Safety, 42(8). Retrieved August 18, 2007 from the
Ebscohost database.
Instructional Design Plan 35
Kelly, M. (2006). Teach an old dog new tricks: Training techniques for the adult learner.
Professional Safety, 51(8), 44-48. Retrieved September 1, 2007 from the
EBSCOhost database.
Mason, R. (2006). Learning technologies for adult continuing education. Studies in
Continuing Education, 28(2), 121-133. Retrieved September 3, 2007 from the
EBSCOhost database.
Miller, M. (2005). Teaching and Learning in Affective Domain. In M. Orey (Ed.),
Emerging perspectives on learning, teaching, and technology. Retrieved August
25, 2007 from http://projects.coe.uga.edu/epltt/
Morrison, G. R., Ross, S. M., & Kemp, J. E. (2007). Designing effective instruction
(5th
ed.). Hoboken, NJ: John Wiley & Sons, Inc.
Underhill, A.F. (2006). Theories of learning and their implications for on-line
assessment. Turkish Online Journal of Distance Education-TOJDE, 7(1), 165-174.
Retrieved August 25, 2007 from the ERIC database.
Wolters, C.A. (2003). Regulation of motivation: Evaluating an underemphasized aspect
of self-regulated learning. Educational Psychologist, 38(4), 189–205.