novakovich march 20 2012
TRANSCRIPT
-
8/2/2019 Novakovich March 20 2012
1/52
Portfolio-Remix Performance Intervention
Online Publishing with Movable Type 5.10
Jeanette Novakovich
Professional Writing Minor Program
English Department
Concordia University
Montreal, Quebec
-
8/2/2019 Novakovich March 20 2012
2/52
PERFORMANCEINTERVENTION 2
2012Novakovich
2
-
8/2/2019 Novakovich March 20 2012
3/52
PERFORMANCEINTERVENTION 3
2012Novakovich
3
TABLE OF CONTENTS
Edition Notice 5
Twitter Microblogging Session for Portfolio-Remix 6Setup Instructions 9Microblogging Instructions 10Microblogging Instructions (student handout) 13Managing the post-activity debriefing 14
APPLICATION NAME - Step-by-Step Quick Guide for Publishing an Entry19
Step-by-Step Quick Guide for Publishing an Entry 21Finding Creative Commons Licensed Images 24Guidelines for Publishing an Entry 27Online Survey Checklist for Publishing an Entry 29Appendix 1: Evaluations 30
Sample Satisfaction Survey (Kirkpatrick Level 1) 31Sample Criterion-Referenced Assessment (Kirkpatrick Level 2) 32Sample of a Level Three Evaluation (Kirkpatrick Level 3) 33Sample of a Level Three Evaluation (Kirkpatrick Level 3) 35
Appendix Two: Formative Evaluations 38Technical review 39
Draft Email #1 40Draft email #2 41Draft email #3 43Draft #4 44
-
8/2/2019 Novakovich March 20 2012
4/52
-
8/2/2019 Novakovich March 20 2012
5/52
PERFORMANCEINTERVENTION 5
2012Novakovich
5
EDITION NOTICE
Crap Prevention: Troubleshooting Movable Type 5.10
Professional Writing Minor Program
English Department
Concordia University
Montreal, QC
First Version
Copyright 2012 Jeanette Novakovich. All rights reserved.
-
8/2/2019 Novakovich March 20 2012
6/52
PERFORMANCEINTERVENTION 6
2012Novakovich
6
TWITTER MICROBLOGGING SESSION FOR PORTFOLIO-REMIX
Overview of the activity
The purpose of this intervention, a weekly microblogging session, is to provide learners
with a support and recognition feedback system.
Background of the activity
More and more employers are asking applicants if they have a website or other web
presence that they can access easily. For employers who want to see beyond the
traditional one-page rsum, students can develop a rich alternative in the form of an e-
portfolio, which is an online digital form of the traditional paper-based portfolio. For this
reason, an e-portfolio hub, which is a website that aggregates or pools student portfoliostogether in one common location, was established for students enrolled in the program.
Essential to the success of the program is ensuring that students maintain high online
publishing standards as defined by the Professional Writing Minor Program when using
Movable Type 5.10 software, an online open source publishing Content Management
System, to design and publish professional e-portfolios.
The primary goal is to ensure that students comply with online publishing standards set by
the program by providing a support and recognition system in the form of a community
microblogging session.
Summary of the activity
Learners will read published entries on the blog feed of Portfolio Remix and provide
feedback and support in the form of Twitter tweets that include the following hashtag,
#portfolio-remix, a keyword that allows Twitter search engines to create a customized
search feed. Tweets will consist of constructive advice, alerts for publishing mishaps, and
congratulatory notes. Learners can then respond to the tweets as they are fed into a
Twitter search feed on the Portfolio-Remix online support website.
Main objectives
Select [Convert Line Breaks] format 100% of the time
-
8/2/2019 Novakovich March 20 2012
7/52
PERFORMANCEINTERVENTION 7
2012Novakovich
7
Place images appropriately on the page, centered, at the top of entry in [Body] fieldset at 500 pixel width in order to fit the container space or size of entry field
100% of the time and provide consistency of expression
Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the time Publish the bulk of entry in the [Extended] field 100% of the time
Format of the activity
During a thirty-minute microblogging group session, learners will tweet microblogs or
messages that consist of 140 characters or less that include the #portfolio-remix hashtag.
Expectations that learners have
Expect tweeters to be informal and polite Expect nonjudgmental alerts Expect a measure of recognition Expect a measure of fun
Prerequisites of the activity:When the activity takes place students should have basic
working knowledge of online of online publishing and social media, particularly,
microblogging. As experienced micro bloggers, learners will know how to get started and
what to do for the most part.
Time: The 30-minute formative evaluation activity is divided into two parts, including a
brief PowerPoint presentation and a hands-on learning activity, followed up by a debriefing.
Equipment and Services:
PC lab Projector Access to Internet
Material and Props:
Power Point embedded on course website
Printed instructors guide Printed student guide Online activity Online assessment
Schedule:
-
8/2/2019 Novakovich March 20 2012
8/52
PERFORMANCEINTERVENTION 8
2012Novakovich
8
5:45 Classroom setup
6:00 Introduction and explanation of activity
6:10 Formative evaluation activity
6:30 Bio break and snacks (informal Q&A)
6:45 Online satisfaction survey
-
8/2/2019 Novakovich March 20 2012
9/52
-
8/2/2019 Novakovich March 20 2012
10/52
PERFORMANCEINTERVENTION 10
2012Novakovich
10
MICROBLOGGING INSTRUCTIONS
1. Go to Portfolio-Remix main website: http://www.portfolio-remix.comAdministrivia:Common classroom activity. Problems are not anticipated.
2. Select the [Blogs] tab on the navigation bar.Administrivia:Common classroom activity. Problems are not anticipated.
3. Read through the entries presented in the blog feed.
Administrivia:Common classroom activity. Problems are not anticipated.
4. Respond to at least ten entries by tweeting information that evaluates theentry in terms of excellence and adhering to publishing standards.
See Tweet guidelines below.
Administrivia:Common activity. Problems are not anticipated. Every entry page will
have Share Us buttons, which include a Twitter icon. When students select the
Twitter icon, a popup box will open with a link to the entry and its title. Students
can then add a brief personal message. Remind them to include the #portfolio-
remix hashtag. This way the Tweet will enter the Twitter Feed embedded on thePortfolio-Remix online support help website.
5. After responding to an entry, read through the Twitter feed on the Portfolio-Remix online support website and respond to Twitter feedback:
http://www.portfolio-remix.com/twitter-feed.html .
Administrivia:Explain to the students that they will now read through the Twitter
Feed and respond to tweets of their work and their colleagues, following the
guidelines below.
Responding to a professional entry
If an entry is excellent, encourage your Twitter followers to read the entry:Tweet#portfolio-remix BRAVO! + add positive word or two (Twitter will add a
link to the entry and title to all tweets made from the entry page.)
-
8/2/2019 Novakovich March 20 2012
11/52
PERFORMANCEINTERVENTION 11
2012Novakovich
11
Administrivia: Common activity. Problems are not anticipated. If a student enjoys
reading an entry, Tweet it out positively. The student can then Tweet out any
format issues in subsequent Tweets. Students are not limited to one Tweet per
entry.
Responding to text formatting
If the author forgot to use [Convert Line Breaks] ):Tweet#portfolio-remix CLB!!! + add positive word or two
Administrivia: Help students identify entries that have a Word document pasted into
the [Rich Text Editor] format. This can be achieved by observing the blog feed on
the home page. Entries with [Rich Text] will show script or coding, irregular font
styles.
If the author has a weak introduction:Tweet--#portfolio-remix INTRO? + add positive word or two
Administrivia:Challenge students to be critical and supportive. Advise them in the
introductory paragraph does not compel them to read further, click the Continue
reading link, to help the author by letting them know in a friendly and supportive
manner.
If the entry does not have a Continue reading link:Tweet--#portfolio-remix Extended? + add positive word or two
Administrivia:Show students that they can identify that an [Extended] entry field is
empty if the blog feed is long and if the following link is not present: Continue
reading
Responding to images
If the author forgot to include an image:Tweet#portfolio-remix Image? + add positive word or two
Administrivia: No problems anticipated.
If the author failed to provide attribution for any images:Tweet#portfolio-remix Attribute? + add positive word or two
-
8/2/2019 Novakovich March 20 2012
12/52
PERFORMANCEINTERVENTION 12
2012Novakovich
12
Administrivia: No problems anticipated.
If the author has off-size or off-center image:Tweet#portfolio-remix Image center? + add positive word or two
Administrivia: No problems anticipated.
Responding to tweets
If a tweet is positive: [Reply] and tweet a warm thank you. If a tweet is constructive: [Reply] and tweet a warm thank you. If a tweet is positive and funny: Retweet.
Administrivia: No problems anticipated.
Demonstrating the activity
Demonstrate the activity by asking the students to identify an entry that they think is
polished and professional. Open the entry by clicking on the link in the blog feed to the
entrys page. Demonstrate the positive Bravo! tweet. Next, ask the students if they have
any questions. And finally, ask the students to continue the activity on their own and offer
to assist any students who are experiencing technical difficulties.
-
8/2/2019 Novakovich March 20 2012
13/52
PERFORMANCEINTERVENTION 13
2012Novakovich
13
MICROBLOGGING INSTRUCTIONS (STUDENT HANDOUT)
1. Go to Portfolio-Remix main website: http://www.portfolio-remix.com 2. Select [Blogs] tab on the navigation bar.3. Read through the entries presented in the blog feed.4. Respond to at least ten entries.5. After responding to an entry, read through the Twitter feed on the Portfolio-Remix
online support website and respond to Twitter feedback.
Responding to a professional entry
If an entry is excellent, encourage your Twitter followers to read the entry:Tweet#portfolio-remix BRAVO! + add positive word or two (Twitter will add a
link to the entry and title to all tweets made from the entry page.)
Responding to text formatting
If the author forgot to use [Convert Line Breaks] ):Tweet#portfolio-remix CLB!!! + add positive word or two
If the author has a weak introduction:Tweet--#portfolio-remix Intro? + add positive word or two
If the entry does not include a Continue reading link:Tweet--#portfolio-remix Extended? + add positive word or two
Responding to images If the author forgot to include an image:
Tweet#portfolio-remix Image? + add positive word or
If the author failed to provide attribution for any images:Tweet#portfolio-remix Attribute? + add positive word or
If the author has off-size or off-center image:Tweet#portfolio-remix Image Center? + add positive word or two
Responding to tweets
If a tweet is positive: [Reply] and tweet a warm thank you. If a tweet is constructive: [Reply] and tweet a warm thank you. If a tweet is positive and funny: Retweet.
-
8/2/2019 Novakovich March 20 2012
14/52
PERFORMANCEINTERVENTION 14
2012Novakovich
14
MANAGING THE POST-ACTIVITY DEBRIEFING
1. Review the results with the learners:
2. Ask learners about what they didwhat were the results, what they enjoyed aboutthe activity, what they disliked about the activity, and where they went right and
-
8/2/2019 Novakovich March 20 2012
15/52
PERFORMANCEINTERVENTION 15
2012Novakovich
15
where they went wrong.
-
8/2/2019 Novakovich March 20 2012
16/52
PERFORMANCEINTERVENTION 16
2012Novakovich
16
Screen 1 of 5
APPLICATION NAME Twitter Microblogging Feed
File OPTION OPTION OPTION OPTION Window Help
Option Option Option Option Option Option Option Option
Netsite
Objective(s) to be covered in this topic: This twitter feed will provide support
and feedback for online publishing.
Performance objectives that will be addressed:
Select [Convert Line Breaks] format 100% of the time Place images appropriately on the page, centered, at the top of entry in
[Body] field set at 500 pixel width in order to fit the container space or
size of entry field 100% of the time and provide consistency of
expression
Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the
time
Publish the bulk of entry in the [Extended] field 100% of the timeObjectives will be presented in a Twitter feed during microblogging sessions. In
addition, students can seek support anytime by tweeting for help and using the
#portfolio-remix hashtag.
STATUS BAR
STATUS BAR
Production Instructions: Choose the button on the toolbar and paste the followingHTML code onto a new page of the online help support website, tag @topnav, and publish
the page.
-
8/2/2019 Novakovich March 20 2012
17/52
PERFORMANCEINTERVENTION 17
2012Novakovich
17
Twitter feed code for #Portfolio-Remix
new TWTR.Widget({
version: 2,
type: 'search',
search: '#portfolio-remix OR portfolio-remix',
interval: 30000,
title: 'Portfolio-Remix',
subject: ' ',
width: 500,
height: 800,theme: {
shell: {
background: '#f9fae8',
color: '#a1092f'
},
tweets: {
background: '#ffffff',
color: '#444444',
links: '#7d0628'
}},
features: {
scrollbar: true,
loop: true,
live: true,
behavior: 'default'
}
}).render().start();
-
8/2/2019 Novakovich March 20 2012
18/52
PERFORMANCEINTERVENTION 18
2012Novakovich
18
The published page will have the following appearance:
-
8/2/2019 Novakovich March 20 2012
19/52
PERFORMANCEINTERVENTION 19
2012Novakovich
19
Screen __2_ of _5___
APPLICATION NAME - STEP-BY-STEP QUICK GUIDE FOR
PUBLISHING AN ENTRY
File OPTION OPTION OPTION OPTION Window Help
Option Option Option Option Option Option Option Option
Netsite
Performance objective(s) (competencies) it addresses
Select [Convert Line Breaks] format 100% of the time Include an appealing Creative Commons licensed image 100% of the
time
Place images appropriately on the page, centered, at the top of entry in[Body] field set at 500 pixel width in order to fit the container space or
size of entry field 100% of the time and provide consistency of
expression
Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the
time
Publish the bulk of entry in the [Extended] field 100% of the time Provide search engine optimization 100% of the time
The objectives will be presented in the form of a quick procedural step-by-step
guide to online publishing.
STATUS BAR
STATUS BAR
-
8/2/2019 Novakovich March 20 2012
20/52
PERFORMANCEINTERVENTION 20
2012Novakovich
20
Production Instructions: Screenshots of steps will be provided.
-
8/2/2019 Novakovich March 20 2012
21/52
PERFORMANCEINTERVENTION 21
2012Novakovich
21
STEP-BY-STEP QUICK GUIDE FOR PUBLISHING AN ENTRY
The following instruction set provides you with the steps you need to in order to publish
an entry according to the guidelines established by the Professional Writing Minor Program
at Concordia University. Before beginning this procedure, you should have an entry
composed in a Word document, and you should have an image downloaded into your
personal computer for uploading a Creative Commons licensed image with a pixel
resolution of 500 pixel width or greater. The overall procedure should take no more than
10 to 15 minutes.
1. Creating a new entry1.1.Log into dashboard.1.2.In the sidebar, select [Entries] > [New]1.3.In the top entry field, type your title.1.4.Select a Format: [Convert Line Breaks].
2. Uploading an image2.1.Select Image Icon from the toolbar.2.2.On the popup screen, select [Upload a new image from computer].2.3.Select the image you prepared for your entry.2.4.On the next popup screen, set pixel width to 500 pixels.2.5.Select [Center].2.6.Select [Finish].2.7.Provide attribution for your entry by linking caption to source.
3. Adding the text3.1.Copy and paste the first paragraph of your entry.3.2.Select [Extended] entry field.3.3.Copy and paste the remainder of the entry.3.4.Switch format back to [Rich Text Editor].
4. Optimizing SEO4.1.Select from the available links offered by Zemanta.4.2.Select from the available tags offered by Zemanta.
5. Publishing an entry5.1.Select [Preview].
-
8/2/2019 Novakovich March 20 2012
22/52
PERFORMANCEINTERVENTION 22
2012Novakovich
22
5.2.Select [Publish entry].6. Socially bookmarking your entry
6.1.Select [View Entry].6.2.Select a [Twitter icon] or [Facebook icon].6.3.Tweet (add #portfolio-remix and/or post to Facebook.
-
8/2/2019 Novakovich March 20 2012
23/52
PERFORMANCEINTERVENTION 23
2012Novakovich
23
Screen __3_ of __5__
APPLICATION NAME - Finding Creative Commons Licensed Images
File OPTION OPTION OPTION OPTION Window Help
Option Option Option Option Option Option Option Option
Netsite
Performance Objective:
Include an appealing Creative Commons licensed image 100% of the time
How the objective(s) will be presented (either a sketch or a narrative)
The objective will be presented in a descriptive bullet list of resources. As well,
the Creative Commons license will be defined and illustrated.
STATUS BAR
STATUS BAR
Production Instructions: describe any graphics, photos, and audio, animation, and video
sequences needed on this display:
Screenshots of the Creative Commons license will be provided and a Creative Commons
licensed image will provide a visual argument.
-
8/2/2019 Novakovich March 20 2012
24/52
PERFORMANCEINTERVENTION 24
2012Novakovich
24
FINDING CREATIVE COMMONS LICENSED IMAGES
A Creative Commons license allows the maker of an image to retain the copyright of their
work and, at the same time, gives permission for others to reuse the work. The license
provides the original artist a number of conditions to select from that the user must abide:
"You let others copy, distribute, display, and perform your copyrighted
work and derivative works based upon it but only if they give credit the
way you request (CC Org).
"You allow others to distribute derivative works only under a licenseidentical to the license that governs your work (CC Org).
"You let others copy, distribute, display, and perform your work and
derivative works based upon it but for noncommercial purposes only (CC
Org).
"You let others copy, distribute, display, and perform only verbatim
copies of your work, not derivative works based upon it (CC Org).
The following websites provide a number of Creative Commons licensed images or
search engines to help you locate images that are available for reuse:Google Image advanced search- This search engine allows you to search for images
that are licensed for reuse.
Image Links from Creative Commons This website provides numerous links to the
Creative Commons licensed databases of images.
Flickr Advanced Search This photo sharing website allows you to select a searchfor Creative Commons licensed images and videos.
Wikimedia Commons - This website provides a number of public domain images.
-
8/2/2019 Novakovich March 20 2012
25/52
PERFORMANCEINTERVENTION 25
2012Novakovich
25
NASA Image Archive - NASA has a number of non-copyrighted government images
related to space flight, astronomy, and aeronautics.
US Government This link will lead you to public domain photos and images
created by the United States government.
Open Clip Art- This website provides an archive of free-to-use clipart.
When you locate an image that you would like to use on your portfolio, check the license
to find out how to attribute the image and how you are allowed to use the image,
whether it can be a derivative work or a verbatim copy of the work.
-
8/2/2019 Novakovich March 20 2012
26/52
PERFORMANCEINTERVENTION 26
2012Novakovich
26
Screen __4_ of __5__
APPLICATION NAME - Guidelines for Publishing an Entry
File OPTION OPTION OPTION OPTION Window Help
Option Option Option Option Option Option Option Option
Netsite
Objective(s) to be covered in this topic: Select [Convert Line Breaks] format 100% of the time Include an appealing Creative Commons licensed image 100% of the
time
Place images appropriately on the page, centered, at the top of entry in[Body] field set at 500 pixel width in order to fit the container space or
size of entry field 100% of the time and provide consistency of
expression
Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the
time
Publish the bulk of entry in the [Extended] field 100% of the time Provide search engine optimization 100% of the time
How the objective(s) will be presented (either a sketch or a narrative)
Guidelines will be presented in a descriptive bullet list.
STATUS BAR
STATUS BAR
Production Instructions: None at this time. Suggestions?
-
8/2/2019 Novakovich March 20 2012
27/52
PERFORMANCEINTERVENTION 27
2012Novakovich
27
GUIDELINES FOR PUBLISHING AN ENTRY
Maintaining a high level of professionalism is key to the success of the Professional WritingMinor Programs implementation of our electronic portfolio website hub. To ensure that you
are maintaining consistent and professional publishing standards, the following guidelines
are provided:
Use [Convert Line Breaks] when pasting any text from a Word document. Include an image at the head of every entry to contribute to an attractive entry
feed to the main portfolio website. The image should have high pixel resolution (at
least 500 pixel width); and when you upload the image, it should be centered and
the width should be set to 500-pixel width to prevent gutter bleed.
Only one paragraph of text should be pasted into the [Body] of the entry. Thisintroductory paragraph should be polished and appealing.
The rest of your entry should be pasted into the [Extended] entry field. Here, youcan also add an assortment of images and videos.
Avoid adding videos to the [Body] entry field; videos will slow down the blog feedsignificantly.
After entering the text, use [Rich Text Editor] to activate the Zemanta plugin. Thiswill give you an opportunity to create links to outside sources and add tags to your
entry, thereby, creating search engine optimization.
Once you have published your entry, view it and socially bookmark the entry toincrease web traffic.
If you follow these guidelines, both your portfolio and the main hub will present a
professional and consistent image, which will greatly enhance the credibility of both thewriter and the program. Thank you very much for adhering to the guidelines.
-
8/2/2019 Novakovich March 20 2012
28/52
PERFORMANCEINTERVENTION 28
2012Novakovich
28
Screen _5__ of ___5_
APPLICATION NAME - Online Survey Checklist for Publishing an Entry
File OPTION OPTION OPTION OPTION Window Help
Option Option Option Option Option Option Option Option
Netsite
Objective(s) to be covered in this topic:
Select [Convert Line Breaks] format 100% of the time Include an appealing Creative Commons licensed image 100% of the
time
Place images appropriately on the page, centered, at the top of entry in[Body] field set at 500 pixel width in order to fit the container space or
size of entry field 100% of the time and provide consistency of
expression
Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the
time
Publish the bulk of entry in the [Extended] field 100% of the time Provide search engine optimization 100% of the time
How the objective(s) will be presented (either a sketch or a narrative)
This intervention will take the form of an embedded online survey.
STATUS BAR
STATUS BAR
Production Instructions: Poll Daddy survey maker.
-
8/2/2019 Novakovich March 20 2012
29/52
PERFORMANCEINTERVENTION 29
2012Novakovich
29
ONLINE SURVEY CHECKLIST FOR PUBLISHING AN ENTRY
1. Did you select [Convert Line Breaks] Yes or No2. Did you add an image? Yes or No3. Was your image centered and set to 500-pixel width? Yes or No4. Did you provide attribution? Yes or No5. Did you paste the first paragraph of the entry into the [Body] field? Yes or No6. Did you paste the remainder of the entry into the [Extended] entry field? Yes or No7. Did you switch back to [Rich Text Editor]? Yes or No8. Did you select [Links] and [Tags] to optimize SEO? Yes or No9. Did you socially bookmark your published entry? Yes or No
-
8/2/2019 Novakovich March 20 2012
30/52
PERFORMANCEINTERVENTION 30
2012Novakovich
30
APPENDIX A: EVALUATIONS
-
8/2/2019 Novakovich March 20 2012
31/52
PERFORMANCEINTERVENTION 31
2012Novakovich
31
SAMPLE SATISFACTION SURVEY (KIRKPATRICK LEVEL 1)
(Adapted from Carliner, 2003)
Please provide feedback to the following questions:
1. How would you rate this activity on a scale of 1 - 5 with 5 being the highest rating?1 2 3 4 5
2. How much did you know about online publishing software before doing this trainingcourse?
Nothing Very little Little Some Somewhat
3. How much do you know after completing the program?Nothing Very little Little Some Somewhat
4. How likely are you to use the skills taught in this program on your own outside ofclass without supervision?
Nothing Very little Little Some Somewhat
5. What was the most valuable part of this program in terms of skill sets taught?_____________________________________________________________________
6. What was the most effective part of this program?_____________________________________________________________________
7. What is one aspect of the training that could improve this program most?_____________________________________________________________________
-
8/2/2019 Novakovich March 20 2012
32/52
PERFORMANCEINTERVENTION 32
2012Novakovich
32
SAMPLE CRITERION-REFERENCED ASSESSMENT (KIRKPATRICK LEVEL 2)
Performers will be evaluated through observation. The course instructor of English 396 willask a representative sample of students that include novice, average and expert performers
to create a new entry. The instructor will record observations in a chart.
Main Task: Within a span of 30 minutes, create a new entry for your portfolio. Observer
will circle if the behavior is present.
Composes on word (not necessary but common) Present
Opens dashboard Present
Creates new entry Present
Composes a title for entry Present
Selects [convert line breaks] format Present
Finds a creative commons image Present
Uploads image first, 500 px, centered [body] Present
Provides attribution Present
Locates 1st paragraph of text in [body] field Present
Locates remainder of text in [extended] field Present
Selects suggested links Present
Selects suggested tags Present
Saves entry PresentRefreshes website Present
Score /130
-
8/2/2019 Novakovich March 20 2012
33/52
PERFORMANCEINTERVENTION 33
2012Novakovich
33
SAMPLE OF A LEVEL THREE EVALUATION (KIRKPATRICK LEVEL 3)
Twelve weeks after training, in order to evaluate whether performers are publishing entries
according to the programs standards, 10 direct observations of the most recent web
entries will be evaluated. Google analytics will also be used to gather web metric data on
published entries.
Entry
#1
Entry
#2
Entry
#3
Entry
#4
Entry
#5
Entry
#6
Entry
#7
Entr
y
#8
Entr
y
#9
Entr
y
#10
Selected
[convert line
breaks] format
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
NoUsed a
creative
commons
image
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Image 500 px,
centered,
[body] field
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Provided
attribution
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
NoIncluded no
more than 1
paragraph in
[body] field
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Located bulk
of entry in
[extended]
field
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Provided links Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Provided tags-
search terms
(found in tag
cloud)
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
-
8/2/2019 Novakovich March 20 2012
34/52
PERFORMANCEINTERVENTION 34
2012Novakovich
34
Score
Yes = 10 pts
No = 0 pts
/90
Google analytics will also be used to collect web metrics, which measure the success of
individual web entries, in terms of readership, to identify entries that are tagged and linked
effectively.
Top ten entries over the last month
Title of entry Number of
visitors
Engagement (average # of
seconds spent on page or entry)
#1#2
#3
#4
#5
#6
#7
#8
#9
#10
-
8/2/2019 Novakovich March 20 2012
35/52
PERFORMANCEINTERVENTION 35
2012Novakovich
35
SAMPLE OF A LEVEL THREE EVALUATION (KIRKPATRICK LEVEL 3)
After a twenty-four week period, a brief survey will be given to students to check for their
understanding of the publishing procedure six months done the line. After this, a repeat of
observations can be taken of the most recent published entries. Finally, Google Analytics
will be used to gather web-publishing metrics.
Student survey of students understanding of the publishing procedure:
Provide an effective title Circle Yes or No
Select [convert line breaks] format Circle Yes or No
Use creative commons licensed image Circle Yes or No
Size and center images at 500 px Circle Yes or No
Provide attribution or cite your sources Circle Yes or No
Locate 1st paragraph of text in [body] field Circle Yes or No
Locate remainder of text in [extended] field Circle Yes or No
Select suggested links Circle Yes or No
Select suggested tags Circle Yes or No
Score /90
-
8/2/2019 Novakovich March 20 2012
36/52
PERFORMANCEINTERVENTION 36
2012Novakovich
36
In order to evaluate whether performers are publishing entries according to the programs
standards, ten direct observations of the most recent web entries will be evaluated.
Entry
#1
Entry
#2
Entry
#3
Entry
#4
Entry
#5
Entry
#6
Entry
#7
Entr
y
#8
Entr
y
#9
Entr
y
#10
Selected
[convert line
breaks] format
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Used a
creative
commons
image
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Image 500 px,
centered,
[body] field
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Provided
attribution
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Included no
more than 1
paragraph in
[body] field
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Located bulk
of entry in
[extended]
field
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Provided links Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Provided tags-
search terms
(found in tag
cloud)
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Yes
or
No
Score
Yes = 10 pts
No = 0 pts
/90
-
8/2/2019 Novakovich March 20 2012
37/52
PERFORMANCEINTERVENTION 37
2012Novakovich
37
Google analytics will also be used to collect web metrics, which measure the success of
individual web entries, in terms of readership, to identify entries that are tagged and linked
effectively.
Top ten entries over the last month
Title of entry Number of
visitors
Engagement (average # of
seconds spent on page or entry)
#1
#2
#3
#4
#5#6
#7
#8
#9
#10
References
Carliner S. (2003) Training Design Basics. Baltimore, MD: ASTD press.
Gilbert Model (1996) adapted by Stolovitch, Harold D. & Keeps, Erica J. (2004) Training
AintPerformance. Baltimore, Md: ASTD press.
-
8/2/2019 Novakovich March 20 2012
38/52
PERFORMANCEINTERVENTION 38
2012Novakovich
38
APPENDIX B: FORMATIVE EVALUATIONS
A formative evaluation plan for Performance Intervention: Online Publishing with Movable
Type 5.10 will take place in a series of stages to determine the effectiveness of the
course in terms of its being accurate, understandable, and functional as described by
Carliner (2003): Technical review Pilot test Editorial review
-
8/2/2019 Novakovich March 20 2012
39/52
PERFORMANCEINTERVENTION 39
2012Novakovich
39
TECHNICAL REVIEW
Technical Review Participants:
EL Instructional design expert with a specialization implementing Movable Type
MK Movable Type expert SK Educational technology expert at the university level
Procedure:
Date Action Materials
First Contact September 17, 2012 Email
Affirm participation September 24, 2012 Email
Reminder November 17, 2012 Email
Delivery November 28, 2012 Email withattachment
Draft 1:Instructional
materials, learners
handout and
storyboards for
web entries
Return date November 29, 2012 Email with
attachment
-
8/2/2019 Novakovich March 20 2012
40/52
PERFORMANCEINTERVENTION 40
2012Novakovich
40
DRAFT EMAIL #1
To: EL, MK, or BK
From: Jeanette NovakovichDate: 17 September 2012
Subject: Portfolio-Remix Performance Intervention
Hello,
[Congratulations on your recent nuptials. I hope that you had an enjoyable
honeymoon. I would like to thank you again for the work that you managed to
complete for me during your exciting summer.] (Vary with recipient)
This winter, I am completing a performance intervention for students using Movable
Type software. I am writing to request a review of the performance intervention
materials for the program. Coming from one of the few Movable Type experts, your
feedback will provide an invaluable insight into the accuracy of the project.
The review will take two forms:
An evaluation of the effectiveness and accuracy of the technical content A walk-through of the training material
The review materials will be sent out electronically in eight weeks, during the first
week of November 2012. The review shouldnt take more than a few hours to
complete. The scheduled review will take place between the following dates:
November 28- Reviewer receives review materials electronically
November 29- Reviewer returns reviews electronically
Thank you for your consideration.
Kindest regards,
Jeanette Novakovich
Assistant Professor, English Department
Concordia University, Montreal QC
-
8/2/2019 Novakovich March 20 2012
41/52
PERFORMANCEINTERVENTION 41
2012Novakovich
41
DRAFT EMAIL #2
To: EL, MK, BKFrom: Jeanette Novakovich
Date: 24 September 2012
Subject: Performance Intervention for Online Publishing with Movable Type 5.10
Hello,
Thank you for agreeing to review the performance intervention materials that I am
developing for the Professional Writing Minor Program at Concordia University. As
one of the few Movable Type experts, your willingness to provide detailed feedbackis truly appreciated.
The review will take the form of an evaluation of the effectiveness of the content
in terms of its accuracy, understandability, and the functionality of the materials. A
feedback form will be provided.
The review materials will be sent out electronically in six weeks, during the last
week of November 2012. The review shouldnt take more than a few hours to
complete.
Please review the schedule:
November 28- Reviewer receives review materials electronically
November 29- Reviewer returns feedback electronically
Please find attached an I-Tunes gift certificate as a gesture of goodwill and
appreciation for your anticipated timely feedback.
Kind regards,
Jeanette Novakovich
Assistant Professor, English Department
Concordia University, Montreal QC
-
8/2/2019 Novakovich March 20 2012
42/52
PERFORMANCEINTERVENTION 42
2012Novakovich
42
-
8/2/2019 Novakovich March 20 2012
43/52
PERFORMANCEINTERVENTION 43
2012Novakovich
43
DRAFT EMAIL #3
To: EL, MK, BK
From: Jeanette Novakovich
Date: 17 November 2012
Subject: Portfolio-Remix Performance Intervention
Hello,
Thank you for agreeing to review the performance intervention materials for our
program here at Concordia.
The review materials will be sent out electronically in five days, during the first
week of November 2012:
November 4- Reviewer receives review materials electronically
November 6- Reviewer returns feedback electronically
I look forward to receiving your feedback.
Kindest regards,
Jeanette Novakovich
Assistant Professor, English Department
Concordia University, Montreal QC
-
8/2/2019 Novakovich March 20 2012
44/52
PERFORMANCEINTERVENTION 44
2012Novakovich
44
DRAFT #4
To: EL, MK, BK
From: Jeanette NovakovichDate: 28 November 2012
Subject: Portfolio-Remix Performance Intervention
Hello,
Thank you for agreeing to review the performance intervention materials for our
online publishing program.
Please find the following items attached for the technical review: Instructors Manual Learners Handout Storyboards for web entries
The overall objective of this review is to note any inaccuracies of content in terms
of correctness, currency and completeness. In addition, the material can be reviews
for accessibility issues for learners. The learners can be assumed to have no more
than basic training using Movable Type software. And finally, please note any
aspects of the materials that are not currently functional.
Please feel free to make comments and corrections on the electronic drafts and
be sure to complete the feedback form. I look forward to receiving your feedback
on Tuesday, November 29th.
Kindest regards,
Jeanette Novakovich
Assistant Professor, English DepartmentConcordia University, Montreal QC
-
8/2/2019 Novakovich March 20 2012
45/52
PERFORMANCEINTERVENTION 45
2012Novakovich
45
BLANK FEEDBACK FORM
In addition to making comments directly on course material, please complete the following
feedback form.
Storyboards for
web entries, note
slide #.
Instructors manual
When providing
feedback, note
section and page
#.
Learners Material
When providing
feedback, note
section and page
#.
Information
complete? If not,
note missing
information.
Is the informationAccurate? Note
inaccuracies.
Is information
current? Note
missing updates.
Is Information
Understandable to
a lay audience?
Note any issues
or confusion.
Is information
functional? Note
any places where
the material did
not link or
operate according
to basic web
standards.
Any additional
comments.
-
8/2/2019 Novakovich March 20 2012
46/52
PERFORMANCEINTERVENTION 46
2012Novakovich
46
HYPOTHETICAL PILOT TEST
To: English 395 Technical Writing Students
From: Jeanette NovakovichDate: 1 December 2012
Subject: Pilot test for Portfolio-Remix Performance Intervention
Hello,
As mentioned in class earlier today, you are invited to pilot a test of Portfolio-
Remix Performance Intervention Tuesday December 6, 2012. The test will take place
at the start of class in Hall Room 923. Students who participate are eligible to
take part in an I-Tunes gift card drawing at the close of the pilot test.I look forward to receiving your invaluable feedback. See you in class next week!
Kindest regards,
Jeanette Novakovich
Assistant Professor, English Department
Concordia University, Montreal QC
-
8/2/2019 Novakovich March 20 2012
47/52
PERFORMANCEINTERVENTION 47
2012Novakovich
47
REVISION OF INTERVENTIONS BASED ON PILOT TEST
Category A: Showstopper
Category B: Must change
Category C: Nice to change
-
8/2/2019 Novakovich March 20 2012
48/52
PERFORMANCEINTERVENTION 48
2012Novakovich
48
EDITORIAL REVIEW
DRAFT #1
To:
From: Jeanette Novakovich
Date: 1 October 2012
Subject: Editorial review for Portfolio-Remix Performance Intervention
Hello,
Congratulations on your recent and well-deserved promotion. As we discussed lastweek, I am writing to formally request an editorial review of the training materials
for Portfolio-Remix Performance Intervention. I appreciate your having expressed
your willingness to participate in the final review process.
The review materials and style sheet will be sent out electronically in ten weeks,
during the second week of December 2012. The review shouldnt take more than a
few hours to complete and will take place between the following dates:
December 12- Reviewer receives review materials electronicallyDecember 13- Reviewer returns reviews electronically
I look forward to receiving your keen feedback. Please identify any errors in
grammar, style, spelling and layout. As we agreed, you will be sending me the
materials to review your latest in-the-works project.
Thank you for your consideration.
Kindest regards,Jeanette Novakovich
Assistant Professor, English Department
Concordia University, Montreal QC
-
8/2/2019 Novakovich March 20 2012
49/52
PERFORMANCEINTERVENTION 49
2012Novakovich
49
DRAFT #2
To:
From: Jeanette NovakovichDate: 7 December 2012
Subject: Portfolio-Remix Performance Intervention
Hello,
Thank you for agreeing to review the training materials for Portfolio-Remix
Performance Intervention.
The review materials will be sent out electronically in five days, during the secondweek of December:
December 12- Reviewer receives review materials electronically
December 13- Reviewer returns feedback electronically
I look forward to receiving your feedback.
Kindest regards,
Jeanette NovakovichAssistant Professor, English Department
Concordia University, Montreal QC
-
8/2/2019 Novakovich March 20 2012
50/52
PERFORMANCEINTERVENTION 50
2012Novakovich
50
DRAFT #3
To:
From: Jeanette NovakovichDate: 12 December 2012
Subject: Portfolio-Remix Performance Intervention
Hello,
Thank you for agreeing to review the training materials for Portfolio-Remix
Performance Intervention. Please find the following items attached for the editorial
review: Instructors manual Learners handout Storyboard for web entries Stylesheet
The overall objective of this review is to note any grammar, style, spelling or layout
errors. Please feel free to make comments and corrections on the electronic drafts.
I look forward to receiving your feedback on Tuesday, December 13 th.
Kindest regards,
Jeanette Novakovich
Assistant Professor, English Department
Concordia University, Montreal QC
-
8/2/2019 Novakovich March 20 2012
51/52
PERFORMANCEINTERVENTION 51
2012Novakovich
51
PERFORMANCE INTERVENTION STYLESHEET
Canadian English
Style Guides:
The Canadian Style: A Guide to Writing and Editing (1997). Dundurn Press,Public Works, Government Services Canada.
Canadian Oxford Dictionary. Online reference:http://www.oupcanada.com/reference_trade/canadian_oxford_dictionaries/dicti
onary_online.html
Preferred Web Entry Layout:
Introductory and close-out slides standard PPT heading and bullet listsummary.
Content or unit slides: Assertion-evidence style that includes an assertion inthe header followed by visual evidence. Furthermore, each unit includes an
introduction to the troubleshooting area, provides a tip to solve or prevent
the problem, and concludes by explaining the best practice.
Clean crisp slide layout. Warm beige background, crisp modern black font, and white background for
text.
Delicious font for heading: size 28pt Delicious font for text: size 16.5pt Red spot color in visuals
Course Materials:
Euphemia UCAS font for headings: size 14 pt + Euphemia UCAS font for text: size 12 pt Maroon red spot color Preferred active voice Technical jargon defined in text boxes
-
8/2/2019 Novakovich March 20 2012
52/52
PERFORMANCEINTERVENTION 52
REFERENCES
The Canadian Style: A Guide to Writing and Editing (1997). Dundurn Press,Public
Works, Government Services Canada.
Canadian Oxford Dictionary. Online reference:
http://www.oupcanada.com/reference_trade/canadian_oxford_dictionaries/dictionary_
online.html
Carliner S. (2003). Training Design Basics, Baltimore, MD: ASTD Press.
Note: Some text adapted from Carliner, 2003.