1 system for administration, training, and educational resources for nasa training evaluations
TRANSCRIPT
![Page 1: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/1.jpg)
1
System for Administration, Training, and Educational Resources for NASA
Training Evaluations
![Page 2: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/2.jpg)
2
Agenda
Lesson 1: Overview for Training Evaluation Lesson 2: Research Item Evaluation
Surveys Lesson 3: Gather and Analyze
Survey Results Lesson 4: Evaluate Learning Using Pre-
and Post-Test
![Page 3: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/3.jpg)
3
Lesson 1Overview for Training Evaluations
![Page 4: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/4.jpg)
4
Lesson Objectives
Identify all levels of evaluation and explain which evaluation tools you will use in SATERN.
Understand evaluation terms and definitions Explain how evaluations are implemented at
NASA Identify SATERN configurations needed to
support evaluations
![Page 5: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/5.jpg)
5
Evaluating Training Results Using SATERN
Levels of Evaluation
![Page 6: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/6.jpg)
6
How Evaluations are Implemented at NASA
A questionnaire survey will automatically be assigned to leaners after successful completion of some of the instructor-led course or an online module.
SATERN administrators can use their own discretion and decide whether to use Level 2 evaluations pre- and post-tests.
NASA will continue to use Metrics that Matter surveys. Administrators will not create or track follow-up evaluations in SATERN.
![Page 7: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/7.jpg)
7
Configuration to Support Evaluations Completion Status Triggers Survey Assignment
Rating Scale Used for Surveys
StronglyDisagree
Disagree Neither Agreenor Disagree
Agree Strongly Agree
1 2 3 4 5
![Page 8: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/8.jpg)
8
Lesson Wrap-Up Identified all levels of evaluation and explained
which evaluation tools you will use in SATERN. Understand evaluation terms and definitions Explained how evaluations are implemented at
NASA Identified SATERN configurations needed to
support evaluations
![Page 9: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/9.jpg)
9
Lesson Check
![Page 10: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/10.jpg)
10
Lesson 2Research Item Evaluation Surveys
![Page 11: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/11.jpg)
11
Lesson Objectives Identify the purpose of an item evaluation Identify possible question types for a item
evaluation survey Research and preview item evaluation surveys
used in SATERN Determine configuration options for item
evaluation surveys Associate an item evaluation survey with an item
![Page 12: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/12.jpg)
12
Item Evaluations Collect Data on Learner Satisfaction Assesses a learner’s reaction to a training event Most effective if delivered immediately after the
training event ends Sometimes referred to as a ‘smile sheet’ Measures how well learners liked the training
![Page 13: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/13.jpg)
13
Question Types for Item Evaluations
Rating Scale
One Choice
Multiple Choice
Open Ended
![Page 14: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/14.jpg)
14
Lab 1
Research and Preview a Published
SATERN Survey
![Page 15: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/15.jpg)
15
Configuration Options for Level 1 Surveys
![Page 16: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/16.jpg)
16
How to Associate a Survey with an Item
There are two ways to make an association between an item and a survey:• Item Usage tab of
questionnaire survey
• Evaluations tab of item record
![Page 17: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/17.jpg)
17
Lab 2
Method 1: Associate an Item
with a Survey
![Page 18: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/18.jpg)
18
Lab 3
Method 2: Associate a
Survey within an Item
![Page 19: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/19.jpg)
19
Lesson Wrap-Up
Identified the purpose of an item evaluation Identified possible question types for a item
evaluation survey Researched and previewed item evaluation
surveys used in SATERN Determined configuration options for item
evaluation surveys Associated an item evaluation survey with an
item
![Page 20: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/20.jpg)
20
Lesson Check
![Page 21: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/21.jpg)
21
Lesson 3Gather and Analyze Survey Results
![Page 22: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/22.jpg)
22
Lesson Objectives Launch and complete assigned surveys from the
learner side of SATERN Check learners’ progress from a survey record Run reports from the survey record Run item evaluation reports from the reports tab Remove item evaluations from learner records
![Page 23: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/23.jpg)
23
How Learners Complete Surveys
An alert displays on their SATERN home page
A learner is also sent a notification If an answer is missed, a warning message displays to inform the learner to complete the question Learners can save the survey and return later to continue editing
![Page 24: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/24.jpg)
24
Lab 4
Launch and Complete an
Assigned Survey
![Page 25: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/25.jpg)
25
Check Progress from the Survey Record
![Page 26: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/26.jpg)
26
Reports Related to Evaluations
Item Evaluation Report Item Evaluation by Individual Response Report Item Evaluation by Instructor Report
![Page 27: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/27.jpg)
27
Lab 5
Run an Item Evaluation Report
![Page 28: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/28.jpg)
28
Removing Outdated Surveys
The best way to capture accurate results is to have learners complete the survey soon after training. If a considerable amount of time has passed without survey completion, you may want to remove the survey from the learner record.
![Page 29: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/29.jpg)
29
Lab 6
Method 1: Remove Surveys via
Learner Needs Management
![Page 30: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/30.jpg)
30
Lab 7
Method 2: Remove Surveys Directly from the Learner
Record
![Page 31: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/31.jpg)
31
Lesson Wrap-Up
Launched and completed assigned surveys from the learner side of SATERN
Checked learners’ progress from a survey record Ran reports from the survey record Ran item evaluation reports from the Reports tab Removed item evaluations from a learner record
![Page 32: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/32.jpg)
32
Lesson Check
![Page 33: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/33.jpg)
33
Lesson 4Evaluate Learning Using Pre- and Post-Tests
![Page 34: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/34.jpg)
34
Lesson Objectives Explain the purpose of Level 2 evaluations Associate a pre-test and post-test as Level 2
evaluations
![Page 35: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/35.jpg)
35
Level 2 Learning Evaluations
Purpose of a pre- and post-exam is to assess the learner population’s knowledge of the content prior to the training, and then just after the training.
If the training program is effective, there will be an increase in knowledge when the two exam results are compared
![Page 36: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/36.jpg)
36
Lab 8
Associate Pre- and Post-Exams with
an Item
![Page 37: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/37.jpg)
37
Lesson Wrap-Up
Explained the purpose of Level 2 evaluations Associated a pre-test and post-test as Level 2
evaluations
![Page 38: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/38.jpg)
38
Lesson Check
![Page 39: 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f075503460f94c1d023/html5/thumbnails/39.jpg)
39
Course Summary Identified all levels of evaluation and explained
which evaluation tools are used in SATERN Understand evaluation terms and definitions Explained how evaluations are implemented at
NASA Identified SATERN configurations needed to
support evaluations