Digital Exams in Blackboard — The Test Manager Manual Blackboard 9.1 – Faculty of Humanities

Blackboard Helpdesk Faculty of Humanities

Authors: Karen Vegt, Lucia van den Brink, Susanne Keesman, Maud Westendorp – Teaching-assistants Blackboard Helpdesk & Maarten Hijzelendoorn (red.)

Do you have any further questions after reading this manual? Please do not hesitate to

contact the Blackboard Helpdesk.

Address

Faculty of Humanities/IFZ

Lipsius Building, Room 135

Cleveringaplaats 1

Phone: 071-527 2773

Email: Blackboard@hum.leidenuniv.nl

Website: FGW > Medewerkers >

Informatisering en Facilitaire Zaken >

Blackboard

WARNING

Deploying summative, i.e. graded Tests and Assessments, requires a stable and safe

environment. The (implementation of the) current version of Blackboard cannot

guarantee such an environment (yet). Suggestions in this manual for the best Test

options are still under construction. We recommend to use Blackboard for formative, i.e.

diagnostic Tests and Assessments only. We advise you to ignore remarks on summative

Tests and Assessments for the moment.

2

Contents

1. Introduction ............................................................................................................................3 1.1 Structure of this Manual .............................................................................................4

2. Important: Checklist before Creating a Test ...........................................................................5 2.1 Introduction and Goals of the Test ...................................................................................5 2.2 Test Design .......................................................................................................................5 2.3 Questions and Grading .....................................................................................................6 2.4 Deploying the Test and Availability Options ...................................................................6 2.5 Name, Description and Instructions .................................................................................7 2.6 Showing Results and Feedback to students .....................................................................7 2.7 Respondus LockDown Browser .........................................................................................8

3. Creating a New Test ...............................................................................................................9 3.1 Using the Test Manager ...................................................................................................9 3.2 Creating a New Test ...................................................................................................... 10

4. Creating Questions .............................................................................................................. 11 4.1 The Test Canvas Explained ............................................................................................ 11 4.2 Question Settings .......................................................................................................... 12 4.3 Creating New Questions: Using the Test Canvas .......................................................... 13 4.4 Managing the Test Questions ....................................................................................... 15

5. Deploying a Test .................................................................................................................. 17 5.1 Deploying a Test ............................................................................................................ 17 5.2 Setting the Test Options ................................................................................................ 17 5.3 Best Practices ................................................................................................................ 21 5.4 Where to Find the Grades? ........................................................................................... 23 5.5 Test Statistics ................................................................................................................. 23

6. Important Information for Using Tests ................................................................................ 25 6.1 General Precautions ...................................................................................................... 25 6.2 Using Multimedia or Special Characters (e.g. “ç”, ”é” or Chinese) ............................... 25 6.3 Information to Communicate to students ..................................................................... 26 6.4 Troubleshooting during Test: Possible Scenarios .......................................................... 27

7. Grading a Test ..................................................................................................................... 29 7.1 Grading per Student ...................................................................................................... 29 7.3 Manually submitting Grades: “Quick Grading”............................................................. 30 7.4 Making the Grades available to Students ..................................................................... 30 7.5 Viewing Results per Question ........................................................................................ 31

Appendix A: List of Question Types .......................................................................................... 32 Appendix B: Exporting and Importing Tests ............................................................................ 35 Appendix C: Creating Test Questions with a Text Tool ............................................................ 36

3

1. Introduction This manual contains information about creating digital exams (Tests) in Blackboard 9.1.

Tests are sets of questions that are graded to measure student performance, for instance to

check the knowledge and skill level of users enrolled in the Course. Tests permit the

Instructor to assign point values to each question in the Test. When a student completes a

Test, it is submitted for grading. The results are recorded in the Grade Center. Depending on

the used question types, Blackboard will automatically calculate and assign the total point

value to a student’s Test. Open questions, however, will need to be reviewed by the

Instructor.

Fixed question list vs. question pools This manual discusses how to create a basic Test with a fixed questionnaire. Blackboard also

features a Pool Manager, that can be used to create a Test that picks random questions out

of Question Pools to create a different set of questions for every student. Creating Tests

using the Pool manager will not be discussed in this manual.

Difference between Tests and Surveys Surveys are not the same as Tests. Surveys are not graded, anonymous, and the questions in

Surveys are not assigned a point value. Therefore, Surveys are useful for polling purposes,

and class evaluations. Information about Surveys will not be featured in this manual.

Assumed knowledge before reading this manual This manual assumes you are familiar with the basic actions in Blackboard 9.1, and that the

Edit Mode is “ON”. These actions are discussed in the Basic Instructor Manual Blackboard

9.1. All manuals (e.g. Manuals on Assignments and the Grade Center, Safe Testing with the

Respondus LockDown Browser) can be found on our website.

Further reading

SURF, Stappenplan ‘Toetsvragen ontwikkelen in 8 stappen’ (in Dutch).

4

1.1 Structure of this Manual

Decide on the

appropriate Test

settings for your Test

Create a new Test

Create Questions

Deploy the Test

Students take the Test

• There are many Test settings that are applied on various levels of the Test. Use the checklist to make sure all bases are covered.

• See chapter 2

• Create a new Test by navigating to Control Panel > Tests > Surveys and Pools > Test > Build Test.

• See chapter 3

• Create questions using the Test Manager. • See chapter 4

• Deploying will make the Test available for students in the course. Deploying allows the Instructor to set the Test Options.

• See chapter 5

• Important considerations, troubleshooting, and information to communicate to students should be reviewed.

• See chapter 6

5

2. Important: Checklist before Creating a Test

2.1 Introduction and Goals of the Test When you decide to create a Test, it is important to think about some things before starting

to build the Test. The following checklist will help you make decisions and will ascertain that

you will not have to do a lot of extra work once you have started creating a Test, as a lot of

the options take a lot of time to change once the test and the questions have been created.

The first step is to determine the goal of your Test in order to decide whether you need to

take any precautions to limit student’s access to the Test and to secondary auxiliary tools.

Test goal Explanation

Self assessment Students can use the Test to test their knowledge on a subject, e.g. in preparation of a (written) exam or as an example for an official Blackboard Test.

The Tests can be created without any precautions;

read § 2.2 to 2.5.

Diagnostic Test (diagnostic, formative)

The Test is used to assess the student’s knowledge, but there are no grades that will be based on the Test results.

Some precautions could be necessary, based on personal preferences;

read § 2.2 to 2.7.

Exam or Final Test for official grade (summative)

The Test is used to determine an official grade for a student. It is important that students cannot use other means of communication (internet, cell phones, working together, etc.).

take appropriate measures to prevent the use of means of communication that could influence the Test results;

read § 2.2 to 2.7.

2.2 Test Design The design of the Test can be modified in different ways; on Test level (chapter 4) and on

Test Options level (chapter 5.2).

Subject Question How to Question type What type of questions do you want to use? See §2.3

The appendix lists all available question types.

§ 4.3

Layout Text color, size and font can be changed:

make sure to use a color, font and text size that are clearly visible and readable;

keep in mind that students can be color-blind.

§ 4.3.1

Answer order Do you want the answers to the questions to be shown randomly to students or not?

Randomizing answers prevent you from using a lot of the same answers (for instance a lot of B’s).

§ 4.3.2

Question order Should the questions be shown in a fixed or a random order?

this could prevent students from working together;

less suited if there is a specific arrangement in your question list.

§ 5.2.6

Display questions Do you want students to see all questions at once or one at a time?

Students prefer showing all questions at once, as one at a time results in a lot of clicking for students;

prevent backtracking can be used to prevent students to

§ 5.2.6

6

go back to view or change their answers.

If some questions contain answers of other questions, you can make the students view them one at a time and prevent backtracking.

Feedback Do you want to enter feedback for students when questions are answered right and/or wrong?

Feedback improves the scores of students.

Feedback is for diagnostic Tests only.

§ 4.3.4 § 5.2.5

2.3 Questions and Grading How to: chapter 4. All possible question types can be found in the Appendix.

Subject Question How to

Question Type What type of questions do you want to use?

Blackboard offers both open and closed questions.

Keep in mind that open questions will require a manual correction of the given answers.

§4.3

Point Value How many points should be assigned to a good answer?

Determine in advance how many points a student can get for a good answer.

It is possible to weigh the questions by assigning different point values to different questions.

§4.2.4 §4.4.3

Partial Credit Determine whether students can score a partial credit when answering the question partially right.

This is possible when using for instance multiple answer questions.

§4.3.2

Extra Credit Do you want certain questions to count as Extra Credit only?

If this option is selected points are added to the score if the question is answered correctly; no points are taken away if the question is answered incorrectly.

§4.4.3

URLs, Files and images

Do you want to use URLs, files and images in the Test?

To do so the default settings have to be changed.

§4.3.1 §4.2.2

2.4 Deploying the Test and Availability Options Read more about this and how to change these options in chapter 5.

Subject Question How to

Location Decide where (Menu Button) you want to deploy the Test.

Make sure to place the Test at a location that is easily accessible for students (e.g. in Assignments).

-

Availability period Ascertain the period the Test should be visible to students:

e.g. one week before class, always, or only one day.

§ 5.2.2

Password Do you want to use a Password?

This way students can only access the Test once they know the password thus restricting Test access to students that are not physically in the classroom.

§ 5.2.2

Amount of attempts How many times is a student allowed to make the Test?

Blackboard offers the options once, unlimited, or a number of attempts to take the Test.

Multiple attempts is recommended to undercut technical problems. When making the Test for an official grade,

§ 5.2.2

7

however, students can abuse this function to take the Test twice. The Grade Center will display the amount of attempts for each student.

Time limit Do you want to set a time limit in which a student has to finish the Test?

Only use this option if absolutely necessary.

A student will receive a notification when they have one minute to go, but the Test will not be saved and closed automatically. Tests that have been submitted after the timer reaches 0 will be accepted, but are displayed without a score in the Grade Center. The attempt can be reviewed and accepted manually by an Instructor.

§ 5.2.2

Save and resume later

Should the student finish his Test the first time it is started or is the student allowed to save an attempt and finish it later?

§ 5.2.2

2.5 Name, Description and Instructions You can add a description and instructions to each Test. Before starting a Test the students

will see the description. After opening the Test the students will see the instructions. Read

more about this and how to change these options in chapter 3.2 and 4.1.

Subject Question How to

Name What name should the Test have?

When deploying multiple Tests, make sure the name clearly distinguishes the correct Test to ensure all students start the right Test.

§ 3.2

Description This will be available before starting the Test. This is the place to display:

Topic;

allowed auxiliary materials (books, hand-outs etc.);

the expected time it will take to complete the Test;

how to Test the hardware facilities.

§ 3.2 § 4.1

Instructions This will be available after starting the Test. Notify students about:

The types of questions;

if backtracking is allowed;

how many questions have to be answered;

how to correct an error in their answer;

how to act when a student encounters a technical problem;

how to finish the Test: use button Save and Submit.

§ 3.2 § 4.1

2.6 Showing Results and Feedback to students Determine if and what kind of feedback you want to communicate to students after the Test

is finished. Read more about this and how to change these options at chapter 5.2.

Subject Question How to

Score Do you want to show the achieved score after a student finished the Test?

The score will be instantly shown if you only use closed questions;

When using open questions, the scores will be shown after reviewing the open questions.

§ 5.2.5

Given answers Do you want to show the given answers? § 5.2.5

8

Blackboard will show the given answers and if they are correct or wrong;

If a student is allowed to take the Test more than once, this could help them achieve a higher score the second time!

Correct answers Do you want to show the correct answers?

Blackboard will show the given answers and whether they are correct or wrong. If they are wrong, Blackboard will also show the correct answer;

If students are allowed to take the Test more than once, this could help them achieve a higher score the second time!

§ 5.2.5

Feedback (toggling Feedback changes a diagnostic test into a summative one, and vice versa)

Do you want to show feedback to students?

There can be different feedback entered for correct and wrong answers;

The feedback has to be entered when creating the question;

The decision to show the feedback is made when entering the Test Options. It is possible to not show entered feedback;

The feedback is shown to students after completion.

§ 5.2.5

2.7 Respondus LockDown Browser When the Test is taken for an official grade, the Respondus LockDown Browser can be used

to block all activities on the computer other than Blackboard (internet, instant messaging,

etc.). It is important to note that pop-ups are also blocked. A few considerations should be

taken into account as a result of this:

Make sure the Test is not opened in a new window. As opening the Test in a new

window is considered as a pop-up, students will not be able to open the Test

because pop-ups are blocked.

When the timer is activated, the student will not receive a notification when the last

minute has started to count down.

Multimedia files (videos and sound clips) have to be embedded with Windows

Media Player in HTML to ensure they can be played.

YouTube movies have to be embedded because pop-ups are blocked.

The Respondus Lockdown Browser only operates with Internet Explorer under Windows and

with Safari on Apple computers. The Respondus Lockdown Browser is installed in the

computer labs at the Lipsius building, operating under Windows. Tests with the Respondus

Lockdown Browser can only be taken there.

A manual on the use of the Respondus Lockdown Browser is available on our website. Please

review this manual for further information or send us an e-mail.

9

3. Creating a New Test

This section of the manual discusses how to create a

Test. As discussed in the introduction, Tests are sets

of questions that are graded to measure student

performance. Remember that we only discuss Tests

with a fixed questionnaire. (Please see the Pool

Manager manual on how to draw questions

randomly from a pool of questions.) Tests are

created within the Test Manager. After their

creation, questions can be added and question

settings can be changed (Chapter 4). Tests then

have to be deployed to make them available for

students. In chapter 5 the Test options for

deployment are explained.

3.1 Using the Test Manager The Test Manager is the place where all Tests in a

course can be found. Here, new Tests can be

created and already created Tests can be reviewed,

changed, or updated. It is therefore possible to

create a Test and work on it in phases. The Test is

saved automatically, and adding questions can be

done later.

To navigate to the Test Manager, take the following steps:

1. Go to the Control Panel > Course Tools. 2. Choose Tests, Surveys and Pools. 3. In the screen that appears next, choose

Tests. You will now be directed to the Test Manager.

The Test manager will show all Tests created in a course. In this example, there are two Tests

already created.

Note: Before creating a new test and entering any questions, create your questions,

answers, and metadata in a separate file outside of Blackboard, e.g. in a spreadsheet.

Once entered, it is not so easy to extract your assets from Blackboard, if you ever want

to. The rationale is to become ‘future proof’, that is not more dependent on systems than

necessary.

10

In the Test Manager, you can do three things: 1. Build a new Test. 2. Import an existing Test. 3. Edit, Export, Copy or Delete the already created Tests (to view this menu, click the

arrow behind the Test name).

3.2 Creating a New Test To create a Test, take the following steps:

1. Navigate to the Test Manager. 2. Choose Build Test. 3. Fill out the form that appears next:

Name: this name will be shown to students and in the Grade Center;

4. Choose Submit.

Note: the Description is visible before students open the Test. Use this field to inform

students about for instance the due date. Instructions become visible after opening the

Test. Use this field to inform students about the Test options (chapter 5) like a set timer.

Both fields can be changed at all times.

11

4. Creating Questions

4.1 The Test Canvas Explained The Test Canvas allows Instructors to add and edit questions, add Question Sets or Random

Blocks, reorder questions, and review the Test before deploying it to users. The Test Canvas

will provide users with the menu interface displayed below. In this section, we will discuss

the functions of this Test Canvas. You can enter the Test Canvas by clicking the arrow ( )

behind the Test name in the Test Manager window (3.1) and choosing Edit.

1. The Title menu

The Title menu can be used to change the name, description and instructions for the Test. To

do so, click the arrow behind the name of the Test (in this example: Test 1).

2. Create Question

The Create Question function can be used to add new questions to the Test. By clicking the

menu button, you will be provided with a list of available question types. In section 4.3 we

will discuss how to create a question step by step. In chapter 6 a list of all question types is

provided.

3. Reuse Question

The Reuse Question function can be used to create Question Sets, Random Question blocks,

or to find (and import) questions from other Tests or Pools in this course. This function will

not be explained in this manual. More information can be found in the Pool Manager

Manual.

4. Upload Questions

Questions can be authored offline in a specially formatted text file and uploaded into Tests,

Surveys, and Question Pools. Once uploaded, questions can be edited and used like

questions that have been authored in the system. The function Upload Questions can be

used to upload such text file formats. However, this function is very error-prone, and we do

not recommend using it. Examples are in Appendix C: Creating Test Questions with a Text

Tool.

5. Question Settings

Question Settings controls the options available when creating Test questions. In the next

paragraph, all Question Settings will be explained in more detail.

12

4.2 Question Settings The function Question Settings controls the options

available when creating Test questions. The image

to the right shows the options and the fields that are

enabled or disabled by default. It is important to

realize that checking boxes will create fields for the

concerned options. It is for instance possible to give

feedback to individual answers instead of to the

whole question. This is however not required, so

feedback fields can also be left empty even if you

have checked this box.

4.2.1 Feedback

By default, it is possible to give feedback for the

whole question. By checking this box, Blackboard

will show feedback fields for all individual answer

options. Individual feedback cannot be provided for

answers in True/False questions, Multiple Answer

questions, Ordering questions or Matching

questions.

4.2.2 Images, Files, and Web Links

By checking the first box, Blackboard will display

option fields to attach images, files, and web links to

questions. Checking the second box gives students the option to attach images and files to

answers.

4.2.3 Question Metadata

Allows the user to use Categories, Keywords, and Levels of Difficulty to add metadata to

questions, making searching for questions from a Question Pool or from other Tests and

Surveys easier. This box is checked by default. The question type is by default part of the

metadata.

4.2.4 Scoring

Determines how points are awarded for questions:

Set a Default point value for each question. This value can be changed for each individual question added in the Test Canvas. When adding questions from a Question Pool or from another Test, you can specify whether you want these questions to keep the point value originally assigned or to use the default points when adding the question to your Test.

Specify partial credit: shows a field to award a (linear) proportion of the total number of points for a question that is partly correct. This is useful, e.g., for Multiple Answer questions.

Specify negative points options for answers: checking this box will allow you to penalize students with negative points for incorrect answers (often used in multiple-choice tests to discourage guessing). It is recommended that you mention using this option to students in the Test Instructions.

13

Provide Extra credit: provides an option to reward points for a correct answer but do not deduct points for a wrong answer when creating questions. This makes it possible for a student to achieve a score greater than 10 on the Test.

4.2.5 Display

Determine how questions are displayed to students:

Random order: shows answer choices randomly for each student.

Horizontal or vertical display: provides the option of formatting the positioning of questions and answers.

Numbering options: provides a selection of alphanumeric labels to identify each answer.

4.3 Creating New Questions: Using the

Test Canvas This section will discuss how to create questions using

the Test Canvas. To create any of these question types,

take the following steps:

1. Navigate to the Test Canvas (see section 3.1). 2. Click Create Question. 3. Select the desired question type.

In this section, we will discuss the creation of a single

question. In the Appendix, all question types and their creation specifics are listed.

4.3.1 Entering the Question

To create a new question, take the following steps:

1. Navigate to the Test Canvas (see chapter 3.1). 2. Click Create Question. 3. Select the desired question type. In this example, we will work with Multiple Choice.

In the screen that appears next, the Question text can be entered in section 1. Verify that a

clearly visible and readable sans serif font is selected, like Arial 12pt, black. Bear in mind that

once you decide to change the font, font size, and/or colour this cannot be done for all

Note: there are open and closed question types.

Blackboard automatically calculates the score for

a Test when there are only closed question types

used. When open question types (like Essay) are

used, the instructor has to assign the score

manually. These Tests will be displayed in the

Grade Center with the icon.

14

questions at once, but must be set per question, which is very time-consuming. Further, you

can add a File (mp3 clip/image) or an External Link to a website by using the buttons

highlighted with the red frame. Hovering over these buttons will display a text bubble

explaining the options.

4.3.2 Answer Display Options

In section 2 of the form, the display of the answer

options can be altered.

Answer Numbering: show numbers in front of each answer option.

Answer Orientation: show answers horizontally or vertically aligned.

Allow Partial Credit: assign a partial point credit (in %) to all answer options.

Show Answers in Random Order: shows the answer options to each student that takes the Test in random order. This setting is recommended.

4.3.3 Answer Options

In section 3, the Answer options can be specified.

Add answer options by selecting the desired amount in the field Number of Answers. A Multiple Choice question has 4 answer options by default.

Remove Question options by clicking Remove behind a question option

Write down the possible answers.

If Partial Credit is enabled (see 4.3.2), you can enter the percentage of points that is to be assigned if this answer option is selected

Select the Correct answer by checking the box in front of it.

4.3.4 Giving Feedback

In part 4 of the form, it is possible to enter

(optional) feedback that students will see after

completing the Test. Consult chapter 4.2.1 when

feedback fields are not displayed. You can enter feedback for both correct and incorrect

answers. Depending on the correctness of the answer of the student the correct or incorrect

response feedback will be shown. If partial credit is allowed, answers that are partially

Note: if the answers are shown in random order, the answer can be entered in the first

field as Blackboard will randomly order all answer options for each individual student.

When not using the random answer option, you will have to vary the answer options

yourself. Make sure to select an even amount of every answer choice (A, B, C, D).

Note: the Feedback is not automatically shown to students. You can later decide whether

or not to show the entered feedback. How to show feedback is explained further in

section 5.2.5: Test feedback.

15

correct will receive the feedback for an incorrect answer. Enter the desired feedback in the

textbox.

4.3.5 Using Metadata: Categories and Keywords

Categories, Topics, Levels of Difficulty and Keywords are types of metadata added to

questions to make them easier to retrieve in searches, to create Question Sets, or to use

questions from Pools and or other Assessments.

This information is optional. Click Add to add a

Category, Topic, Level of Difficulty, or Keyword to

the question. Bloom’s Taxonomy is often found

helpful for determining metadata.

4.3.6 Adding Extra Questions

You may want to make some introductory questions

such that students can get acquainted with the way

of testing in Blackboard.

4.4 Managing the Test Questions

4.4.1 Editing and Deleting Existing Questions

Once a question is created, it will be displayed on

the Test Canvas. To change the question, take the

following steps:

1. Click the arrow behind the question title. 2. To edit the question, select Edit from the

drop down menu. 3. To delete the question, choose Delete from the

drop down menu.

To Delete multiple questions, select the checkboxes in

front of the questions you want to delete and select

Delete from the menu bar. It is also possible to select all

questions at once by clicking Select: All in the menu bar.

4.4.2 Changing the Question Order

Changing the Question order is done using Drag and Drop.

1. Grab the handle ( ) in front of the question. 2. Drag the question to the desired location in the

question list.

4.4.3 Changing the Point Value of Questions and Using Extra Credit Questions

To change the point

value of questions,

click the point value

behind the question.

In the pop-up menu

16

that appears, you can now enter the new point value, select whether the question should be

marked as an Extra Credit Question, and Submit the changes. Extra credit means that points

are added to the score if the question is answered correctly, but no points are taken away if

the question is answered incorrectly, therefore students can obtain a score higher than

100% if you use Extra Credit Questions.

A final warning is at stake. Be sure about the assigned point values, as it is not possible to

change them for all questions at once. They can only be modified per question.

17

5. Deploying a Test This section discusses how to deploy a Test in your Blackboard course. Deploying the Test

allows the Instructor to make a Test available to students. Also, all display options will be

discussed.

5.1 Deploying a Test To deploy a Test, take the following steps:

1. Navigate to the content area (menu button) in which you want to make the Test

available for students (e.g. Assignments). 2. Choose Create Assessment. 3. Choose Test.

In the screen that appears next, take the following steps:

1. Select the Test you want to deploy from the list (e.g. Test).

2. Click Submit.

You will now be redirected to a form that allows

you to set all the Test Options. It is very important

to consider all options and their consequences for

the Test. Therefore, please read chapter 5.2 for a

description of all these options.

5.2 Setting the Test Options The Test Options can be opened by taking the following steps:

1. Navigate to the Content Area in which the Test is deployed (e.g. Assignments).

2. Click the arrow behind the Test name. 3. Choose Edit the Test Options.

5.2.1 Test Information

The Test Information allows the Instructor to set

the general display options of the Test within the

Course Menu. You can set the Test Name and the

Description that are displayed in the content area of

the menu button. Choose a clear name such that

students know which Test to take. Use the

Description to provide students with the

information they need before starting the Test. This

description could also be entered when creating the Test. It is displayed in this step again to

let you check the description one last time.

Open Test in New Window will open the Test in a new browser window (as a pop-up) if set

to Yes. This allows students to simultaneously take the Test and view course content.

However, when using a safe browser, like the Respondus Lockdown Browser, or when a pop-

18

up blocker is enabled, this setting can cause problems for students when trying to start the

Test. The setting No is recommended.

5.2.2 Test Availability

Make the Link Available: Select Yes to allow

students to access the Test. To set a specific range

of time when the Test is available, make the Link

available and then limit this availability by entering

the dates and times of the range using the drop-

down menus below (Display After/Until).

Add a New Announcement for this Test: Select Yes

to add an automatically generated Announcement

to notify students about the Test.

Multiple Attempts: Allow students to take the Test

an unlimited or set number of times. Left unchecked, students are allowed one attempt to

take the Test.

Force completion: By default, students can save and exit the Test to work on it later

(recommended). When checking this box, students must complete the Test the first time it is

launched (this is effective, but error sensitive).

Set timer: When a timer is set, students are required to finish the Test within the set time.

The timer will be displayed to the students during the Test. When the Auto-Submit option is

ON, the Test is submitted automatically when the timer reaches 0. When Auto-Submit is

OFF, students can finish the Test even after the timer reaches 0, but submissions after the

set time will be marked late with an -icon in the Grade Center. We advise to put the Auto-

Submit function to ON.

Note: Giving students one attempt and forcing them to complete the Test the first time it

is launched can be useful in an official Test setting, as students will not be able to take the

Test multiple times to obtain a better score. However, the combination of these settings

can block students from their Test when they encounter a technical problem. Therefore,

pros and cons of these settings should be reviewed before deciding whether or not to use

them. Multiple Test attempts are visible in the Grade Center when reviewing the score

details, but not in the main overview.

Note: When Force Completion is off, the student can reenter the exam. The exam timer

(see below) continues to record time even if the student has exited the attempt.

Therefore it is better to make the exam times by choosing the Set Timer option.

Note: Students will be notified by a pop-up when they have one minute left. When using

the Respondus Lockdown Browser, this pop-up will be blocked!

19

Display After/Until: you can enter dates between which the Test is visible for students. This

can be used to limit the time frame in which students can take the Test. For safety, make the

interval as small as possible, e.g. 15 minutes. It is also possible to enter only one of the two

dates. Always make sure to Make the Link Available (see above) when using the display

settings or the Test will not be displayed at all.

Password: when activated, a password is required to access the Test. Passwords are limited

to 15 characters and are case sensitive. Passwords are useful when students have to take

Tests at the university in different groups. This way the Test can then be made available

without students at other locations being able to taking the Test unauthorized.

5.2.3 Due Date

The Due Date can be used to show students that

this Test is due in the To Do tool on their

Notifications Dashboard at the home page. There

are no consequences when students take a Test

after the Due Date.

5.2.4 Self-assessment Options

Include this Test in Grade Center Score

Calculations: Test scores do not need to be revealed

to students to be used in Grade Center calculation.

Self-Assessment Tests, i.e. diagnostic, or formative

Tests, are generally not included in Grade Center

calculations.

Hide Results for this Test Completely from Instructor and the Grade Center: hides Test

scores in the Grade Center. Hiding the score does not hide the Test Column in the Grade

Center. This Test Column will display if students Tests are Complete or Incomplete to

instructors only. If this option is checked, the Instructor will not be able to see any student

grades, view answers, aggregate results, or download result details. To protect student

privacy, this choice cannot be reversed later without deleting all attempts.

5.2.5 Test Feedback

Test Feedback becomes visible for students after

completing the whole Test, and regarding all given

answers. Feedback cannot be given per question.

Instructors can select which information is visible.

Note: do not enter a password here when using the Respondus Lockdown Browser! When

using the Respondus Lockdown Browser, a password can be set as well but has to be

entered somewhere else. It will be stored in encrypted form (see the Respondus

Lockdown Browser manual).

20

Score: Shows the score the student achieved out of points possible.

Submitted Answers: Shows the scored points for all questions, shows all the submitted

answers, and whether they were correct or incorrect.

Correct Answers: Shows all answers the student entered, and shows the correct answers

and alternative answers when a student answered the question incorrectly, or when there is

more than one correct answer to a question.

Feedback: Shows Instructor-generated feedback messages for each question. This feedback

has to be entered manually by the Instructor for each question while creating the Test. See

chapters 4.2.1 and 4.3.4.

5.2.6 Test Presentation

The section allows Instructors to control the way the

Test is presented to students using these settings:

All at Once - The entire Test is displayed on one

screen. Students may have to scroll down to answer

questions.

One at a Time - The questions are displayed one by

one on the screen. Students control going to the

next question by clicking a button. Only use this

option if really necessary, as this option results in a lot of clicking for students.

Prohibit Backtracking - The setting can only be used when questions are displayed one at a

time. When checked, students are not allowed to go back and change the answer to a

question that has already been answered.

Randomize Questions – This option shows the questions in a random order to each

individual student. This way, every student taking the Test will see the questions in a

different order. When multiple attempts are permitted, the same student will see questions

in a different order when taking the Test multiple times.

Note: Showing students correct answers or feedback might not be practical when

working with different groups. On the other hand, students will appreciate feedback,

because they will want to know what kind of mistakes they made.

Note: Showing Submitted Answers allows students to immediately view their mistakes,

while not sharing the correct answers. However, because it is displayed if answers are

correct or incorrect it is easy for the student to get a better score when taking the Test

again.

Note: Feedback has to be enabled to show the feedback that was entered when creating

the questions of a certain Test. Typically, this option is checked for a diagnostic or

formative Test, while it is unchecked for a summative Test.

21

Finally, click Submit to deploy the Test with the selected settings.

5.2.7 Calculation of Score with Partial Credit Enabled

When partial credit is enabled, students can receive a partial score for a partially correct

answer. Depending on the Question type, the partial credit score can be entered for each

incorrect question when creating the Question (e.g. Multiple Choice questions) or it will be

calculated by Blackboard automatically (e.g. Multiple Answer questions). In the second case,

students will receive a percentage for each correct answer and be deducted an equal

measure of partial credit for each incorrect answer. The amount of partial credit for each

answer is derived from the total number of points for the question divided by the number of

correct answers. Negative points are not given, 0 is the lowest score for the question. So, if a

question is assigned 10 points and there are five answer options, with three correct answers,

each correct answer receives 3 1/3 point. A wrong answer receives 3 1/3 point deduction.

5.3 Best Practices Practice has shown that not all available Test options can be combined freely. Factors are,

among others: complexity of the Test (question types), student group size (we think

Blackboard’s advice to the maximum of 300 users simultaneously taking the Test is too high),

and the use of audio/video files. Furthermore, options for diagnostic Tests may differ from

options for summative, graded Tests. These best practices are still under development.

Subject Value and Reason How to

When to Use Blackboard for Digital Exams?

Do not use Blackboard for summative, graded Tests, yet. Chances on failure cannot be ignored. You may want to use Blackboard for diagnostic Tests, and for self-study modules.

§ 5.2.6

Respondus LockDown Browser

You may want to use the Respondus LockDown Browser for diagnostic Tests, the grades of which do not count as an independent score, but rather entitle students access to a summative Test (on paper), or which give students extra credits for their final paper etc. ● Set Open Test in New Window to No. Opening a new window would be blocked by Respondus Lockdown Browser. ● Take care when using embed audio and video fragments since attached files are opened by a local media player and will be blocked by Respondus LockDown Browser. ● The preferred Language of the course module containing the Test is English or System Default (see Control Panel > Customization > Properties > Select Language Pack). Other languages might prevent students to log on.

§ 5.2.1 § 2.7

Note: when using the Prohibit Backtracking function in combination with the Randomize

Questions function while showing the questions One at a Time, questions will be

presented in a different order to each student and students are unable to go back to

already answered questions. This ensures students can’t cheat by looking at the screen

of another student during the Test.

However, problems with the combination of these presentation modes have been

reported when large amounts of students take the Test at the same moment. When

using multimedia fragments, the continuous requests to load images or audio/video

content can cause the Test to jam.

22

Test Availability ● Check Multiple Attempts, Number of Attempts = 2 or unlimited. This leaves room for restarting the Test a second time when students encounter problems. ● Do not check Force Completion, in order to allow students to save, exit and resume the Test later. This minimizes problems when students accidentally close the browser. ● Do not use the Timer function. Do not set an explicit time limit, but tell your students in class or rather in the Test instructions about the duration of the Test. This leaves room for unexpected time delays while starting up the Test. Furthermore, when students run out of time, a timer pop-up might not show up when a pop-up blocker is installed on the computer.

§ 5.2.2

Presentation Mode

● Set Open Test in New Window to No. ● Set Presentation Mode to One at a Time, although this means that students have to do a lot of clicking. Each time a student clicks for accessing the next question, the connection between client and server is confirmed. After 30 minutes of idle time, e.g. when making an Essay assignment, Blackboard breaks the connection. Especially use this setting for large (>100) groups. As students work at their own speed, server load and bandwidth usage is spread over time (still to be checked whether this is true). ● When making an Essay Assignment, set the option Hide course menu to No. The recommended height for the editor is 40. ● Set Presentation Mode to One at a Time when there are many audio or video fragments in the Test; they will successively be loaded when needed in the Test, not taking too much bandwidth at the start of the Test anymore (still to be checked whether this is true). ● Leave Prohibit Backtracking unchecked, to allow students to browse through the questions. ● Set Presentation Mode to All at Once when your Test takes shorter than 30 minutes, or when students regularly save their answers (still to be checked whether this true). ● When you set Presentation Mode to All at once, make sure students do not all start the Test at the same time to prevent a server overload (large groups of students) ● Set Presentation Mode to Randomize Questions, unless your questions have a specific order or when you want to do an extensive item analysis afterwards. Blackboard will shuffle the questions for each individual student, which hinders aggregation.

§ 5.2.6

Save Answers Always stress the importance of saving the Test answers before submitting the test (manually or AutoSaved). The AutoSave functionality saves given answers approx. every 8 seconds on the server (and not locally in a cookie). Manual as well as automatic saving also confirms the client-server connection, which prevents the Test from being timed-out.

Blackboard’s overall advice – Nov. 2014

● “Avoid creating large exams involving many/complex questions and presented all at once. - 50 questions is already a lot of questions. - Consider breaking them into smaller exams taken in sequence instead. - Large exams also create much more server load when submitting the final exam, which can lead to failures due to application overload when groups of users submit at the same time, so when taking large in-classroom exams, please stagger the submissions.

23

● Avoid randomized display order for exams using question-by-question display. ● When using random selection of questions from pools, keep the overall exam especially short. ● Give the exam one question at a time. This ensures that the user's Java session is continuously updated, and the likelihood of a user not hitting the system to persist his/her session is reduced. ● Turn off the Force completion option. This allows the user to persist his/her answers. In case of any issues with the session timing out, the user will be able to re-enter the exam from the last point he/she saved, allowing them to submit the exam with minimal data loss. In addition, the Save button will also function the same as the solution above, allowing the java session to persist for a longer period of time. ● Do not use randomized tests. While this is most likely an unusable workaround, the reason randomization is a problem is that each version of the exam in a randomized test is created in Java memory only. Therefore, during a randomized test if the Java session for the user dies, the attempt will not persist anywhere and will be completely unrecoverable. Removing randomization will use a version of the test from the database, so the user would get the same test with the same order the next time he/she took the test. ● Create tests not having more than 50 to 100 questions maximum , especially when multiple large tests are being attempted at the same time. In these cases you need to display One question at a time, turn off the Force completion option and also avoid using the Randomize feature.

These are the best practices for reducing the charge. Otherwise it all mostly depends on your hardware sizing.“

5.4 Where to Find the Grades? Instructors can find the scores for closed questions in the Grade Center. Open questions still

need to be graded manually. See chapter 4 and 5 of the Advanced Manual on Assignments,

SafeAssignments and the Grade Center on our website. Note that a score does not always

relate 1:1 to a grade. Therefore, an Instructor might want to hide the scores for students. To

do this:

1. Find the column that corresponds to the Test in the Grade Center;

2. Click the arrow in the column header > choose Edit column information; 3. Note that the one-and-only place to publish official grades is the student

Information System uSis. Blackboard can be used to communicate grades for different course components, but the final grade should always be entered in uSis as well.

Students can find their grades via the My Institution page > Tools > My Grades or by a hand-

made menu button in the course. See also chapter 6 in the Grade Centre Manual.

5.5 Test Statistics Blackboard does not provide a full-fledged analysis tool that can be used to analyse the

results of the Test. However, some Test Statistics are available.

1. Open the Full Grade Center. 2. Find the column with the name of the Test.

24

3. Click the arrow behind the Test name. 4. Click Column Statistics.

Here you will find Count (number of students that took the Test), Minimum & Maximum

Value, Standard Deviation and Variance, an overview of the Status of the Test and the

Distribution of the Grades.

More detailed statistics are available under Attempts Statistics:

1. Open the Full Grade Center. 2. Find the column with the name of the Test.

3. Click the arrow behind the Test name. 4. Click Attempts Statistics.

Here you will find per question (depending on the question type): Average Score, Percent

(In)Correct, Total Attempts, Unanswered Attempts, Correct Answer, Partially Correct

Answer, Given Answers & Frequency.

One of the indicators of the quality of a question is the so-called p-value. The p-value is

defined by the proportion of the students that answered the (closed) question correctly. The

p-value can be found under Attempts Statistics, Attempt Count percents. In a (closed)

question a student can guess the correct answer. The corrected p-value can be computed by

the formula pc = p – ( (1-p) / (K-1) ), where K is the number of answer alternatives. pc stands

for the proportion of the students that answered the (closed) question correctly without

guessing. The ideal p-value is higher than (1 + 1/K) / 2 (and lower than or equal to 1,0). For

small groups (<25) the p-values must not be overestimated.

For more information on Tests and psychometric analysis see:

Berkel, Henk van & Anneke Bax, Toetsen in het hoger onderwijs, 2e herziene druk, Bohn Stafleu Van Loghum, Houten, 2006 (ch. 3.4),

Gruijter, D.N.M. de, Toetsing en toetsanalyse, nov. 2008.

De Gruijter (2008) discusses among other things determining the caesura and the

transformation from score to grade.

25

6. Important Information for Using Tests

6.1 General Precautions There is always a chance of technical problems when using an electronic environment to

take a Test. Computers can break down, servers can be overloaded, while students may have

problems logging in to the computers. In order to minimize problems, we recommend the

following precautions.

Notify the Blackboard Helpdesk about your Test use. We can provide assistance, and could be available for backup the first time your students take a Test, in case something goes wrong.

When making reservations for a computer room, make sure to book a room with a few extra computers so students can switch when a technical problem occurs.

Make, if possible, a paper-and-pencil version of the Test as a backup for technical failures.

If students are not familiar with other question types than Multiple Choice or are not familiar with Digital Examination in general, consider making a practice Test available.

6.2 Using Multimedia or Special Characters (e.g. “ç”, ”é” or Chinese)

When using digital media content (e.g. audio or video files) always check if the content works and make sure students test the facilities (speakers, headset etc.) before starting the Test.

When using multimedia, be careful with presenting questions One at a Time with the Prohibit Backtracking function enabled. The continuous loading of (large) files by (large) groups of users can cause Blackboard to jam.

The Blackboard Helpdesk has standard test items available to test the sound and the keyboard before starting the Test. Please send us an e-mail if you want these copied to your course. They can be modified to your personal preferences.

26

6.3 Information to Communicate to students Depending on the way Tests are used, the appropriate information should be communicated

to students.

Before taking a Test, take the following into account:

Does your Test require the use of multimedia facilities?

Make sure students test the computer facilities before taking the Test. Problems of this category usually cannot be solved once the Test is started. See also section 6.2.

Does your Test require students to enter special characters (think of ß, é, û, etc.)?

Make sure students test the keyboard and language settings by entering special characters, before starting the Test. Problems cannot be solved once the Test has started. See also section 6.2.

Which browser should students use? There is a list of supported browsers for the current Leiden University Blackboard environment, version 9.1, SP 11, including Internet Explorer, Firefox, Google Chrome, and Safari.

Internet Explorer 9 is installed in the computer labs in the Lipsius building;

For summative Tests a safe browser like the Respondus LockDown Browser can be used (see below). This browser is installed in the computer labs in the Lipsius building.

Pop-up blockers can cause problems, e.g. while playing media files in a formative Test. Make

sure students disable pop-up blockers in their browser before starting the Test.

Notify students (e.g. in the Test instructions):

How to log on. o Try to avoid that all students have to log on at the same time. This may

result in a heavy load on the server.

How to start the Test. o Make sure students start the Test some time after each other (this applies

especially to larger groups taking an official Test in the computer rooms of the faculty or in case of complicated Tests with multimedia content). This may be organized by a “wave” through the computer room (e.g.: one row after the other), and raising hands when the Test is ready. This is the case when the final question has been loaded, all images (if any) are visible, and all audio and video fragments (if any) have been setup by a small media player window.

o Explain how to use the Respondus Lockdown Browser, if appropriate. Students cannot surf the internet, read e-mails, instant messages,

nor can they access any other program on the computer than the Respondus Lockdown Browser

Tests that require the Respondus Lockdown Browser can’t be taken from home.

How many questions there are.

How much time is available, and how to handle a time limit, if used. o The timer will give a pop-up notification when the student has 1 minute left

and the remaining time is visible at all times while taking the Test. The Instructor can choose whether the Timer should be enforced once it reaches 0, or if students are allowed to continue after the timer is completed (see

27

section 5.2.2). Late submissions will not receive a score but are marked with

an -icon in the Grade Center. o When using the Respondus Lockdown Browser, the pop-up notifying the

student he/she has one minute left will not occur (as the Respondus Browser blocks all pop-ups).

How to correct an error in their answer.

How to save an answer: as soon as possible (Note: answers are saved automatically).

How to behave when there is a technical error (can they make the Test twice or resume?).

How to finish the Test: o Click the Save and Submit button; o Confirm that you really want to finish the Test; o after clicking OK, the results including answers and feedback (when

specified) are displayed.

While taking a Test, students are advised to:

Be aware of time-outs. If a student is inactive for 30 minutes, Blackboard will break the connection with the server, making it impossible to save and submit the Test; after 90 minutes of inactivity Blackboard will automatically log a student out.

Advise students to Save answers per question to make sure nothing gets lost.

Students can see their progress. At the top of the screen is the Question Completion Status, which can be unfolded.

After taking a Test The results of a Test (Assignment, Survey etc.) are stored in the Grade Center. However, official grades should be kept in the student administration system, which is external to Blackboard. In case of an official, that is a summative, graded Test it is important to save the results of the students in a place outside Blackboard for security reasons. Instructors are urgently advised to: Save a copy of the Grade Center corresponding to the Test; a copy can be made by

1. making a Print Screen (for small groups only) and store it in a Word file, or … 2. by exporting the Grade Center to an Excel spreadsheet (see the Manual on the

Grade Center).

A formative or diagnostic test seeks to improve the learning capabilities of students by

providing feedback. Storing the grades of such a Test is left to the instructor. It may be

necessary to do this when the formative test is used as a prerequisite for taking the official,

summative test.

6.4 Troubleshooting during Test: Possible Scenarios This section discusses some problems that can occur during a Test in the computer rooms and what to do to solve these problems.

A student has no sound on the headset: check cables (hardware) and volume settings (software) on the computer. If this does not work: let the student log off and use another computer.

A student can’t type any special characters: let the student log off and use another computer.

28

The Test occurs to have been jammed. The student can’t save or submit the Test or can’t go to the next question. Do not let the student log off! Go to another computer and log in, reopen the Test and continue. After the Test is submitted from the other computer, shut down the jammed computer using the Power button (do not log off first!).

When a student, while submitting a Test, gets the message “Cannot contact server” the connection with the server has been lost underway, for example after 30 minutes of inactivity. The student must close the Test, if possible in a regular way, and otherwise by pressing the Ctrl+Alt+Delete buttons, or even the Power button. (S)he will not be able to see the Score, nor the Correct Answers, if provided. As Blackboard saves answers automatically the Instructor can still see all given answers in the Grade Center, and is able to Save the Last Attempt. The Instructor must evaluate/grade each separate answer manually.

29

7. Grading a Test The Grade Center includes features to communicate and share assessment data directly with

students, fellow instructors, and institutional administrators. The Grade Center is a complex

module of Blackboard which justifies a separate manual. It is available at our website. A

short instruction will be given here on how to grade a Test. To open the Grade Center,

navigate to Control Panel > Grade Center > Needs Marking.

7.1 Grading per Student Blackboard automatically creates columns for Tests created in the course. To view and grade

an individual attempt, find the column with the name that corresponds with the name of the

Test. Move the mouse over the concerned cell of a student and click the arrow . From the

menu choose View Grade Details.

In the following screen, choose View Attempts to see the Test, and the given answers by the

student.

By clicking the button Test Information, which is by default collapsed, you will find the

Current Grade based on Last Graded Attempt, the Status of the Test (Completed or still

30

busy), Time Elapsed, and a Date and Time stamp. By using the < and > symbols one can

browse through all students.

Just under the Test Information the Test itself is repeated, exactly as when it was presented

to the student, including the given answers by the student, the correct answers, the

evaluation method and the scored points.

7.3 Manually submitting Grades: “Quick Grading” If you want to override the automatically calculated grade by your own, you can use Quick

Grading. It can be done for columns created by Blackboard or for columns created manually.

Just click the cell in which you want to enter a grade. Type the grade you want the student to

credit with. When finished, press Enter. The grade will now be displayed in the Grade

Center. Under Test Information the comment “Original grade has been overridden” is

added.

7.4 Making the Grades available to Students Grades are by default visible for students, but accessing them is not always easy. The best

option is to create a menu button in your course which enabled students to access the My

Grades section. Here, students can only view their own

grades. To create this menu button:

1. In the menu, click the button and choose

Tool Link.

2. Fill in the form:

a. Enter a Name.

b. Select My Grades as Type.

c. Check the box Available to Users.

3. Click Submit. The new menu button will now

appear in the course menu.

31

For an alternative method, and for keeping grades invisible for students, see the Manual on

the Grade Center.

7.5 Viewing Results per Question After students have completed a Test the results of each question can not only be viewed

per student, but also per question. The results will be presented in the form of percentages

for all questions types, except for fill-in-the-blank and essay questions which list each

response individually. See chapter 5.5 Test Statistics.

32

Appendix A: List of Question Types This appendix gives a full overview of all available question types.

Calculated Formula

A Calculated Formula question contains a formula with variables that change for each user.

The variable range is created by specifying a minimum value and a maximum value for each

variable. Answer sets are randomly generated. The correct answer can be a specific value or

a range of values. Partial credit may be granted for answers falling in a range (numeric or

percentage).

Calculated Numeric

This question resembles a fill-in-the-blank question. The user enters a number to complete a

statement. The correct answer can be a specific number or within a range of numbers.

Either/Or

Users are presented with a statement and asked to respond using a selection of pre-defined

two-choice answers.

Essay

Users are given the opportunity to type an answer into a text field. There is no word count

option. Sample answers can be added for users or graders to use as a reference. Feedback

cannot be assigned. This type of question is graded manually.

File Response

Users upload a file from the local drive as the answer to the question. This type of question

is graded manually.

Fill in the Blank

Fill in the blank answers are evaluated based on an exact text match. It is important to keep

the answers simple and limited to as few words as possible. Answers are not case sensitive

but are evaluated based on spelling. The position of the blank can be marked with one or

more underscores. Instead of an exact text match, answers can also be evaluated by a

pattern match (using regular expressions) – typically used to neutralize spelling equivalences

– or by requiring containment – typically used for proper names.

Fill in Multiple Blanks

This question type provides multiple fill-in-the-blank responses that can be inserted into a

sentence or paragraph. Separate sets of answers are defined for each blank. A maximum of

10 blanks per question is allowed. In the question text, mark each blank with a letter (a

‘variable’) between square brackets (e.g. [x]). Then, specify one or more answers for each

variable.

Hot Spot

Users indicate the answer by marking a specific point on an image. A range of pixel

coordinates is used to define the correct answer.

33

Jumbled Sentence

Users are shown a sentence with a few parts of the sentence as a blank with a drop-down

list to choose from to assemble the sentence. Only one set of answers, compiled into one

drop-down list is used for all blanks in the sentence. If the sentence has more than one

blank, enter the answers of all blanks in random order in the drop-down list.

Matching

Matching questions allow users to pair items in one column to items in another column.

Instructors may include different numbers of questions and answers in a Matching question.

Partial credit is given, if possible.

Multiple Choice

Multiple choice questions allow users to choose one correct answer. In multiple choice

questions, users indicate the correct answer by selecting a radio button. The number of

answer choices is limited to 20.

Multiple Answer

Multiple answer questions allow users to choose more than one answer from a list of

answer options. All correct answers have to be checked in order to gain the maximum

number of points. Blackboard does not have a question type where one answer out of some

alternatives can be chosen. To neutralize spelling equivalences, use the Fill in the Blank

question type with pattern matching.

Opinion Scale / Likert

Opinion questions are based on a rating scale designed to measure attitudes or reactions.

Users indicate the answer that represents their attitude.

Ordering

Ordering questions require users to provide an answer by selecting the correct order of a

series of items. Users will be granted partial credit for ordering questions if they answer part

of the question correctly.

Quiz Bowl

Quiz Bowl questions are a way to add fun and creativity to Tests, such as self-assessments or

in class contests. The user is shown the answer and responds by entering the correct

question into a text box. An answer must include a phrase and a question word, such as

whom, what, or where, to be marked as correct. Partial credit may be given if the question

word is not included in the answer.

Short Answer

Short Answer questions are similar to essay questions. The length of the answer can be

limited to a specified number of rows (lines) in the text box. Like Essay questions, Short

Note: With some question types, when partial credit is enabled, the scored points are

determined by assigning points for the correct answers and deducting points if the

wrong answer is checked or when a right answer was left unchecked.

34

Answer questions must be graded manually. The number of rows is meant as a guideline

when entering an answer, it does not impose an absolute limit on answer length.

True/False

True/False questions allow the user to choose either true or false. True and False answer

options are limited to the words True and False.

35

Appendix B: Exporting and Importing Tests

Tests can be exported and saved from Blackboard. This can save time later if you want to

reuse the same Test for another course or next semester. A Test is exported in a so-called

package file, which is a .zip file.

B1. Exporting a Test

1. Go to Control Panel > Course Tools.

2. Choose Tests, Surveys and Pools. In the screen that appears next, choose Tests. All

Tests created in the course will be shown.

3. Click the arrow behind the name of the Test to be exported. Choose Export.

4. A popup will come asking if you want to open or save the export file containing the

Test. Save the file to your computer. The format of the file is .zip (compressed).

B2. Importing a Test

1. Go to Control Panel > Course Tools.

2. Choose Tests, Surveys and Pools. In the screen that appears next, choose Tests. All

Tests created in the course will be shown.

3. Click on the button Import Test, just above the overview of all Tests.

4. Browse your computer for the .zip file of your Test.

5. Click Submit and you will be processed to a page called Test Import Complete.

B3. Deploying an Imported Test

See chapter 5.

Note: only Test packages created by Blackboard can be transferred to other courses as

long as they are in the proper format. This means that Test packages created at other

institutions, or with older versions of Blackboard might not always be successfully

imported. The exported .zip file must not be edited before it is imported.

36

Appendix C: Creating Test Questions with a Text Tool

Instead of creating questions one-by-one, as was explained in chapter 4, it is also possible to

import a batch of questions at once. This can save time when a Test has many questions of

the same type. Further, as Blackboard cannot export questions in a human-readable format,

it is best to maintain your questions outside Blackboard.

For this to succeed we need to create a flat, plain text file (.txt).

C1. Creating a text File

A flat, plain text file is a file containing only human-readable text

without any formatting. Typically, flat files are used by computer

programs to exchange information. All modern word processors have

the option to save a file as a text file, that is: in .txt format. Examples

are: Microsoft Word, WordPad, and Notepad.

Questions come in different types, like Multiple Choice, or True/False. Each question type

has a Question Code, consisting of two or three capital letters. See the list of codes. First,

type the Question Code. Next, hit the Tab key, and type the question itself. The question

type determines the amount and content of additional information.

In the case of a Multiple Choice or a Multiple Answer question, type one answer, hit

the Tab key and type either ‘Correct’ when it is a correct answer or ‘Incorrect’ when

it is not. Everything else than ‘Correct’ is treated as ‘Incorrect’. The maximum of

answers you can create is 20.

When making a True or False question, hit the Tab key and type either ‘True’ or

‘False’ (without the quotes).

An Essay Question is rather an assignment; formulate your question as such. Hit the

Tab key and enter a sample answer.

In the case of a Ordering Question enter the answers in the correct order and hit the

Tab key in between. The answers will be shown to students in random order.

A Matching question should have the answers filled in in the correct order. Hit the

Tab key in between. Blackboard will show the options randomly to students.

Although the file looks as if organized in columns, the information in each column may be of

a different type.

Question Codes

Multiple Choice – MC

Multiple Answer – MA

True or False – TF

Essay – ESS

Ordering – ORD

Matching – MAT

37

Put all the question information on one line (row). Do not use a header row. Do not use

blank lines. Do not use other keywords than Correct, Incorrect, True, and False, nor

abbreviate them.

Finally, save the file as a text file:

In your word processor go to the File menu > Save As

Change the document type to Plain Text (*.txt) or Text Document (*.txt).

C2. Uploading Questions to a Test

To use the questions and answers as saved in the plain text file, first create a New Test (see §

3.2), go to the Test Canvas (see § 4.1) and press the button Upload Questions. Browse your

computer to select the plain text file. Maintain 10 points per question. Press Submit.

More information on the format and limitations of the plain text file can be found via the

link More Help on the Upload Questions screen.

Question properties can be edited after uploading. For example, it is standard that all

questions will be uploaded with the same point value. Once in the Test Manager, this can be

changed per question.

It is also possible to upload questions to a Pool in the Pool Canvas. This works the same way

as for a Test. First make a Test and then go to the Pool Canvas. Press the button Upload

Questions.

Note. Be very careful with adding Tabs. There must be exactly one Tab in between fields,

otherwise fields will shift and an answer may be taken for a question. Using a Tab is not

the same as typing one or more spaces.