universal student ratings of instruction (usri) course evaluation information session oct 2014 jean...

UNIVERSAL STUDENT RATINGS OF INSTRUCTION (USRI)Course Evaluation Information SessionOct 2014

Jean Gomes, Office of Institutional Analysis

General Faculties Council (GFC)

VP Planning and Resource Allocation

Academic Programs, Departments and Faculties

IT andEnterprise Reporting

Imagine Printing

VP Teaching and Learning

Office of Institutional Analysis

USRI

Course and student data files are loaded into ClassClimateAdmin-verified course data

file is finalized

Surveys (etc.) are generated. Imagine Printing prints and delivers forms to Admins

Class evaluations (paper or online) are conducted. Admins coordinate paper, in-class evaluations. OIA conducts online evaluations.

Admins return USRI envelopes to Imagine Printing or to USRI Team

Paper USRI surveys are scanned by Imagine Printing

Scanned and online results are checked/tested/re-scanned if necessary

Check for USRI (paper) surveys-with-missing-data

Export data from ClassClimate

USRI reporting system creates Section Rating Reports (USRI results)

Instructors/current sessionals may view results (only their courses) via My UofC portal

Online Faculty Survey results emailed to Admins by USRI Team

Students may view results via ‘Class Search’ in their Student Centre

USRI FLOWCHART

USRI Admins/Deans/Dept Heads may view results via USRI website

RELEASABLE RESULTS ONLY

USRI Course Evaluation Results Release Dates:1) Fall – January 152) Winter – May 153) Spring and Summer – September 15

What happens?- Results for the latest term can now be accessed- Official summary reports are generated and prepared for Department Heads and Deans

USRI Timeline

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb

Spring

Summer

Winter Fall

Winter

= USRI evaluations are conducted

= USRI project activities happening

Outline

I. USRI Course Evaluation Process• Validating course data • Coordinating in-class paper evaluations

— Handouts for Admins; Coversheets; Blank/generic USRI form• Faculty Surveys (student comments)• Online Surveys

— Accessed through student portal; “How Students Complete their Online Surveys”

II. Accessing USRI Results (“Section Rating Reports”)• a) Instructors• b) USRI Admins• c) Deans/Dept Heads • d) Students• Adding and deleting a person’s access to USRI Results

III. Communication Plan and Advertising

IV. Imagine Printing• Printing forms, supplies, delivery, pick-up

V. Resources - USRI Website

Why do we ask Admins to review the list of classes?

• To make any needed corrections to Student Records (BAS PeopleSoft Student Administration)

• To confirm the list of classes to be evaluated. Surveys are generated for each class on the list.

• To inform us of any classes to be added/deleted from evaluation.

• To inform us of anything needing special attention (e.g., an online survey instead of paper; instructor name change, class end date does not reflect actual course instruction, etc.)

• To produce a “cleaner file” -> reduce surveys with missing data

I. COURSE EVALUATION PROCESS Validating Course Data

Some of the criteria for selecting course data extracted (using Cognos Query) - example

Report: Instructor and Course Information Course Term: Fall 2014

Instruction Role: AL - Alternating Instructor, PI - Primary Instructor, SI - Simultaneous Instructor

Session Code: -, 1-Regular Academic, 6W1-First 6 Weeks, 6W2-Second 6 Weeks, BW-Block Week, GSP-Group Study Programs, NS-Non-Standard, PA-Multi-Term: Part A, UCQ-Qatar Nursing Program

Instruction Mode: BL-Blended Learning, EP-English Language Program, P-In Person, WW-Web-Based Instruction

Coordinating In-class Paper Evaluations

Two handouts are given to USRI Admins:1. HOW TO CONDUCT USRI PILOT IN-CLASS ADMINISTRATION

Notes for Faculty/Department USRI Administrators -- In-class Administration

At the agreed upon time, the Department USRI Administrator/designate should arrive at the classroom with the paper USRI forms, and Faculty Survey forms (if applicable).

It is important to keep the USRI forms and the Faculty Survey forms in separate envelopes. If possible, bring extra blue or black ink pens to be given out to students that do not have a black or blue ink pen. (NOTE: do

not use red ink.) The Department USRI Administrator/designate should complete each of the following:— If the course and instructor information is pre-printed on the survey, ask the instructor to verify course name, number and their

name as printed. — Ask the instructor to leave the room, and agree upon a location that the Administrator will find the instructor when the evaluation

is completed. Inform the instructor that administration will take approximately 15-20 minutes.— Read the prepared statement below explaining the purpose of the evaluation and instruct the students on how to complete the

evaluation forms. (See second handout.)— Have student volunteers in larger classes help hand the evaluation forms around the room. Students should not start filling in the

evaluation until instructions have been explained. — Collect the completed and incomplete USRI and Faculty Surveys. It is important that the two types of surveys are kept in separate

piles.— Place the completed USRI evaluations in the USRI envelope.— Place the completed Faculty Surveys in the Faculty Survey envelope. — Unused forms should be collected, but not placed, in the envelopes. Keep unused forms in your office for future use.— Seal each envelope, and have the seal signed by the last student submitting or a volunteer.— The completed forms should be securely stored by the Department USRI Administrator until completion of the survey period.— Starting Winter 2011: coversheets not delivered inside survey envelopes. Admins asked to place the coversheet into envelope

before returning survey to Imagine Printing for scanning.

2. PREPARED SCRIPT TO READ TO THE CLASS The University of Calgary is currently working on improving the administration of the Universal Student Ratings of

Instruction (USRI) with the goal to increase student response rates. For this course, the USRI is being completed in class. You may be asked to complete online evaluations in some of your

other classes. You may or may not receive an email. All students should check for online surveys via their portal.

You may use either a black or blue ink pen. DO NOT USE RED INK PLEASE. Mark responses with an X inside the box. If you need to make a correction, black-out the box with the incorrect X and then enter an X in the correct box.

The information you provide on this rating instrument is intended for use by students to aid them in selecting courses, by Deans and Department Heads to assess instructors, by instructors to assist them in improving instruction, and by the University for administrative research purposes. Therefore it is essential that you answer the questions carefully. If you have concerns about the course or instruction that are not addressed by this instrument, please speak to the Department Head or Dean.

Your participation in this evaluation process is gratefully acknowledged by the Students' Union, Graduate Students’ Association, Instructors, and Administration of the University.

When you have finished please hand in your completed surveys. If you are doing USRIs and Faculty Surveys please put them into separate piles. I will ask for a volunteer to initial the sealed envelope with all of the completed evaluations before I leave the room. I will also sign the envelope, and will then take them to be processed. Your individual evaluation will not be shown to your instructor. Your instructor will get a report on the aggregated results from this class after the final grades have been officially submitted.

Please begin to fill out the evaluation now. Thank you.

Sample of Pre-printed Coversheet (24 lb paper) Sample of Manual Coversheet

Blank/Generic USRI Form.

Since Winter 2011, the USRI questionnaire excludes the student id number question.

Some reminders:

1. Always complete the manual coversheet information:— Print identical name for instructor, as entered into BAS (PeopleSoft)— Print the course name/title, as listed in BAS (PeopleSoft)

2. In the pre-printed coversheet, the barcode contains the information. You cannot change the information manually, (e.g. crossing-out text and writing in the correction) because your changes are not recognized by the scanner.

**Do not write on the USRI questionnaire, especially over a checkbox and barcode.

Please advise USRI team if there are changes/corrections needed for this class.

3. Per GFC: When there are 4 or more multiple instructors in the course, one survey is conducted to represent all the instructors. Admin should select one of the instructors to ‘hold’ the data. - In classes with 3 or fewer instructors, each instructor is surveyed.- Combined courses with multiple instructors (error in calculating response rate)

4. USRI forms are printed on 24 lb paper- Forms cannot be photocopied. - Contact USRI team if more copies are needed. “Normal” paper will not scan.

Faculty Surveys (student comments)

• Faculty Surveys are printed on yellow-coloured normal paper as they are not scanned

• You cannot use only a Faculty Survey, USRI must also be used• Responses from online Faculty Surveys will be emailed to Admins, as

soon as possible following the release date – within 2 to 3 weeks.• ALL Faculty Survey responses will be emailed to Admins

Online Surveys

Used for classes held off campus, web-based classes except for Environmental Design, and Werklund School of

Education

Changing from paper to online by special request from Department USRI Admin

USRI team sends out a mass email and possibly reminder emails to students who have online surveys to complete.

Students access via MyUofC Portal (see following screenshots): “How students complete their online surveys”

“How students complete their online surveys”

Login to My UofC portal (top right)

Once logged in, click on the ‘Academic’ tabClick on “Evaluate my courses” which will take you to the USRI site with the list of available surveys. Click on Survey to start the survey.(See next page for example)

“How students complete their online surveys” continued

II a) Accessing USRI Results - Instructors

From MyUofC portalStep 1: Click on the “Academic” tabStep 2: Click on “View Course Evaluations”

Accessing USRI Results (Instructors) continued

Step 3: Click on “View USRI Results” tab

Step 4: Click on “Section Rating” to view the report.

II b) Accessing USRI Results - USRI Admins

1. Go to USRI web site http://www.ucalgary.ca/usri/- Click on Administrators tab

http://www.ucalgary.ca/usri/

Accessing USRI Results (USRI Admins) continued

2. Click on “Go to Results”


3. Sign in with secure id


4. Select Term, Course Career*** Admins are able to access only releasable results


5. Select parameters:

Faculty-> Department

-> Program/Subject-> Course

(Optional)

(Remember to refresh the next section after each parameter is filled out)

e.g. Faculty of Arts -> Refresh Department


6. Search for instructor name.- Scroll to search area in the bottom

half of the window- Type a name in the search window

(e.g. Hunter)- Click Options

- You will get a list of items, choose “Contains any of these keywords”


Result of the Name Search:— The Instructor name(s) and ID(s) will display in the

Results window. Select and click ‘Insert’ to bring to the choices window. Click Select all, then click ‘Finish’.


7. Choosing Section Rating ReportIn the grey box below …- All blue categories (e.g. Term, Program/Subject, Course Career, Course, Faculty, Instructor, Department) can be edited to view different reports (e.g. Winter 2014 can be changed to Spring 2010)


Your role determines which Overall Rating reports you can view:

Department Admin -> Can view only Department Level ReportFaculty Admin -> Can view Faculty Level Report, and Department Level ReportSuper user -> Can view ‘Error Reports’ (see example above)


8. Click on ‘Section Rating’ to get the report

Accessing the USRI Results (USRI Admins) continued

Section Rating Report example (page 1)


9. Click on ‘Instructor Rating’ for reports for all the courses taught by the instructor in past terms (up to the past 10 years)

II c) Accessing USRI Results - Deans and Department Heads

1. Go to the USRI web site: http://www.ucalgary.ca/usri

- Click on Administrators tab

http://www.ucalgary.ca/usri

Accessing USRI Results (Deans, Department Heads) continued

Click “Go to Results”


3. Sign in with secure id


4. Select Term, course career- Note: As of September 2013: USRI Admins can access only releasable

results


5. Optional ParametersNote: Only dept/faculties linked to your courses will display in the

window.

Department(s) ->Refresh Program/Subject(s)

->Select Program/Subject(s)

->Refresh course

-> Select course

-> Finish

Selections may be made on the following:

II d) Accessing USRI Results - Students

1. From the U of C’s website, click on MY U OF C

Accessing USRI Results (Students) continued

2. Enter UCID and password


3. Once signed in, the Student Centre should automatically open. Click on the ‘Search For Classes’ button on the right hand side of the screen


4. Enter search criteria, and once finished, click ‘Search’ at the bottom-right hand side of the page- Additional search criteria can be added to the search if wanted

E.g. Search for Psych 433 course in Winter 2014


5. Using the previous Psych 433 example, the sections are displayed here.

- If there are USRI results available for the class, a link will be displayed under the course name and number.

Click on the USRI link


6. The Usage Advisory screen will open. If you agree to the user conditions, click “I AGREE”.


7. You can choose whether you want to look at reports for your course for all the terms it was taught, or only the reports for the latest term. In this example, ‘USRI results for all terms’ is selected


8. Following the previous example, you will then see a list of all the terms in which the course you selected was taught. The USRI Reporting System contains reports since 1999. Selecting ‘Section Rating’ will give you the results for the specific class and instructor. Selecting ‘Instructor Rating’ will give you a list of all the different courses this specific instructor has taught since 1999.

II. Adding/Deleting a Person’s Access to USRI Results

A USRI Admin must contact the Office of Institutional Analysis, who will in turn submit a request to IT.

Please notify OIA:— changes in USRI Admin— changes in Dept Head, Associate Dept Heads— Changes Dean

III. Communication Plan and Advertising

USRI Admins promote USRI within their own units Course instructors Students’ Union Graduate Students’ Association Student and Enrolment Services, Residence

Services – Orientation; New Student Registration Assistance; Service Stop

UCIT (e.g., portal messages, Course Desire2Learn) University Relations (e.g., website messages) Various communications offices & media (e.g.,

NUTV)

IV. Imagine Printing

- Printing USRI and Faculty Survey forms USRI forms (white color and 24 lb special paper) Faculty Survey forms (yellow color and normal paper) Pre-generated coversheets (white and 24 lb special paper for USRI

and yellow normal paper for Faculty Survey). We do not print Faculty Survey coversheets

- Delivery: Based on the delivery list provided by OIA (updated by Admins)

USRI/Faculty Survey forms, pre-printed coversheets and envelopes (with course information labels on the envelopes) are delivered to every department/faculty.

For each class, all of the coversheets will be placed in the class’s envelope.

After the survey is done, it is the Admin’s responsibility to insert the coversheet into each envelope.

IMAGINE PRINTING SERVICES (continued)

Contact for Pick up and Delivery: Please phone Imagine Printing Services at 220-7497 to

make arrangements. Let them know your location and contact information.

Carol Fitton: [email protected]; [email protected]. Let them know your location and contact information.

Faculty Survey forms will NOT be scanned. Do not send any faculty survey forms to Printing Services. Please keep the completed forms in your own department/faculty.

V. Resources

www.ucalgary.ca/usri


Other important contacts:

Scheduling Coordinator, Timetabling, SES: to correct course data in BAS PSoft

- Barb McCutcheon, [email protected]

IT Support: problems logging into the USRI reporting system via www.ucalgary.ca/usri

Imagine Printing: deliveries/pick-ups- Carol Fitton, Production Manager, [email protected]

mailto:[email protected]


mailto:[email protected]

Thank you to all the USRI Admins for your ongoing work on the USRI course evaluations!

universal student ratings of instruction (usri) course evaluation information session oct 2014 jean...

Documents

usri teampaper usri

usri paper surveys

usri teamstudents

usri envelopes

usri resultsiii

online results

resources usri website

student data files