#BbWorld14#BbWorld14

Discussion Tracking with BIRT:A Greek Tragedy in Two Acts

Jeffrey Berman and Michael Shelmet

Drexel University

BbWorld 2014

#BbWorld142

Interaction

• QR Codes provided for links

• Interactive responses (maybe)

• Have your mobile devices ready– Or use your Analog Response Mechanism

#BbWorld14

Rejected Presentation Titles

• Discussion Tracking with BIRT and the Men Who Cheat on Them

• Discussion Tracking with BIRT – FREE BEER!

• Happy BIRThday: Our First Reporting Building Block

• BIRT Reporting Or: How We Learned to Stop Hating Built-in Reports and Build Our Own

• BA-BA-BA-BIRT BIRT BIRT, BIRT is the Word (for Course Reports)

• How Do We Spell Course Reports? BIRT

#BbWorld14

Playbill

Act I• Background

• Issues with Learn reports

• Utilizing BIRT

• Obtain our B2

Act II• Your Needs

– Open forum• We’re Roman now!?!?

• Our ideas

• Tragic death scene

#BbWorld14

Act I

• Background

• Issues with Learn reports

• Utilizing BIRT

• Obtain our B2

#BbWorld14

Background Information

• Drexel University FTEs 25,000– ASP model with partner institutions– Additional 10,000 FTEs

• Self-hosted Learn 9.1 SP14– Oracle database, Linux application servers

• Previously Bb Vista self-hosted, Academic Suite manage-hosted– Merged in Learn Summer 2012

• Banner integration with all courses having shell

#BbWorld14

Problems with Existing Course Reports

• Require items be set up as tracked

• Don’t often provide relevant data– Number of clicks isn’t entirely meaningful

• Not all content items are trackable– Files attached to Items (content type)

• Tend to be slow

• Far less information than Bb Vista tracking

• Training session on course reports

#BbWorld14

Bb Vista Discussion Tracking

• Bb Vista provided tracking for discussions– Messages Posted– Total Messages Read

• Lacked Unique Messages Read– Far more informative than total

• Wrote a db query to pull Unique Message Read

#BbWorld14

Front End Discussion Tracking (GUI)

• Through performance dashboard– Drill down to the number of forums posted in, messages per forum

• Through graded forums– Report on the number of posts made for the student in question

• Through the discussion boards themselves– Number of times a message has been viewed– Not broken down by user

• User Activity in Forums– Tracks clicks, but mostly meaningless

#BbWorld14

Back End Discussion Tracking (Database)

• Through queries– Messages posted per student– Unique messages read per student– Total message read per student

• Either– Across all course-wide discussion boards– For an individual forum

• Run on-demand by instructor request– Required Jeffrey to run– Wasn’t well advertised, by design

#BbWorld14

Addressing the Issues

• Whined to Blackboard– Provided detailed notes about how and why the data is useful– Provided the queries we used to calculate our data– Provided mockups of what we wanted to see

#BbWorld14

Addressing the Issues

• Advice from Bb– Build it yourself using BIRT (Business Intelligence and Reporting Tools)

• http://projects.oscelot.org/gf/download/docmanfileversion/944/5275/CustomSystemReportingInLearn9-10-12-2013.docx– Documentation provided to us by Blackboard

• https://qmu.adobeconnect.com/_a887358170/p72jn2mt1s7/?launcher=false&fcsContent=true&pbMode=normal– Video walkthrough of setting up BIRT

#BbWorld14

Addressing the Issues

Documentation Video Walkthrough

#BbWorld14

Discussion Tracking B2

#BbWorld14

#BbWorld14

#BbWorld14

BIRT Requirements

• Understanding of the Blackboard Database– Open Document Schema– Knowledge of SQL

• Eclipse/Blackboard Building Block format– Does not require knowledge of Java– Programming background recommended

• 3 levels of reporting– System, Course and Content

#BbWorld14

Deployment

• Early feedback– Sent HTML output to stakeholders

• Revisions– One rewrite of all queries to address efficiency

• No issues in Production– Deployed late March to Drexel and 3 partner institutions– Blog post to announce roll out

• Drexel Usage– As of June 25th (90 days post launch) – 39 instructors– 56 courses– 110 instances

#BbWorld14

Feedback

• Both unsolicited and via an informal request– Used tracking data to target users who had run the report

• Total number of posts per forum/course could be displayed

• Display data based on New Threads Created vs Replies Posted

• Display data based on threads, in addition to per forum

• Display data by student across forums– Perhaps as an option before the report runs

• Show group discussion forums as well as course-wide– Perhaps built as a new report with the B2

#BbWorld14

Known Issues

• Blackboard “bug” in user_msg_state– Creating a new thread doesn’t count as a read message

• Creating a reply does

– Logically “broken”

• Option to save to Content Collection is broken– Affects all Learn course reports– October 2014 fix

#BbWorld14

Other Considerations

• Reading a message isn’t indicative of comprehension– Nor is the number of posted messages

• Reading outside of the system isn’t tracked– Discussion subscriptions

• Possibly serve as a retention/engagement tool

• Better to have the data to lead to the next question

#BbWorld14

Getting Our Discussion Tracking B2

• Join our Course Sites– https://www.coursesites.com/s/_DiscussionTrackingB2– http://bit.ly/discussiontracking

• Download the B2 from the course

• Use the discussion board to makesuggestions/report bugs

#BbWorld14

Act II

• Your Needs– Open forum

• Our ideas

#BbWorld14

What Are Your Needs?

• Administrators– What queries are you running at the system level?– What queries are you running at the course level?

• Instructors– What information do you wish you had?

• Everyone else– Why are you here?– Love of Greek Tragedy?

• Is your attendance ironic?

#BbWorld14

Adaptive Release Rules (Course)

• Build a Vista-like Power View

#BbWorld14

Full Activity Log (Course)

• Single student clicks through course

#BbWorld14

Student Last Access across Term (System)

• For active courses

#BbWorld14

Course Items by Type (Course/System)

• Specifically to determine Building Block Usage

#BbWorld14

Printable Course Map (System)

• Course Menu DIFF – Show differences between course menus across the “same” course

#BbWorld14

Contact

• Jeffrey Berman– 215-895-4925– jtb77@drexel.edu

• Michael Shelmet– 215-895-1042– mshelmet@drexel.edu

• Course Sites– https://www.coursesites.com/s/_DiscussionTrackingB2– http://bit.ly/discussiontracking