report specifications for iitbombayx · 1. disk usage of course contents 2. amount of data being...

Report Specifications for IITBombayX

20th March 2015- Aparna Pansare

Table of Contents1 Overview...........................................................................................................................22 References.........................................................................................................................23 Differences between proposed reporting platform and edX Insights...............................24 Admin reports ...................................................................................................................35 Course reports...................................................................................................................56 Student reports..................................................................................................................77 Feedback reports...............................................................................................................98 Lists.................................................................................................................................119 On demand reports/Ad-hoc reports.................................................................................1310 Cohort related reports...................................................................................................1511 Trend reports.................................................................................................................17 Appendix A.......................................................................................................................22

Report Specification v 1.0IITBombayX – Confidential of 23IITBx_report_spec_v0.3d.doc

1 OverviewThis document specifies the reporting requirements of a MOOC platform running online courses, in particular IITBombayX. It briefly talks about the analytics feature provides by edX in their latest Birch release for the purpose of making a comparison and to highlight the differences between what is already present in the edX platform versus what is proposed to be built. The analytics being offered in edX are based on Hadoop platform. The document also proposes to use other suitable technologies for building reports instead of Hadoop, in order to reduce the maintenance and setup expenses.

2 References

Note on edX(Birch) Insights in Appendix A 22

3 Differences between proposed reporting platform and edX Insights

The proposed system will differ from edX(Birch) Insights in the following ways:

1. It will have more comprehensive set of reports

2. It will have reports catering to needs of specific users/roles

3. It will incorporate the learnings and feedback from faculty conducting the online courses

4. It will use non-Hadoop technologies in order to reduce maintenance and setup effort

5. It will attempt to provide a feedback to the faculty with meaningful inputs to further improve the quality of courses


4 Admin reports

General InformationReport Name Admin reportsSpecification Version #Creation DatePrepared ByBanner ReleaseBusiness Area To monitor health of the system and provide early alerts in terms

of server sizing, bandwith usage etc.Report Type Scheduled report & Exception reports

Frequency: Daily for the previous day's data Reporting Tool & Version

DescriptionScheduled reports:

1. Disk usage of course contents2. Amount of data being stored on external server such as Amazon etc.3. Number of users registered for a course4. Number of registrations per day5. Number of emails sent per day, course-wise

Exception reports:1. Disk usage for a course exceeds certain threshold2. Total registrations per course reaches a threshold

How often is this report to be run and by whomSystem administrators : need based

Is this a Standard Report or Ad Hoc ReportStandard

Source Data

Element Name SourceTBD


Grouping/Order By

Sub Totals/Summary

Filters

View/Table Column/Data Element ValuesInclude orExclude

Input Parameters

Parameter

Optional or Required(O or R)

DefaultValue

Single or Multi-value

(S or M)Wildcards

(Y or N)

Note: The default prompt is the parameter name.

Data Elements

Name View/Table View/Table Column

Notes

Attached Documents


5 Course reports

General InformationReport Name Course reportsSpecification Version #Creation DatePrepared ByBanner ReleaseBusiness Area Course-level report for monitoring the student involvement and do

course-correction, if neededReport Type Scheduled/Demand report

Frequency : DailyReporting Tool & Version

Description1. Discussion Forum activity: How many comments in discussion forum for each course2. Grades: What is the average grade, students above average/below average3. Number of users registered for a course, Summary of level of engagement of

students(% who completed, maximum number completed till which section)4. Section level reports : which section has most failures, which section is most/least

attended

How often is this report to be run and by whomFaculty : need based


Source Data


Grouping/Order By


Sub Totals/Summary

Filters


Input Parameters

Parameter


DefaultValue


(S or M)Wildcards

(Y or N)


Data Elements


Notes

Attached Documents


6 Student reports

General InformationReport Name Student reportsSpecification Version #Creation DatePrepared ByBanner ReleaseBusiness Area Student-level report for focussing on special needs of certain

studentsReport Type Demand report

Frequency : As neededReporting Tool & Version

Description1. all courses attended2. Grades3. Level of engagement4. time taken for answering quizzes



Source Data


Grouping/Order By


Sub Totals/Summary

Filters


Input Parameters

Parameter


DefaultValue


(S or M)Wildcards

(Y or N)


Data Elements


Notes

Attached Documents


7 Feedback reports

General InformationReport Name Feedback reportsSpecification Version #Creation DatePrepared ByBanner ReleaseBusiness Area Monitoring results where student response was explicitly soughtReport Type Demand report

Frequency : Need basedReporting Tool & Version

Description1. Number of queries asked per course2. Level of collaboration in discussion forum/wiki3. Monitor student responses and feedback: most difficult question, easiest questions4. Survey results: Create a survey (Xblock?) and analyse the results



Source Data


Grouping/Order By

Sub Totals/Summary


Filters


Input Parameters

Parameter


DefaultValue


(S or M)Wildcards

(Y or N)


Data Elements


Notes

Attached Documents


8 Lists

General InformationReport Name ListsSpecification Version #Creation DatePrepared ByBanner ReleaseBusiness Area Simple reports generated on a daily basis; output being just the

contents of specific tables with sorting, searching Report Type Demand report


Description1. All students registered to a course2. All courses of a particular subject3. All students of a particular subject4. All passed student, all failures

How often is this report to be run and by whomCourse staff : need based


Source Data


Grouping/Order By


Sub Totals/Summary

Filters


Input Parameters

Parameter


DefaultValue


(S or M)Wildcards

(Y or N)


Data Elements


Notes

Attached Documents


9 On demand reports/Ad-hoc reports

General InformationReport Name On demand reports/Ad-hoc reportsSpecification Version #Creation DatePrepared ByBanner ReleaseBusiness Area A facility to specific users to contruct and fire queries; may modify

the query to give access only to specific courses etc. depending on the user credentials

Report Type Demand report Frequency : Need based

Reporting Tool & Version

DescriptionType in a query and get the resultUser can fire queries on a restricted set of tables

How often is this report to be run and by whomCourse staff : need based


Source Data


Grouping/Order By

Sub Totals/Summary


Filters


Input Parameters

Parameter


DefaultValue


(S or M)Wildcards

(Y or N)


Data Elements


Notes

Attached Documents


10 Cohort related reports

General InformationReport Name Cohort related reportsSpecification Version #Creation DatePrepared ByBanner ReleaseBusiness Area To measure effectiveness of blended model;comparison between

different centers; productivity of mentorsReport Type Demand report


Description1. How many students in each cohort2. How many posts per cohort3. How many unanswered queries4. Effectiveness comparison of various cohorts for a course

How often is this report to be run and by whomTAs : need based


Source Data


Grouping/Order By


Sub Totals/Summary

Filters


Input Parameters

Parameter


DefaultValue


(S or M)Wildcards

(Y or N)


Data Elements


Notes

Attached Documents


11 Trend reports

General InformationReport Name Trend reportsSpecification Version #Creation DatePrepared ByBanner ReleaseBusiness Area Longer term feedback to the program for program level course-

correctionReport Type Key-indicator report


Description1. Courses with dwindling student participation2. Courses with continuously failing students

How often is this report to be run and by whomProgram Stakeholders : need basedFaculty : need based


Source Data


Grouping/Order By

Sub Totals/Summary


Filters


Input Parameters

Parameter


DefaultValue


(S or M)Wildcards

(Y or N)


Data Elements


Notes

Attached Documents


Revision HistoryVersion # Date Author Note(s)


Approval to ProceedThe signatures below indicate this report specification meets the approval of the undersigned and thereby grants SunGard HE the approval to proceed with the work outlined in the document. Both the Functional User and IT Representative will sign off on this specification.

Please fax the signed document to

Name: DateTitle:

Name: DateTitle:

Name: DateTitle:

Name: DateTitle:


Appendix A

Note on EdX Insights

IntroductionedX Insights is a Hadoop based analytics and reporting platform made available in edX Birch release.

RolesOnly those course team members who have the Course Staff or Instructor role.

TypesTypes of analytics are:

• Enrollment Activity • Daily Student Enrollment Chart• Enrollment Metric• Change in Last Week Metric• Verified Enrollment Metric• Change in Verified Enrollments Last Week Metric• Enrollment Over Time Report• Age (self-reported age, median, age band, • Educational background• Gender (self-reported gender chart, breakdown over time) etc.

• Enrollment Geography • Location : geographic distribution map, total countries

represented, Top/second country, breakdown

• Engagement with Course Content • weekly engagement chart, active last week, watched video last

week, tried problem last week, content engagement breakdown,

• Graded Content Submissions • Ungraded Problem Submissions


TechnologyThe edX(Birch) Insights uses following technologies:

• Hadoop version 1.0.3 or higher • Hive version 0.11.0.2 or higher • Sqoop version 1.4.5 • Python 2.7 • Either Debian version 6.0 or higher, or Ubuntu version 12.04 or higher. • A MySQL server version 5.6 or higher

ArchitectureThe Learning Management System (LMS) gathers data about student activity. This data is aggregated by the edX Analytics Pipeline.

The edX Analytics Pipeline reads the MySQL database used by the LMS as well as the tracking log files produced by the LMS. The data is processed and the resulting summary data is published to the result store. The Scheduler schedules the execution of data computation tasks which update parts of the result store. The result store is a MySQL database. The aggregated data is exposed by the edX Analytics Data API.

EdX Insights reads data from the APIs and presents it to the team members.


http://hadoop.apache.org/

http://sqoop.apache.org/

https://hive.apache.org/