database and data mining tools for electronic teamwork assessment tool e-tat

Database and Data Mining Tools for Electronic

Teamwork Assessment Tool e-TAT

Masters Project Oral DefenseBy

Ravi Soni

Graduate Student, Department of Computer ScienceSan Francisco State University

Project Advisors/Committee Members

Dr. Dragutin Petkovic, Gary Thompson Dr. Marguerite Murphy

Date: September 7th 2010, CCLS Page 1

Outline Problem Statement

Electronic Teamwork Assessment Tool (e-TAT)

e-TAT project

My Contributions to e-TAT

Use cases for Data Mining Tools

Requirements for DB, Access SW, DM Tools

Architecture

Implementation Issues

SE Methods Used

Testing

Conclusion

Future Work Page 2

Problem Statement• Teamwork assessment is the measurement and analysis of team participation metrics during collaborative Software Development process

• Important for the instructors to know how well students apply, exhibit and subjectively experience the teamwork and soft skills during the development of a class project

• Difficult to measure

Page 3

Background• Teamwork assessment was done manually by using

classical approach (assessed by exams)

• Was focused on “SE concepts, practices and theory” and not on how students apply them

• No method to automatically measure actual teamwork participation, usage of tools etc.

• Hence there was a need of an automated system, Electronic Teamwork Assessment Tool, e-TAT, to assess these soft skills

Page 4

Electronic Teamwork Assessment Tool, e-TATCombines survey tool, annotation and Data Mining

Tools

Developed by a team of four members at SFSU

Designed to be consistent with different platforms

Easy to use

Database Schema to allow Data Mining

Page 5

E-TAT Project GoalsCollecting participation metrics from different

platforms (Google groups, Google code) automatically

Data Mining tools for analysis of the collected data

Survey tool for conducting weekly surveys

Storing instructor annotations for each group

Page 5

My Contributions to e-TATData Mining(DM) tools for analyzing stored data.

DM tools include:

Total posts for groups Correlation Graph Monitor groups Sanitize Database

Design of the database schema

Design and development of Data Access Software

More info: [1] Page 5

Use cases for Data Mining Tools Instructor tries to assess groups in a class for

grading

In real time, the instructor uses e-TAT Data Mining Tools to track groups.

Instructor wants to check all groups for the number of posts they have made till now

Notes the group progress graph for the semester and grades individual groups

Page 5

Requirements for Data Mining Tools

• Customizable graph to display the total posts for groups and correlation between post types

• Version 1 of “Total Posts for Groups” tool displayed graph for ‘forum’ posts (non-customizable)

• Post type, time frame and granularity options added during requirements gathering sessions

• Monitor groups daily, weekly, monthly. Send email to instructor if posts fall below limit

Page 7

Requirements for Data Mining Tools

• Database sanitization to transform sensitive data in the database into sanitized data to be used for further analysis of data.

Page 7

Requirements for Database

• Identification of data items to collect

• Identification of tables and relationships between them

• Integration of tables containing different types of data (User data, survey data, project data, instructors’ annotations)

Page 7

Requirements for DB access softwareProviding layer at the database to store and

retrieve data objects from the database

Class to handle the database operations like connect, select, insert and update

Page 8

Architecture and Design

Page 9Figure: Overall architecture of e-TAT and our contributions (shaded)

Implementation issues

Page 9

No privileges for changing crontab using HTTP request

To protect security of the server, we did not change system privileges for a particular user.

We designed a temporary cron file (cron_replace.txt) containing list of schedules

Implementation issues

Page 9Figure: Privileges issue handled in “Monitor Groups” tool

Software Engineering Methods used

Page 11

Agile Software Development

SCRUM Meetings

User Centered Design (UCD) for User Interface

Testing Methods

Page 13

• Black Box and White Box testing

• User Centered Approach for the UI

• Tested with original data available from ppm java.net website [3]

• Unoriginal data available using populate database scripts

Conclusions

Page 14

• Implemented Data Mining tools for e-TAT

• Designed and developed data access software for e-TAT

• Designed database schema for e-TAT

Future Work

Page 15

• Ability to further mine the sanitized data to find interesting patterns of group participations metrics over the semester

• Designed database schema to store data from different courses

• Analyze data integrated from different universities e.g. SFSU, Fulda university and Florida

Questions?

Page 16

References [1] D. Petkovic, G. Thompson, R. Todtenhoefer, S. Huang, B.

Levine, S. Parab, G. Singh, R. Soni, S. Shrestha : “Work in Progress – e-TAT: Online Tool for Teamwork and “Soft Skills” Assessment in Software Engineering Education”, submitted to Frontiers in Education FIE 2010 (undergoing final review)

[2] Dragutin Petkovic, Rainer Todtenhoefer, Gary Thompson: “Assessment and Comparison of Local and Global SW Engineering Practices in a Classroom Setting”, ITiCSE’08, June 30-July02, Madrid, Spain

[3] Software for Tracking and Analyzing Team Synergy [STATS], Software Engineering (Dr. Dragutin Petkovic and Gary Thompson), Fall 2008 SFSU, https://ppm-6.dev.java.net/

[4] M. Al-Yahya: “Using Wikis to Support teamwork Skills in Software Engineering Courses”, 22nd Conf. on Software Engineering Education and Training, 2009

Page 25

References [5] Orit Hazzan, Y. D. (2008). Agile Software

Engineering. Springer. [6] Google Chart Tools/ Image Charts. (2010).

Retrieved from http://code.google.com/apis/chart/ [7] Bulmer, M. (1979). Principles of Statistics. Oxford:

Dover Publications, New York. [8] Pressman, R. (2005). Software Engineering: A

Practitioner's Approach. Mc Graw Hill.

Page 26

References [9] "Software Engineering 2004": Curriculum Guidelines for

Undergraduate Degree Programs in Software Engineering, The joint Task Force on computing Curricula, IEEE Computer Society, Association for Computing Machinery, August 2004.

[10] Aspray W., Mayadas F., Vardi M.Y., Editors: “Globalization and Offshoring of Software, A Report of the ACM Job Migration Task Force.” ACM 2006, http://www.acm.org/globalizationreport/

[11] Atlee, J.; leBlanc Jr., R.; Lethbridge, T.; Sobel, A.; Thompson, J.: “Reflections of Software Engineering 2004, the ACM/IEEE-CS Guidelines for Undergraduate Programs in Software Engineering”, P. Inverardi, M. Jazayeri (Eds.)., ICSE 2005 Educational Track, LNCS 4309, pp11-27, 2006, Springer Verlag, Berlin Heidelberg 2006

[12] Charette, R. N: “Why Software Fails.” IEEE Spectrum, September 2005, pp. 42.

Page 27

References [13] Damian, Daniela; Hadwin, Allyson; and Al-Ani, Ban: “Instructional

Design and Assessment Strategies for Teaching Global Software Development: A Framework.” Proceedings of the 28th international conference on Software Engineering (ICSE 2006) Shanghai, China, pp.: 685 – 690, 2006.

[14] Sebern, M.: "The Software Development Laboratory: Incorporating Industrial Practice in an Academic Environment", Proc. 15th Conf. Software Engineering Education and Training (CSEE&T 2002) IEEE CS Press, Los Alamitos, Calif., 2002, pp. 118-127.

[15] K. Garg, V. Varma: “Case Studies as Assessment Tools in Software Engineering Classrooms”, 22nd Conf. on Software Engineering Education and Training, 2009

[16] Host, M.: “Introducing Empirical Software Engineering Methods in Education.” Proc. Of the 15th Conf. on Software Engineering Education and Training (CSEET’02), 2002.

[17] Huffman-Hayes, J.: “Energizing Software Engineering Education thought Real World Projects as Experimental Studies.” Proc. Of the 15th Conf. on Software Engineering Education and Training (CSEET’02), 2002.

Page 28

Thank You!

Page 17