presentation pick a card - newman 17-04-13 - final
DESCRIPTION
Open card sort as a research toolTRANSCRIPT
Pick a CardDiscovering Student’s Perceptions
Andrew CsizmadiaWednesday 17th April 2013
Purpose
• Discover trainee ICT Computer Science teachers’ perception of Computational Thinking
Relevance
“We need to bring computational thinking
into our schools.”
(DfE, 2013a)
“Computational thinking is a fundamental skill for everyone, not just for computer
scientists. To reading, writing and arithmetic, we should add computational thinking to every child’s analytical ability.“
(Jeannette Wing, 2006)
“…change the world through computational thinking.”
(DfE, 2013b)
What is Card Sorting?• Qualitative participatory
design technique, used to explore how participants group items together into categories and relate concepts to one another (Martin & Hanington, 2012)
• Tool to determine participant’s mental model of grouping items (Ross, 2011)
• Generative method (Nielsen, 2004) using open card sort
• Evaluative method (Ross, 2011) using closed card sort
• Generate a category tree or folksonomy
Approach – Card Sorting
Advantages Disadvantages
Simple Content-Centric technique
Cheap Results may vary
Quick to Execute Analysis – time consuming
Established May capture “surface” characteristics only
Involves users
Provides a good foundation
(Source: Spencer & Warfel, 2004)
Design/Methodology/Approach
Card Selection• 104 Computational Thinking
related cards identified following desk based literature review
Recommended Time• Allow 30 minutes for each
multiple of 50 cards (Martin & Hanington, 2012)
Recommended Sample Size• Between 15 (Nielsen, 2004)
and 30 (Tullis & Wood, 2004) provides a correlation between 0.90 and 0.95 respectively
Design/Methodology/Approach
Card Selection• 104 Computational Thinking
related cards identified following desk based literature review
Recommended Time• Allow 30 minutes for each
multiple of 50 cards (Martin & Hanington, 2012)
Recommended Sample Size• Between 15 (Nielsen, 2004)
and 30 (Tullis & Wood, 2004) provides a correlation between 0.90 and 0.95 respectively
Design/Methodology/Approach
1 2 3
Analysis of Data
Process
Know Your Data
Focus the Analysis
Categorize Information
Identify Patterns Interpretation
(Source: Taylor-Powell & Renner, 2010)
• Identify themes
• Organize them into coherent emergent categories
• Larger categories
• Relative importance
• Relationships
Analysis of Data
K-Mean Analysis
How often a card is placed in a category
Single-linked dendrogram
Analysis of Data
Average linkage dendrogram
Complete linkage dendrogram
Dissimilar matrix
Analysis of Data
Two-dimensional multidimensional scaling analysis
Co-occurrence matrix
Originality/ValueThe tool provides an original approach to capturing trainee teachers’ perceptions of Computational Thinking.
ReferencesBerg, E., A. (1948) A Simple Objective Technique for Measuring Flexibility in Thinking. The Journal of General Psychology. 39(52) pp.15-22.Coxon, A. and MacMillan, P. (1999) Sorting Data: Collection and Analysis. Thousand Oaks, CA: Sage Publications.Department for Education (2013a) Computer Science to be included in the EBacc [Online]. Available at: http://www.education.gov.uk/inthenews/inthenews/a00221085/ebacccompsci (Accessed: 15 April 2013).Department for Education (2013b) Computing Programmes of study for key stages 1-4 [Online]. Available at: http://media.education.gov.uk/assets/files/pdf/c/computing%2004-02-13_001.pdf (Accessed: 15 April 2013).Martin, B. and Hanington, B. (2012) 100 ways to Research Complex Problems, Develop Innovative Ideas and Design Effective Solutions. Beverley, MA: Rockport Publications, pp.26-27.Nawaz, A. (2012). A Comparison of Card-sorting Analysis Methods. The 10th Asia Pacific Conference on Computer Human Interaction (APCHI2012).Nielsen, J. (2004) Card Sorting How Many Users to Test? [Online]. Available at: http://www.nngroup.com/articles/card-sorting-how-many-users-to-test/ (Accessed: 15 April 2013).Nielsen, J. (1995) Usability Testing for the 1995 Sun Microsystems' Website. [Online]. Available at: http://www.nngroup.com/articles/card-sorting-how-many-users-to-test/ (Accessed: 15 April 2013).Ross, J. (2011) Comparing User Research Methods for Information Architecture. [Online]. Available at: http://home.comcast.net/~tomtullis/publications/UPA2004CardSorting.pdf (Accessed: 15 April 2013).Spencer, D. (2009) Card Sorting: Designing Usable Categories. New York: Rosenfeld Media.Spencer, D. and Warfel, T. (2004) Card sorting: A Definitive Guide. [Online]. Available at: http://boxesandarrows.com/card-sorting-a-definitive-guide/ (Accessed: 15 April 2013).Taylor-Powell, E. and Renner, M. (2010) Analyzing Qualitative Data. Madison: University of Winconsin. [Online]. Available at: http://learningstore.uwex.edu/assets/pdfs/g3658-12.pdf (Accessed: 15 April 2013).Tullis, T. and Wood, L. (2004) How many users are enough for a card-sorting study? In Proceedings UPA'2004, Minneapolis, MN. Available at: http://home.comcast.net/~tomtullis/publications/UPA2004CardSorting.pdf (Accessed: 15 April 2013).Wing, J.,M. (2006) Computation Thinking. Communication of the ACM. 49(3) pp. 33-35 [Online]. Available at: http://www.cs.cmu.edu/afs/cs/usr/wing/www/publications/Wing06.pdf (Accessed: 15 April 2013).
ReferencesFincher, S. and Teneberg, J. (2005) Making sense of card sorting data. Expert Systems, 22(32), pp.89-93.Hannah, S. (2005) Sorting Out Card Sorting: Comparing Methods for Information Architects, Usability Specialists and Other Practitioners. Available at: https://scholarsbank.uoregon.edu/xmlui/bitstream/handle/1794/7818/2005-hannah.pdf?sequence=1 (Accessed: 15 April 2013).Packer, T., L., Boshoff, K. and DeJonge, D. (2008) Development of the Activity Card Sort — Australia. Australian Occupational Therapy Journal, 55(3), pp.199-206.Rugg, G. and McGeorge, P. (1997) The sorting techniques: a tutorial paper on card sorts, picture sorts and item sorts. Expert Systems, 14(2).Santos, G., J. (2006) Card sort technique as a qualitative substitute for quantitative exploratory factor analysis. Corporate Communications: An International Journal, 11(3), pp.288 – 302.Tetting, D. (1988) Q-Sort Update. Western Journal of Nursing Research, 10(6), pp.757-765.Tullis, T. and Albert, B. (2008) Measuring the User Experience: Collecting, Analyzing and Presenting Usability Metrics. New York: Elsevier: Morgan Kaufmann.