Critiquing CS Assessments from a

CS for All LensHow much do we assume a Community of Practice when we assess?How do we asses for different Communities of Practice, and different

sets of CS skills and pracice?

Mark GuzdialGeorgia Tech

Do all of these use the same CS practices and

knowledge?Would we use the same assessments to measure expertise?

• A software engineer who builds applications for end-users.

• A mathematician who works in Mathematica her whole life.

• A graphic designer who programs in JavaScript to automate Photoshop processes.

• The data scientist who scrubs data with Perl and analyzes in R.

• A chemical engineer who writes 20 new lines of MATLAB code each day, then throws them away.

• The office worker who builds Excel macros weekly for co-workers.

• The homeowner who writes home automation scripts.

• The musician who codes live in front of an audience.

CS for Everyone/All• Efforts in many countries to make

computing education available to all students.

• Do all students need or want the same CS education?

• Do they all want the same expertise? To be the same kind of practitioner?

Sociocognitive Theories of Learning• Situated Learning

(Lave & Wenger) says that students seek to join a community of practice.• They want to adopt

the practices and learn the values of those at the center of the community of practice.

Beyond a Software Development CoPMost people who program are not part of a software developer community of practice (CoP).

Student values based on perceived CoPStudents who value media development want different kinds of programming languages than those who want to be programmers.

Authenticity matters. (Shaffer & Resnick, 1999)

Concept Inventories Holger Danielsiek, Wolfgang Paul, and Jan Vahrenhold. 2012. Detecting and understanding students' misconceptions related to algorithms and data structures. In Proceedings of the 43rd ACM technical symposium on Computer Science Education (SIGCSE '12).

Critique:• “Based on expert interviews and the analysis of 400 exams we were

able to identify several core topics which are prone to error.”• Are those experts in the CoP I care about?• Were those exams by students like me?

FCS1 and SCS1Allison Elliott Tew and Mark Guzdial. 2010. Developing a validated assessment of fundamental CS1 concepts. In Proceedings of the 41st ACM technical symposium on Computer science education(SIGCSE '10).

Critique:• “Previous studies of student programming ability have raised questions

about students' ability to problem solve, read and analyze code, and understand introductory computing concepts.”• What kinds of problems do experts solve in my CoP?• Is that the kind of CS that my CoP uses? That my CoP Values?

(Re: Dorn at CHI 2010 on the use of exception handling among web designers.)

Attitude AssessmentBrian Dorn and Allison Elliott Tew. 2013. Becoming experts: measuring attitude development in introductory computer science. In Proceeding of the 44th ACM technical symposium on Computer science education (SIGCSE '13).

Critique:• “We have begun the process of examining how students perceive the

field of computer science by employing a novice-to-expert continuum framework.”• Is there only one such continuum?• Are those experts in the CoP I care about?

Blocks-Based LanguagesDavid Weintrop. 2015. Comparing Text-based, Blocks-based, and Hybrid Blocks/Text Programming Tools. In Proceedings of the eleventh annual International Conference on International Computing Education Research (ICER '15).

Critique:• Will all CoP value text over blocks?• Blocks are better than text for many tasks.• Should we be promoting blocks-based languages, despite the de-

valuation of blocks-based language by the software development CoP?

Claims• Computer scientists and professional software developers are not the

appropriate target audience when defining the target skills, practices, or attitudes when defining CS for All.

• Computing practices in many CoP are still being defined.We can’t do it. They have to develop within the communities.• Can we influence them?

• Computing is a literacy. There will be more than one kind.