Exploring Dynamic Processes of STEM Education Reform How an Institution’s Contextual, Cultural, and Cognitive Factors Interacted with the SCALE

InterventionMatthew Tadashi Hora, M.A.A., Wisconsin Center for Education Research, University of Wisconsin-Madison

Literature CitedQuinn, N. (ed) (2005). Finding Culture in Talk: A Collection of Methods. Palgrave MacMillan: New York.Resnick et al. (2007). A Framework for Effective Management of School System Performance. Yearbook of the National Society for the Study of Education, 106 (1):155-185.Shore, B. (1996). Culture in Mind: Cognition, culture, and the problem of meaning. Oxford University Press: New York.Spillane, J.P, Reiser, B.J., and Reimer, T. (2002). Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research, 72(3):387-431.

I. BackgroundReform in higher education is “glacial”Despite significant investments in STEM education reform in Institutions of Higher Education (IHE), affecting substantive changes to these organizations has proven challenging. Reasons given for the limited adoption of reforms include the complexity and non-linear dynamics among the various factors that influence institutional life. Complicating matters, these factors operate at and among multiple levels including individual, group, and institutional levels. As a result, how an IHE’s many constituent parts interact and influence reform initiatives are poorly understood and largely remains a “black box.”

Cognition, culture, & complex systemsReform researchers are increasingly focusing on the role that cognition and its relationship to the cultural and structural environment play in influencing how reform messages are interpreted and adopted by actors (Spillane et al, 2002). Yet empirical studies that explore the relationships among individual cognition, culture, and institutional systems are uncommon. Cognitive anthropology offers potential by positing that individual mental models are composed of schema, some of which can be explicitly linked to instantiated cultural forms (i.e., cultural schema), and environmental factors (Shore, 1996).

Implications for evaluationIntegrating this theoretical framework into an evaluation is the goal of this research, which is based on an evaluation of the NSF-funded System-wide Change for all Learners and Educators (SCALE) project. Such an approach addresses concerns that many evaluations of educational reforms rely on simplistic linear logic models, or over-emphasize the outputs of education as opposed to its processes (Resnick et al., 2006).

II. MethodologyThis exploratory research is a qualitative case study that uses a repeated cross-sectional design, with interview and document-based data collected in June 2006 and June 2007 at a single comprehensive IHE. Non-random sampling procedures were used to identify respondents for a total of 34 interviews with 25 unique individuals. The interview protocol elicited combinations of factors that influenced instruction and collaboration. Analytic procedures included using a structured coding scheme comprised of 6 categories (see Fig. 1), casual network analyses, and mental models analysis using a variation of the narrative analysis technique of Quinn (2005). Findings were triangulated with other data and member-checked.

III. Findings: Inside the black box of organizational process

Factors at multiple levels interact to inhibit and support SCALE Respondents identified 35 factors (see Fig. 1) that inhibited and/or supported the SCALE project’s primary goals: to improve undergraduate STEM instruction and encourage inter-disciplinary collaboration. Many of these factors are inter-related, i.e., national trends in educational reform (1a), which led to local pedagogical reforms (2j), which in turn cultivated a cohort of STEM educators (3a). Key factors that inhibited SCALE goals included a demanding workload (2h, 5a) and pervasive values of scientific legitimacy (4b). Key factors that supported SCALE goals included supportive leadership (2c) and a close alignment with local K-12 districts (1e). The cultural complex of scientific credibility is a considerable barrier to reformAnthropologists sometimes consider clusters of practices (e.g., rituals, artifacts) in a specific geographic region to be “cultural complexes.” A set of particularly resilient values (i.e., cultural schema) and the contextual factors that supported them can considered a cultural complex of scientific credibility. This complex posed a barrier to achieving SCALE goals. The focus on research over teaching (4a), equation of success with research accomplishments (4b), and denigration of the “soft” sciences (4c), all of which STEM faculty acquire through doctoral training in research universities (1c), is locally reinforced by tenure policies (2g) and the institution’s drive to increase its external funding (1b, 3c) and status in the field of higher education (3e). This combination of cultural schema and contextual determinants served to limit the pool of potential participants in SCALE, and created a strong and for some untenable tension between the institutional mission and disciplinary culture.

IV. Findings: Mental Models in Action

Sense-making is governed by situated mental modelsA STEM faculty who participated in SCALE was closely analyzed to illustrate the non-linearity and context dependent nature of the sense-making process. This individual’s mental model for STEM education reform (see Fig. 2) was composed of cultural and idiosyncratic schema that were related to contextual factors. The stimulus in this case was the invitation to participate in SCALE, which immediately triggered a consideration about workload and tenure (2g), and the reality that departmental colleagues valued basic research (4b) more than pedagogy-related activities (4d). This individual’s personal experience as a K-12 teacher, SCALE’s provision of release time (3c) and supportive colleagues (3a) mitigated these concerns and led to their participating in SCALE.

V. Conclusions & RecommendationsUnderstanding the context is critical to strategic reformThis research demonstrates how to “trace” the relationships among context, culture, and cognition in relation to a STEM education reform initiative. It is critical for policy makers and practitioners to utilize this knowledge when designing and implementing reforms. In particular, this research suggests that, given the structural and cultural challenges with engaging all STEM faculty in pedagogical reform, the NSF and other reform agents should consider a more strategic approach and more intentionally recruit academic staff, education faculty, and STEM faculty actively engaged in reform efforts.

Oversimplifying institutional culture(s) masks major issuesThe cultural complex of scientific credibility is comprised of different cultural schema that are differentially internalized and expressed by individual faculty. An oversimplified notion of institutional culture serves to mask these important issues, and thus fails to provide actionable knowledge for practitioners.

Future directionsFuture research will employ a mixed methods approach to develop a replicable evaluation design that can be used to more effectively diagnose the specific “moving parts” of educational systems, establish baselines for evaluation, and provide actionable knowledge for practitioners.