the development and validation of a primary science curriculum delivery evaluation questionnaire

This article was downloaded by: [University of Sunderland]On: 20 December 2014, At: 04:30Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of ScienceEducationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/tsed20

The development and validation of aprimary science curriculum deliveryevaluation questionnaireBrian Lewthwaite & Darrell Fishera Faculty of Education , University of Manitoba , Winnipeg ,Manitoba , Canada , R3J 2N2 E-mail:b SMEC , Curtin University of Technology , Bentley Campus, Perth ,AustraliaPublished online: 15 Jun 2012.

To cite this article: Brian Lewthwaite & Darrell Fisher (2005) The development and validation ofa primary science curriculum delivery evaluation questionnaire, International Journal of ScienceEducation, 27:5, 593-606, DOI: 10.1080/0950069042000230758

To link to this article: http://dx.doi.org/10.1080/0950069042000230758

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &

http://www.tandfonline.com/loi/tsed20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/0950069042000230758

http://dx.doi.org/10.1080/0950069042000230758

Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

INT. J. SCI. EDUC., 15 APRIL 2005, VOL. 27, NO. 5, 593–606

International Journal of Science Education ISSN 0950–0693 print/ISSN 1464–5289 online ©2005 Taylor & Francis Ltdhttp://www.tandf.co.uk/journals

DOI: 10.1080/0950069042000230758

RESEARCH REPORT

The development and validation of a primary science curriculum delivery evaluation questionnaire

Brian Lewthwaite Faculty of Education University of Manitoba WinnipegManitoba Canada R3J 2N2; e-mail: [email protected]; DarrellFisher SMEC Curtin University of Technology Bentley Campus Perth Australia

Taylor and Francis LtdTSED100759.sgm10.1080/0950069042000230758International Journal of Science EducationResearch Report2004Taylor & Francis Ltd00000000002004BrianLewthwaiteRoom 259, Faculty of EducationUniversity of ManitobaWinnipegManitobaR3J [email protected] study describes the processes involved in the development and statistical validation of a primary sciencecurriculum delivery evaluation instrument, the Science Curriculum Implementation Questionnaire (SCIQ),used to identify factors influencing science programme delivery at the classroom and school level. The studybegins by exploring the themes generated from several qualitative studies in the New Zealand context pertainingto the phenomenon of primary science delivery. Building on the findings from the qualitative studies, quantita-tive procedures used to develop and validate both a five-scale, 35-item SCIQ and a seven-scale, 49-item SCIQare presented. Finally, current applications of the seven-scale, 49-item SCIQ as a foundation for data collection,staff discussion and collaborative decision-making for the purpose of primary science delivery are brieflydiscussed.

Introduction

Science is acknowledged as an important part of every child’s education, yet there ismuch evidence to suggest that primary science education in many countries is in aparlous state (Mulholland and Wallace 1996). This perilous, ‘hard to put a fingeron it’ situation arises from the fact that, as Fullan (1992) affirms, the success ofcurriculum implementation and improvement efforts is influenced by several systemelements and that no one single factor can be targeted alone to effect change incurriculum delivery. Fullan (1993) asserts that curriculum interventions tend toleave the basic policies and practices of schools unchanged. These interventionstend to ignore the fact that changes in the core culture of teaching require majortransformation in the culture of the school. International efforts indicate thatalthough the intentions of primary science curriculum reviews and reform efforts ofthe past two decades have been admirable, the outcomes of these efforts have prima-rily been limited to increased teacher awareness and not teacher and instructionalchange (Harlen 1997).

Stewart and Prebble (1985) suggest that effective curriculum implementationand improvement come from a systematic, sustained effort at changing learningconditions in the classroom and other internal conditions within the school.Understanding the context in which change is to occur is at the heart of schooldevelopment (Stewart and Prebble 1993). This understanding is establishedthrough the gathering of high-quality information that provides insight into theforces at work within the school. In turn, this information becomes the foundation

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014

594 B. LEWTHWAITE AND D. FISHER

from which discussion, reflection and deliberate focused change can begin(Stewart and Prebble 1985). Because of the role this foundational data can have ininforming strategic school development, the diagnosis or systematic assessment ofthe school environment is seen as an essential means by which the forces that are atwork in a school impeding or contributing to curriculum implementation can beidentified.

Stewart and Prebble (1985) describe a variety of strategies for systematic datagathering, one of which is the use of validated instruments. The study and system-atic analysis of learning environments and educational climates using standardinstruments is a well-developed area of educational research (Fraser 1994, Fraserand Tobin 1998). The research primarily involves the investigation of partici-pants’ perceptions of their educational environment. A systematic analysis isconducted through the use of measurement instruments that are able to assess thevarious attributes of the educational environment. For schools not wishing toinvest the considerable amount of time and energy needed to complete moreformalized and extensive school reviews, the use of standard instruments is seen asa time-efficient, accurate but somewhat superficial means of understanding theforces at work within the educational context (Fullan 1992, Stewart and Prebble1983). When data collected from the instrument application are coupled withnarrative through staff discussion, they provide a foundation for increasing collec-tive knowledge and understanding of organizational procedures and problems(Stewart and Prebble 1985). As stated by Owens (1995), the employment of agood diagnostic tool becomes the starting point for the articulation of a reasonableprognosis.

This study focuses on the methodologies and outcomes of a sequence of studiespertaining to the identification of the factors influencing primary science curriculumdelivery, particularly within the New Zealand context, and the subsequent develop-ment of an evaluation instrument, the Science Curriculum Implementation Ques-tionnaire (SCIQ). The SCIQ was developed with the intent of providing a valuableand practical tool to support schools in the identification of the factors influencingscience programme delivery at the classroom and school level in which the teachingof science is an expected requirement of generalist teachers.

Methodology

The research sequence in this study involved several data collection stages, asoutlined in figure 1. For a number of years, workers in various areas of educationalresearch, especially the area of educational evaluation, have claimed that there aremerits in moving beyond the customary practice of choosing either qualitative orquantitative methods and instead combining qualitative and quantitative methods(Firestone and Pennell 1997, Fraser and Tobin 1998, Stewart and Prebble 1985).Such is the nature of the methods used in this study.Figure 1. Sequence of the investigation.The methods used in the first phase of the study (in-service questionnaire,preservice questionnaire, case study, and literature review) are qualitative and inter-pretivist as they attempt to explain the meaning of a social phenomenon — factorsinfluencing science curriculum delivery (Merriam 1998). The task of the researchwas to work with and make sense of the phenomenon through the frames and pre-understandings of the researched (Scott 1989). The overarching aim of this firstphase was to obtain information that could be analysed so that patterns associated

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014

PRIMARY SCIENCE DELIVERY EVALUATION QUESTIONNAIRE 595

with factors influencing science curriculum delivery in New Zealand schools, wherethe teaching of science is the responsibility of generalist teachers, could be extractedand comparisons made (Bell 1992). Once identified, these factors would becomethe basis for the development of items for the evaluation instrument. The method-ologies and results of this qualitative segment of the research exercise have beenpreviously published (Lewthwaite 1998, 1999a, 1999b, 2000, Lewthwaite andFisher 2004, Lewthwaite et al. 2001) and are not addressed in detail in this paper.Instead, the general methods employed and information obtained and used in thedevelopment of the instrument are presented.

The second stage of the study associated with the development and validationof the SCIQ (focus group consultation, development of the questionnaire, valida-tion, modification, and application) uses primarily quantitative methodologiesassociated with pattern identification and statistical analysis. These processes, usedin the instrument development and validation, are presented in detail in thispaper.

Themes identified from phase one

The initial qualitative study involved a questionnaire survey of 122 primary (years1–6) and intermediate (years 7–8) practising teachers in New Zealand. The ques-tionnaire focused on ascertaining teacher perceptions of factors influencing deliveryof the national science curriculum in schools where all teachers are required to teachscience as a part of their teaching duties. A follow-up case study of a large urbanintermediate school more thoroughly probed the influence of environmental andpersonal attribute factors on science programme delivery at the classroom andschool level. A final qualitative study involving 144 preservice primary and interme-diate teachers enrolled in the final year of a New Zealand-based teacher educationprogramme examined the development of science teaching personal attribute factorsamong primary teachers during their preservice education. Finally, a review of theliterature pertaining to curriculum, especially primary science, delivery was under-taken.

Figure 1. Sequence of the investigation.

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


The data collected in the initial phase of the study indicated that the effective-ness of science program delivery was strongly influenced by the professional scienceadequacy, the professional science attitude and interest, and the professional scienceknowledge of teachers because of the critical and pivotal position teachers hold inthe successful implementation of curricula. Equally, the process of curriculum deliv-ery was mitigated or inhibited by several other factors, many of these associated withthe physical and psycho-social dimensions of the school environment. Althoughteachers may be the critical agents in the curriculum implementation process, thesestudies affirmed that teacher professional adequacy, knowledge and interest are butone dimension in the complex matrix of factors that influence primary science deliv-ery. This matrix is not limited to some of the more salient features (resourceadequacy, time, professional support) that are commonly cited as impediments toeffective science program delivery (Appleton and Kindt 1999, Harlen 1997, Tilgner1990). Of particular significance is the role that school-based curriculum leadership,professional support and, in general, school culture have in influencing sciencecurriculum implementation and program delivery. Curriculum-focused leadershipand a school culture that advocates collaborative curriculum development toenhance educational opportunities for students are factors that strongly influencescience program delivery.

Overall, the case study analysis of the intermediate school accompanied by thedata collected from the in-service, preservice surveys and literature review assistedin the identification of the many factors that influence science programme delivery.These data became the foundation for the development of an instrument to system-atically evaluate factors influencing science curriculum delivery, the focus of thenext phase of this study.

Phase two

Each of the factors influencing science program delivery identified in the phase onestudies was placed on an ‘Instrument Items’ list. In all, 223 items identified in thephase one study were developed as items to be considered for the instrument. Thelist was not categorized or ranked, it simply listed all the specific factors that hadsurfaced during the phase one studies.

The next step in the development of the SCIQ item list was to eliminate someof the repetitive statements. Repeating items that were identical or differed in onlya word or two were eliminated from the list. This procedure reduced the number ofitems on the Item List to 136 items.

As the factors influencing implementation were identified, they were modifiedso that they would be appropriate for a teacher-response questionnaire similar to theform of standard instruments used extensively in learning environment research(Fraser 1994, Fraser and Tobin 1998). That is, a teacher would be able to answeror respond to the statement in the context of their classroom or school environment.As an example, one teacher had mentioned in the case study interviews that:

The way we feel intrinsically about a subject strongly influences our teaching of the subject.We devote more time to it and we teach it more passionately. I don’t think many of us arethat intrinsically interested in it … we see it in many of the children though … it all has a realeffect on us by motivating us to teach science. (Mary)

In order to change it into an item appropriate to the intent of the questionnaire, itwas modified to:

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


Item 12: Teachers at this school are intrinsically interested in teaching science.

and:

Item 13: Children’s interest in science at this school motivates us to teach science.

Focus group consultation

It was anticipated that many of the items would belong to general groupings or cate-gories of factors known to influence science programme delivery. The identificationof these groupings and classification of items was seen as the next critical stage ofthe instrument development. For these reasons a focus group consisting of sixpeople, each representing a different sector of the primary education community,was established. The focus group included a primary principal, a primary scienceadvisor, a senior teacher, an assistant teacher, a science school syndicate leader, anda tertiary science education lecturer. In order to assist in the development of thequestionnaire, the focus group separated into three pairs and each pair was given theItem List. Each pair was also given a Task Completion Sheet that clarified their rolesas focus group members. These tasks, as Knight and Meyer (1996) suggest, were toidentify any gaps in the factors included in the Item List and identify patterns andtrends in these data. The Item List was cut into individual items to assist the focusgroup members in identifying common groupings of factors. As well, the focusgroup members were asked to rank the items in each category according to howsignificant they perceived the items were in influencing science programme deliveryin the educational context in which they worked.

The items were easily identified as being resident within one of several generalclusters, themes, groupings or categories of factors known to influence scienceprogramme delivery. Several of these categories (resource adequacy, provision/avail-ability of professional support, staff interest, staff time availability, staff collegialityand collaboration, and administrative leadership and commitment) were those iden-tified by Fullan (1992). Most of these categories were primarily school culture orenvironmental attributes and failed to address the personal attributes of professionalknowledge and professional adequacy/confidence consistently identified in thephase one studies. Thus, two further categories not specifically identified by Fullanwere evident. Although Fullan had listed ‘teacher capability in dealing with the taskat hand’ as a factor influencing curriculum implementation, he does not specificallyidentify professional adequacy (self-efficacy) and multidimensional aspects ofteacher professional knowledge (Baker 1994, Shulman 1987) as individual, criticalconditions contributing to or inhibiting effective delivery.

Development of the initial instrument

Once the items were sorted and ranked, averages of the item rankings were calcu-lated for each category. Although the authors scrutinized the rank order determinedby the focus group, they were confident that the average rank order, as it existed,represented a hierarchy of items that were representative of the major factors influ-encing science curriculum delivery in the New Zealand context. Several furtherconsiderations were made in the actual development of the SCIQ. These included:

1. Consistency with existing instruments. Although many of the factors influenc-ing science curriculum delivery are unique, consideration was given to the

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


physical layout, dimensions, and scales existing in other learning environ-ment instruments. The School Level Environment Questionnaire (Fisherand Fraser 1990), in particular, provided a practical example on which tomodel the format of the SCIQ.

2. Economy of use. Because of the time constraints imposed on teachers andadministrators, it was essential to ensure that the instrument would requirea relatively short time to complete and process. In order to ensure this, theinstrument would ultimately contain seven items for each of the ‘factor’scales identified. In order to provide some flexibility in statistically refiningthe instrument, each scale on the initial instrument contained 10 items.Thus the top 10 average ranked items from each category were included inthe instrument.

3. Coverage of Moos’ general categories. The dimensions chosen for the SCIQprovided coverage of the three general categories that Moos (1974) identi-fied for all learning environments. These categories—Relationship Dimen-sions, Personal Development Dimensions and Maintenance and SystemChange Dimensions—are all inherent within the extrinsic or intrinsicfactors known to influence science programme delivery.

4. Recognition of Lewin’s and Murray’s theories as critical descriptors for under-standing human behaviour. Both Lewin and Murray regarded human behav-iour as a function of both the personality of the individual and theenvironment. Both the environment and its interaction with personal char-acteristics of the individual were recognized by Lewin as potent determi-nants of human behaviour (Fraser and Tobin 1998). Similarly, Murray’sNeeds-Press Model described an individual’s personal needs and environ-mental press as critical aspects influencing individual behaviour (Murray1938). Incorporating both personal and environment attributes wereregarded as essential in the development of the instrument.

The SCIQ in its initial form thus contained seven, 10-item scales. Table 1 presentsthe seven categories or dimensions contained within the questionnaire, a descriptionof each dimension and an example of one of the 10 items from this dimension.

The first four dimensions (Professional Support, Resource Adequacy, Time andSchool Ethos) are regarded as extrinsic factors influencing science program delivery.The latter three dimensions (Professional Adequacy, Professional Knowledge andProfessional Attitude) are regarded as intrinsic factors influencing science programdelivery.

Validation of the SCIQ

In order to validate the instrument, a large participation of schools and teachers wasrequired. Statistical analysis was able to then be performed to ensure that the SCIQwould measure what it claims to and that there were no logical errors in drawingconclusions from the collected data (Cook and Campbell 1979). Since the percep-tion measures of the SCIQ are measures of social concepts, construct validity anal-ysis was conducted (Cook and Campbell 1979). The validity analysis includeddetermining each scale’s internal consistency (Cronbach alpha coefficient), meanand standard deviation, uniqueness or ability to differentiate it from other scales(discriminant validity — using the mean correlation of a scale with the other scales

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


in the same instrument as a convenient index), and the ability of the scale to differ-entiate between the perceptions of teachers in different schools (significance vari-ance test). The statistical analysis further provided the necessary data to refine theSCIQ by reducing, if necessary, the number of scales and the number of items ineach scale.

The validation of the SCIQ process involved 293 teachers from 43 primary, fullprimary and intermediate schools located within the Central Districts of the NorthIsland of New Zealand. Statistical analyses for the initial validation were performedto determine the internal consistency of each 10-item scale (Cronbach alpha reliabil-ity coefficient), mean, and standard deviations. Based on the alpha reliability data,three items from each scale were eliminated to reduce the length of the scales and,consequently, improve the economy of the instrument. The seven-item scale inter-nal consistency is presented in table 2.

Discriminant validity

Although the alpha reliability coefficients confirmed that there was high internalconsistency in each scale, it was necessary to determine whether the scales overlapped

Table 1. Scales and sample items from the SCIQ.

Scale Description of scale Sample item

ResourceAdequacy

Teacher perceptions of the adequacy of equipment, facilities and general resources required for teaching of science

The school has adequate science equipment necessary for the teaching of science

Time Teacher perceptions of time availability for preparing and delivering the requirements of science curriculum

Teachers have enough time to develop their own understanding of the science they are required to teach

School Ethos Overall school beliefs towards science as a curriculum area. Status of science as acknowledged by staff, school administration and community

The school administration recognizes the importance of science as a subject in the overall school curriculum

ProfessionalSupport

Teacher perceptions of the support available for teachers from both in school and external sources

Teachers at this school have the opportunity to receive ongoing science curriculum professional support

Professional Adequacy

Teacher perceptions of their own ability and competence to teach science

Teachers at this school are confident science teachers

Professional Knowledge

Teacher perceptions of the knowledge and understandings teachers possess towards science as a curriculum area

Teachers have a sound understanding of alternative ways of teaching scientific ideas to foster student learning

ProfessionalAttitude andInterest

Teacher perceptions of the attitudes and interest held towards science and the teaching of science

Science is a subject at this school that teachers want to teach

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


and differentiated between schools. These aspects became the focus of the validationanalysis discussed in this section. The discriminant validity described as the meancorrelation of a scale with the other six scales for the reduced seven-item scale ispresented in table 3.

Mean correlations for all scales, in particular the Professional Knowledge,Professional Attitude and Interest, and Professional Leadership, suggest the scalesmeasure somewhat overlapping aspects of teachers’ perceptions of factors influenc-ing science programme delivery. In order to determine what factors overlapped themost, correlations between all scales were determined. The correlations of eachscale with the other six scales are presented in table 4.

The scales that have the highest correlations are Professional Knowledge andProfessional Adequacy (0.87), Professional Adequacy and Professional Attitude andInterest (0.82), and Professional Knowledge and Professional Attitude and Interest(0.78). These high correlations would suggest that the scales are not discriminatingbetween each other and are measuring very similar attributes. A more detailedanalysis was completed to determine the correlation among all personal attribute

Table 2. Alpha reliability, mean and standard deviation for the seven-scale SCIQ.

Scale Items on finalquestionnaire

Mean Standarddeviation

Alpha reliability (seven-item scale)

Professional Knowledge 1,8,15,22,29,36,43 3.32 3.76 0.77

Professional Attitude 2,9a,16,23,30,37a,44 3.53 5.55 0.88

Professional Adequacy 4,11,18,25,32,39,46a 3.38 5.48 0.92

Professional Support 7,14,21,28,35,42,49 3.60 3.76 0.90

Resource Adequacy 3,10,17,24,31,38,45 3.39 4.01 0.83

School Ethos 5,12,19,26,33,40,47 3.51 3.99 0.90

Time 6a,13,20a,27a,34,41,48 2.78 3.02 0.90

aReverse item scores on the final seven-scale SCIQ.

Table 3. SCIQ: mean correlations of the seven-item scale with the other six scales.

Scale Discriminant validity seven-item scales

Professional Knowledge 0.60Professional Attitude and Interest 0.60Professional Adequacy 0.48Professional Support 0.53School Ethos 0.58Resource Adequacy 0.41Time 0.31

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


factors in the initial 49-item questionnaire. Principal component factor analysisresulted in nearly all items in the extrinsic factor scales (Resource Adequacy, Profes-sional Support, School Ethos, and Time) having a factor loading of at least 0.50 ontheir a priori scale. It was evident that these four scales were measuring independentfactors. On the other hand, factor loadings for the three intrinsic scales (ProfessionalAdequacy, Professional Knowledge, and Professional Interest and Attitude) over-lapped significantly. The factor analysis identified these items as belonging to asingle scale. On the basis of the overlap evident from the factor analysis, the threeintrinsic scales were merged into a single seven-item scale. Thus, a five-scale SCIQ(not addressed further in this study) containing four environmental dimensions andone personal attribute dimension was developed.

This overlap is not surprising as the phase one studies, including the review ofthe literature, showed that these three personal attribute dimensions are suggestedto be closely related. As an example, Baker (1994) and Tilgner (1990) state that lowself-efficacy towards the teaching of science is commonly associated with poorprofessional science knowledge. Despite this suggested association, recent researchwithin the New Zealand context ensuing from this study has indicated that the rela-tionship among these personal attributes is, at best, tenuous and that changes inteacher perception of their own development in professional science teaching, effi-cacy and knowledge during preservice teacher education and in-service professionaldevelopment programs are not directly correlated statistically (Lewthwaite andMacIntyre 2003). Although identified in this study to be related statistically, theyare, individually, of such importance in influencing science programme delivery thatit is beneficial to retain them as separate categories in the questionnaire. In practicalterms, it is worthwhile considering how a school might address low scores in a scalein which these three personal attributes are combined. The manner in which lowprofessional interest is addressed is likely to be quite different than the strategy usedto address low professional knowledge. Similarly, the strategy used to address lowperceptions of professional adequacy is likely to be different from those used toaddress low levels of professional interest (Stewart and Prebble 1985, 1993). Forthese reasons, consideration needs to be given to the purpose of the development ofthe SCIQ. It is regarded as a valuable and practical tool to support schools in

Table 4. Inter-scale correlations for the seven-scale SCIQ.

Professional Support

Time SchoolEthos

Resource Adequacy

Professional Adequacy

Professional Attitude and

Interest


Professional Support 1 0.33 0.67 0.46 0.58 0.56 0.58Time 1 0.35 0.22 0.29 0.33 0.33School Ethos 1 0.49 0.65 0.70 0.62Resource Adequacy 1 0.43 0.40 0.46Professional Adequacy

1 0.82 0.87

Professional Attitude and Interest

1 0.78


1

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


identifying and addressing the broad and complex factors influencing scienceprogramme delivery. If scales are merged that address dimensions that are uniquelyimportant and remedied in different ways, although they load on the same factorstatistically, the scales need to be retained. Consequently, a further seven-scaleSCIQ was retained.

Interschool correlations

A further analysis was conducted to determine whether the seven-scale SCIQ differ-entiated between schools. Initially a multivariate analysis of variance was conductedusing the SPSS General Linear Model procedure. This analysis combined all sevenscales as the dependent variable and the factor as the school. Significant results(p<0.01) were found for the school factor, confirming that the SCIQ has the abilityto differentiate among the perceptions of teachers at different schools. Followingthis, a one-way analysis of variance was conducted to determine the ability of eachscale of the SCIQ to differentiate between the perceptions of teachers at differentschools. The analysis of variance results obtained were again significant (p<0.01),confirming the ability of not only the questionnaire but also each scale’s ability todifferentiate among schools.

A further η2 statistic was calculated to provide an estimate of the strength ofassociation between school membership and the SCIQ. The high variance results(range, 0.040–0.063) suggested that there was a high degree of variability among theschools for each factor. This would be anticipated when one considers that 20 of theschools were only one or two teacher schools and that a single individual responsethat is quite different from the rest could distinguish strongly among the schools andresult in a higher η2 statistic. In order to a reduce the influence of individual schools,a further analysis of variance was conducted with the dependent variable being thescale but the factor being school size. Schools were broken down into three catego-ries: small (one to four teachers), medium (five to nine teachers), and large (>10teachers). This reduced the variance considerably (range, 0.005–0.008). On thebasis of the analysis of variance results it can be concluded that the SCIQ in itsentirety and as discrete scales is able to differentiate among schools. The seven-scale, 49-item SCIQ in its final form, after validation and modification, is presentedin appendix 1.

Current applications of the SCIQ

Current applications of the seven-scale SCIQ are encouraging its usefulness as amanageable and accurate evaluation tool for identifying the complex amalgam ofintrinsic and extrinsic factors influencing science programme delivery in schoolswhere the teaching of science is a part of each teacher’s professional role (Edmondsand Lewthwaite 2002, Gulliver and Lewthwaite 2002, Lewthwaite 2004, Lewth-waite and Fisher 2004, Payne and Lewthwaite 2002). In these studies, sciencecurriculum leaders and principals in New Zealand and Canada have been using theSCIQ to collect data to inform collaborative science curriculum delivery evaluationand improvement at the classroom and school level. The procedure for instrumentapplication involves all teachers with responsibility for the teaching of scienceanswering the items on the SCIQ. The actual form (referred to in this study) allowsparticipants to identify how things actually are in the environment being evaluated.

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


The preferred form is concerned with participants’ perceptions of the environmentpreferred. As an example, item 46 on the actual form states ‘Teachers at this schoolhave a negative self-image of themselves as regards their ability to teach science’,whereas on the preferred form it is worded ‘Teachers at this school would have a posi-tive self-image of themselves as regards their ability to teach science’. The responsescale on both forms is a five-point scale including Strongly Disagree, Disagree, NotSure, Agree, and Strongly Agree. Each response is assigned a corresponding scorefrom 1 to 5. Some items, as indicated in table 2, are reverse-score items. Oncecollected, the questionnaires are processed calculating scale means and standarddeviations. These data are then presented as graphs, tables and descriptive scaleprofiles to those involved in the application exercise and discussed through thesupport of a facilitator familiar with the SCIQ and its intentions. Through a collec-tive discussion with the participating staff, the accuracy of the data is authenticatedand strategic procedures for addressing impediments to effective science delivery areidentified. At the end of a cycle of focused improvement, the SCIQ is applied againand, if necessary, further strategies for improvement are discussed.

Summary

The purpose of this paper has been to outline the procedures involved in the devel-opment, validation and refining of the SCIQ. As Prebble and Stewart (1985)suggest, the use of data-collecting instruments such as the SCIQ as a foundation forschool review is an accurate and time-effective means by which an analysis of theschool can be conducted. The data collected and feedback received from partici-pants from recent SCIQ application exercises in both New Zealand and, mostrecently, Canada would confirm this assertion. For this reason, the use of evaluationtools such as the SCIQ tools to provide a foundation for school discussion, reflec-tion, and strategic educational improvement is encouraged.

References

APPLETON, K. and KINDT, I. (1999). Why teach primary science? Influences on beginning teach-ers’ practice. International Journal of Science Education, 21(2), 155–168.

BAKER, R. (1994). Teaching science in schools: what knowledge do teachers need? Research inScience Education, 24(3), 31–40.

BELL, J. (1992). Doing Your Research Project (Buckingham: Open University Press).COOK, T.D. and CAMPBELL, D.T. (1979). Quasi-experimentation: Design and Analysis Issues for

Field Settings (Boston, MA: Houghton-Mifflin).EDMONDS, G. and LEWTHWAITE, B.E. (2002). Science program delivery evaluation: a principal’s

journey in school self-review. In R.K Coll (ed.) STERpapers 2002 (Hamilton, NewZealand: Centre for Science and Technology Education Research, Waikato University),181–197.

FIRESTONE, W.A. and PENNELL, J.R. (1997). Designing state-sponsored teacher networks: acomparison of two cases. American Educational Research Journal, 2(3), 249–258.

FISHER, D.L. and FRASER, B.J. (1990). SLEQ: School Level Environment Questionnaire. Set:Research Information for Teachers. Item 5 (Melbourne: Australian Council for EducationalResearch).

FRASER, B.J. (1994). Research on classroom and school climate. In D. Gabel (ed.) Handbook onResearch on Science Teaching and Learning (New York: Cassell).

FRASER, B.J. and TOBIN, K. (1998). Qualitative and quantitative landscapes of classroom learningenvironments. In B.J. Fraser and K.G. Tobin (eds.) International Handbook of ScienceEducation (Dondrecht: Kluwer Academic Publishers), 527–564.

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


FULLAN, M. (1992). Successful School Improvement (Buckingham: Open University Press).FULLAN, M. (1993). Change Forces: Probing the Depths of Educational Reform (London: Falmer

Press).GULLIVER, P. and LEWTHWAITE, B.E. (2002). Primary science delivery: a procedure for evalua-

tion. Science Education International, 13(2), 25–29.HARLEN, W. (1997). Primary teachers’ understandings in science and its impact in the classroom.

Research in Science Education, 27(3), 323–337.KNIGHT, G. and MEYER, D. (1996). Critical Factors in the Implementation of the New Mathematics

Curriculum: A Research Report (Palmerston North: Massey University).LEWTHWAITE, B.E. (1998). Teaching primary science: the problem concept areas. New Zealand

Science Teacher, 88, 28–32.LEWTHWAITE, B.E. (1999a). Teacher perceptions of factors influencing science in the New

Zealand curriculum. New Zealand Science Teacher, 90, 14–18.LEWTHWAITE, B.E. (1999b). Will there be a change of state? Primary science education in New

Zealand. New Zealand Science Teacher, 92, 5–9.LEWTHWAITE, B.E. (2000). Teaching primary science—how teachers see the problems. In G.

Haisman (ed.) Exploring Issues in Science Education (Wellington: Research & CurriculumDivisions, Ministry of Education), 11–22.

LEWTHWAITE, B.E. (2004). ‘Are you saying it’s my fault?’ An analysis of a principal’s role in influ-encing science program delivery. Research in Science Education, 34(2), 137–153.

LEWTHWAITE, B.E. and FISHER, D.L. (2004). The application of a primary science curriculumevaluation questionnaire. Research in Science Education, 34(1), 55–70.

LEWTHWAITE, B.E. and MACINTYRE, W.R. (2003). Teacher professional knowledge, attitude andself-efficacy: a vignette study. In R.K Coll (ed.) SAMEpapers 2003 (Hamilton, NewZealand: Centre for Science and Technology Education Research, Waikato University),58–80.

LEWTHWAITE, B.E, STABLEFORD, J. and FISHER, D.L. (2001). Enlarging the focus on primaryscience education in New Zealand. In R.K Coll (ed.) SAMEpapers 2001 (Hamilton, NewZealand: Centre for Science and Technology Education Research, Waikato University),213–237.

MERRIAM, S.B. (1998). Qualitative Research and Case Study: Applications in Education (SanFrancisco, CA: Jossey-Bass).

MOOS, R.H. (1974). The Social Climate Scales: An Overview (Palo Alto, CA: Consulting Psycholo-gists Press).

MULLHOLLAND, J. and WALLACE, J (1996). Breaking the cycle: preparing elementary teachers toteach science. Journal of Elementary Science Education, 8(1), 17–38.

MURRAY, H.A. (1938). Explorations in Personality (New York: University Oxford Press).OWENS, R.G. (1995). Organizational Behavior in Education (Boston, MA: Allyn and Bacon).PAYNE, K. and LEWTHWAITE, B.E. (2002). Systematic curriculum review: a procedure for

analysis. New Zealand Principal—Te Temuaki O Aotearoa, 18(2), 30–33.SCOTT, A.W. (1989). Inservice for primary teachers in science education: some directions for the

future. Research in Science Education, 19, 249–256.SHULMAN, L.S. (1987). Knowledge and teaching: foundations of the new reform. Harvard

Educational Review, 57(1), 1–22.STEWART, D. and PREBBLE, T. (1993). The Reflective Principal: School Development Within a

Learning Community (Palmerston North: ERDC Press).STEWART, T. and PREBBLE, T. (1985). Making it Happen: A School Development Process (Palmerston

North: Dunmore Press).TILGNER, P.G. (1990). Avoiding science in the elementary school. Science Education, 74(4),

421–431.

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014


Appendix 1:SCIQ

There are 49 items in this questionnaire. They are statements to be considered inthe context of the school in which you work. Think about how well the statementsdescribe the school environment in which you work.

Indicate your answer on the score sheet by circling:

SD if you strongly disagree with the statement;N if you neither agree nor disagree with the statement or are not sure;SA if you strongly agree with the statement;D if you disagree with the statement;A if you agree with the statement;

If you change your mind about a response, cross out the old answer and circle thenew choice.

1. Teachers at this school have a good understanding of the science SD D N A SAknowledge, skills and attitudes they are to promote in their teaching.

2. Teachers have a positive attitude to the teaching of science. SD D N A SA3. The school is well resourced for the teaching of science. SD D N A SA4. Teachers at this school are adequately prepared to teach science. SD D N A SA5. The school administration recognizes the importance of science as SD D N A SA

a subject in the overall school curriculum.6. There is not enough time in the school program to fit science in SD D N A SA

properly.7. Teachers at this school have the opportunity to receive ongoing

science curriculum professional support. SD D N A SA8. Teachers at this school have a sound knowledge of strategies known

to be effective for the teaching of science. SD D N A SA9. Teachers at this school are reluctant to teach science. SD D N A SA10. The school-based system of managing of science resources is well SD D N A SA

maintained.11. Teachers at this school are confident science teachers. SD D N A SA12. The school’s ethos positively influences the teaching of science. SD D N A SA13. There is enough time in the school week to do an adequate job of SD D N A SA

teaching the requirements of the science curriculum.14. Collegial support is a positive factor in fostering the implementation SD D N A SA

of science programs in this school.15. Teachers have a sound understanding of alternative ways of SD D N A SA

teaching scientific ideas to foster student learning.16. Teachers have a strong motivation to ensure science is taught at this SD D N A SA

school.17. Teachers at this school have ready access to science materials and SD D N A SA

resources.18. Teachers at this school are competent teachers of science. SD D N A SA19. The school places a strong emphasis on science as a curriculum SD D N A SA

area.20. The school curriculum is crowded. Science suffers because of this. SD D N A SA21. The collegial support evident in this school is important in fostering SD D N A SA

capabilities in teachers who find science difficult to teach.

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014

606 PRIMARY SCIENCE DELIVERY EVALUATION QUESTIONNAIRE

22. Teachers at this school are secure in their knowledge of science SD D N A SAconcepts pertinent to the science curriculum.

23. Teachers at this school have a positive attitude to science as a SD D N A SAsubject in the overall school program.

24. The facilities at this school promote the teaching of science. SD D N A SA25. Teachers possess the personal confidence, and skills necessary to SD D N A SA

teach science competently.26. Science has a high profile as a curriculum area at this school. SD D N A SA27. There is not enough time in the school program to teach science. SD D N A SA28. Teachers have the opportunity to undertake professional SD D N A SA

development in science.29. Teachers at this school possess the necessary science subject SD D N A SA

knowledge to be a good science educator.30. Science is a subject at this school that teachers want to teach. SD D N A SA31. The science resources at the school are well organized. SD D N A SA32. Teachers at this school have positive perceptions of their SD D N A SA

competence as science educators.33. Science has a high status as a curriculum area at this school. SD D N A SA34. Teachers believe that there is adequate time in the overall school SD D N A SA

program to teach science.35. Teachers at this school are supported in their efforts to teach SD D N A SA

science.36. Teachers at this school have a good understanding of students’ SD D N A SA

science background knowledge and thinking. SD D N A SA37. Teachers at this school have a negative attitude to science as an SD D N A SA

essential learning area. SD D N A SA38. The equipment that is necessary to teach science is readily available. SD D N A SA39. Teachers at this school are adequately prepared to teach to the SD D N A SA

requirements of the science curriculum.40. Science as a curriculum area is valued at this school. SD D N A SA41. Teachers have the time to effectively deliver the requirements of the SD D N A SA

national science curriculum.42. The senior administration actively supports science as a curriculum area. SD D N A SA43. Teachers have clear understanding of the goals and objectives of the SD D N A SA

science curriculum.44. Teachers at this school are motivated to make science work as a SD D N A SA

curriculum area.45. The school has adequate science equipment necessary for the SD D N A SA

teaching of science.46. Teachers at this school have a negative self-image of themselves as

regards their ability to teach science. SD D N A SA47. Science is regarded as an important subject in the school’s overall SD D N A SA

curriculum.48. Time is a major factor inhibiting science program delivery at this school. SD D N A SA49. The curriculum leadership in science fosters capabilities in those SD D N A SA

who require support in teaching science.

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

nder

land

] at

04:

30 2

0 D

ecem

ber

2014

the development and validation of a primary science curriculum delivery evaluation questionnaire

Documents