13230 Nagarajah et al./ Elixir Early Childhood Education 58A (2013)

13230-13234

Developing a web based assessment tool for Preschool

Children Assessment Nagarajah Lee, Nanthakumar, Arif Shah

Open University Malaysia.

A R T I C L E I N F O

Article history:

Received: 6 September 2012;

Received in revised form:

1 February 2013;

Accepted: 15 February 2013;

Keywords

Early Childhood,

Assessment,

Web Based System

AB STRACT

Early childhood education involves any group programs serving children from birth to 8 years old that is designed to promote children’s intellectual, social, emotional, language and physical development and learning. Early childhood education is the key to building a great foundation for children’s educational success. In ensuring high-quality early childhood programs, child assessment is crucial and the call for having a holistic assessment framework increases by the day. The assessment of young children's development and learning is seen as the avenue for parents and preschools to gauge the effectiveness of the preschool program. The purpose of this research is to develop a holistic assessment framework for assessing the growth of preschool children on four important aspects : cognitive, psychomotor, emotional; and values. International standards will be used to develop the assessment framework and this will be validated in the Malaysia preschool setting for its’ psychometric properties. A web based system will be built based on the validated standards. Two established preschools in Klang Valley will be adopted and the assessment system will be implemented to test the effectiveness.

© 2013 Elixir All rights reserved.

Introduction

Preschool education consists of specially designed educational

experiences to stimulate, assist, support, and sustain emergent

skills before entering primary education (Ackerman, D.J, 2005) .

Preschools, whether government funded or privately owned, aim

at providing a wide range of developmentally appropriate

experiences that young children need to be exposed to and the

core curriculum content should allow for the attainment of the

defined learning outcomes (Chang, N (2010).

The preschool curriculum is defined as an educational philosophy

for achieving desired educational outcomes through the

presentation of an organized sequence of activities with a

description and/or inclusion of appropriate instructional materials

(Barnett,W.S 2004). In preschool education, assessment is a vital

component to ensure the quality of early childhood programs. Not

only is it an important tool in understanding and supporting young

children’s development, it is essential to document and evaluate

the effectiveness of the preschool program (Stipek,D 2005). For

assessment to be effectively used, it must employ methods that

are feasible, sustainable and reasonable with regards to demands

on budgets, educators and other resources (Wortham,S.C, 2005)

On top of everything, it must be valid in assessing the

development of young children. Thus having a sound framework

and a workable methodology is critical. To date in Malaysia there

is no comprehensive tool to assess the development of preschool

children holistically

This study aims to address the following objectives:

To develop the standards for preschool children in Malaysis on the following domains : cognitive, psychomotor, emotional, and values.

To develop the performance targets on the four domains according to the children’s developmental stage.

To determine the invertentions in addressing the gaps between the actual and expected performance of the children based on the four domains.

To develop a web based assessment system that will

link the performance standards, performance targets,

and intervention as well as generate individual and

group reports

Literature Review

Review of literature is based on the research questions stated, to

ensure that there are relevant areas that have been researched,

the limitations accounted by the researchers, their concerns as

well as the research methodologies carried out. Thus for the

purpose of this assignment different areas of assessing

preschoolers were previewed and considered.

The first of the literature reviewed was by Epstein et.al (2004). In

her article it mentioned the problems related to assessment of

preschoolers.

Tele:

E-mail addresses: dr_f_khurshid@yahoo.com

© 2013 Elixir All rights reserved

Early Childhood Education

13231 Nagarajah et al./ Elixir Early Childhood Education 58A (2013)

13230-13234

Firstly, there is a “proliferation of assessment tools”, then how to

use them to assess the various domains of child development so

as to be able to be effective in the evaluation of preschool

programmes. Some of the types of assessment tools that were

identified included “comprehensive developmental assessment”

which takes into consideration the various aspects of child

development including child care services and parental

involvement. Another form of assessment is screening. It is an

assessment tool developed to check for developmental delays or

late development among preschoolers so as to be able to provide

for early intervention or remediation. Formal assessment was also

included which is specifically to measure the different – physical,

cognitive, language, social and emotional development of

preschoolers. Another mode of assessment is criterion-referenced

wherein the child’s performance is compared to a specific

objective or performance standard. Norm-referenced assessment

is developed to compare the child’s development against pre-

established standards of a larger group.

The writer however has, with experts in child assessment

eliminated “the proliferation of assessment tools” and identified

screening, informal assessment which includes portfolios and

classroom observations, formal assessments i.e. the use of

assessment tools that are developmentally appropriate to

teaching and learning and also parental involvement as

instruments for assessing pre-schoolers.

To obtain data for this study was by a survey, emailed to 32 Early

Learning Coalitions in Florida state which represented 62

counties. Design, survey of questionnaires and field-testing were

by university researchers. Data compilation and findings were

reported quantitatively using Excel.

Bowers, F.B (2007) stated that since this survey focused on

Formal Assessments, information as to whether schools have

used portfolios or other informal tools to support the assessment

of pre-schoolers could not be authenticated.

Other limitations that resulted from this survey were – not being

able to obtain information on staff training, whether there were

variations in the assessment tools used in the different centres.

The findings indicated there were some assessments used at the

centres that were not backed by child assessment experts; only

one instrument, the Creative Curriculum Assessment Tool

(CCAT), used in 10 counties, had the link with curriculum.

Although there was parental involvement but it was not significant.

From this study it is noted that to identify the “common”

assessment tools, it is important to get the endorsement of

preschool assessment experts. There should be a number of

assessment tools that need to be used so as to be appropriate to

the age and development of the preschoolers. The necessity to

have many ways of assessing preschoolers is because they

develop very fast in the various aspects of growth (Guddeni, M.,

and Case, B.J, 2004), Wortham, S. C. (2005) and McCauley,. L.

(2004). Another characteristic of preschoolers is that they have

very short attention span so any form of assessment should take

into consideration this characteristic too.

Gober, S. Y. (2002) in Six Simple Ways to Assess Young

Children, identified and specifically mentioned developmental

checklists, interviews with parents, children’s self-portraits,

children’s scribbling, drawing and writing, audio or visual tapes

as well as anecdotal records, as methods of assessing

preschoolers. Puckett, M. B. and Black, J.K. (3rd ed. 2008) have

two categories of assessment – informal and formal. Informal

assessments include observation, inventories, interviews,

anecdotal records, checklists, rating scales and under formal

assessments are – diagnostic instruments, standardised

instruments, curriculum-related tests and diagnostic systems.

So what could be identified thus far are the use of portfolios,

observation and parental involvement. In the NPSC it is clearly

stated that one of the assessment tools to be used is criterion-

referenced testing which includes continuous observation of

children’s work. Other assessment tools include checklists,

rating scale, anecdotes, continuous records and portfolio.

So there are “common” assessment tools that have been

mentioned by Gober, S.Y. (2002) and Puckett, M. B. and Black,

J.K. (3rd ed. 2008). Hence the task of preschool teachers

particularly in Malaysia having to identify and choose different

assessment tools to assess their students have somewhat been

reduced.

Having identified a few of the “common” instruments the next

primary concern is how the various assessment instruments are

used to assess the developmental domains of pre-schoolers.

This relates to the expertise of preschool teachers because

assessment tools need the people to use them to affect change

or for correct reporting of the development of the pre-schoolers.

Layton, C.A and Lock, R.H (2007) in their study, “Preschool

teachers’ use of assessments and curricula: a state wide

examination”, researched into two main statement problems

which were the degree to which the recommended

assessments and the curriculum were carried out by the

preschool teachers and whether teachers’ qualification and

experience had any effect on the use of assessments and

curriculum.

Their reviews of literature showed that research into

assessment and curriculum for pre-schoolers were few or

outdated. This is also reflected in the Malaysian context where

there were no studies conducted to show the relationship

between assessment and curriculum for pre-schoolers. The

writers also found that the procedure to collect information was

not explained or provided so validating the conclusions were

inadequate. Again, this same scenario occurs in Malaysia

where the reporting of how preschoolers performed or

assessed were usually very general, with statements that

indicate whether preschoolers were - able to achieve, in

progress or yet to achieve. Information and data gathering has

never been a part of assessing the total development of the

preschoolers.

13232 Nagarajah et al./ Elixir Early Childhood Education 58A (2013)

13230-13234

Methodology

This research used the mixed method approach. First a thorough analysis of related documents on pre school standards, curriculum, learning outcomes, and assessment was done to establish a comprehensive list of standards. A meta analysis was carried out to establish the performance indicators for holistic development of pre schoolers. The analysis was also used to set the targets for the identified standards. Documents related to preschool curriculum and teaching and learning materials were reviewed to identify the suitable activities that are associated with the standards identified. The next stage involved interviews with principals and teachers of selected well-established preschools in the Klang Valley. This is to ensure that the standards and the performance targets are suitable for Malaysian preschool children. A group of preschool teachers were trained to develop suitable activities in accordance with the standards established. Based on the analysis the following THREE databases were developed:

Data Base A : Performance Standards Data Base B : Performance Targets Data Base C : Interventions

A prototype web based assessment system was developed incorporating the three databases. The system has the features that will allow the assessors to select the performance standards, performance targets, and interventions that are developmentally appropriate to the child that being assessed. This assessment system also has the features to produce individual as well as group reports. A few well established preschools in Klang Valley were used to pilot the assessment system for one year. Upon proper training, the teachers used the system to assess the children quarterly and record their observation in a specially design log book. The teachers were then surveyed to gauge the effectiveness of the web based assessment tool.

Research Flowchart Establishment of the databases Database A : Standards for the learning outcomes for the

preschool children of different developmental stage (Pre K1, K1 and K2)

Database B : Performance Targets based on the standards

prescribed in Database A Database C : Interventions

Modus Operandi

LITERATURE REVIEW and BENCHMARKING

QUALITATIVE RESEARCH PHASE

(Preschool Teachers & Principals)

FOCUS GROUP DISCUSSION CONFIRMATORY STAGE

(3 – 5 GROUPS0

DEVELOPMENT OF THE DATABASES (A,B and C)

SYSTEM DEVELOPMENT AND

TESTING

EXPERT VALIDATION

DESIGN ASSESSMENT FRAMEWORK

INTERVIEW GUIDE

FOCUS GROUP DISCUSSION EXPLORATORY STAGE

(3 - 5 GROUPS)

DOCUMENT ANALYSIS

SYSTEM DEVELOPMENT

The outcome of the analysis is a comprehensive set of indicators, targets as well as remedial and enrichment activities. The indicators to measure the development of preschool children age between 4 and 5 years are for the following domains:

Gross motor skills

Fine motor skills

Wellbeing and Hygiene

Interacts with adults

Interacts with peers

Language

Curiosity and Interest

Persistence and Attentiveness

Critical and analytical thinking

Problem solving

Mathematical thinking

Family, community , and culture

Creative arts

13233 Nagarajah et al./ Elixir Early Childhood Education 58A (2013)

13230-13234

A web-based assessment system to assess the holistic

development of pre school children was developed. The program

provided component specific indicators as well as the

interventions either as remedial or enrichment activities. The 50

preschool teachers from selected preschools in Klang Valley were

used in this survey. All the teachers were provided with training on

how to use the web based assessment system.

The dimensions for the instrument to assess the effectiveness of

the web based assessment system was developed based on a

thorough review of the existing literature whilst the focus group

interviews were used to generate the specific indicators. The

indicators were translated into a questionnaire. The draft

questionnaire was given to a group of 80 students enrolled in the

Diploma in Early Childhood development programmes in one of

the private college in Klang Valley. This data was used to establish

the psychometric properties of the instrument. The validated

questionnaire is then used to assess preschool teachers

perception on the effectiveness of the web based assessment

system

The psychometric properties of instrument refer to the soundness

of the instrument in measuring the intended construct. This is one

of the major concerns in social science studies since most

constructs are difficult to be measured objectively. The

psychometric properties of the instrument was evaluated in terms

of validity, and the reliability.

a) Validity

The validity of the instrument refers to its’ ability to measure what

it purports to measure. Since the validity of the study very much

depends on the validity of the instrument used, it is an important

issue to be addressed. Broadly, validity refers to how accurately a

particular construct is translated into measurable behaviours.

Among the types of validity discussed in this paper are: face and

content validity, convergent validity, concurrent validity, and

dimensionality.

i Face and Content Validity

Clearly specifying the domain of the construct, generating items

that exhaust the domain, and purifying the resulting scale should

produce a measure, which is content or face valid and reliable

(Churchill, 1976, p. 70). Since a thorough review of the literature

was carried out to determine the constructs, and focus group

discussions were used to generate specific indicators to measure

the defined construct, necessary steps had been taken to

establish sound face and content validity. The factor analyses,

both exploratory and confirmatory shows that students’ ratings

converge with the theoretical description of the construct. Table 1

illustrates the items that were assigned theoretically and the items

that load statistically to the common factors.

Table 1 : Instrument for measuring effectiveness of the web based assessment system

Dimension

No. of items

assigned based

on theory

No. of items that converged

as expected

Theoretical explanation

Effectiveness of the web bases assessment system

Factor 1 (Ease of Use ) 8 4

How easily the system can be used .

Factor 2 (Appropriateness of Content) 7 4

The assessment items’ appropriateness .

Factor 3 (Helpful in children development ) 6 3

Whether the web based assessment helps the preschool children’s development

ii) Convergent Validity

Some researchers claimed that each of the items in the

instrument can be treated as different indicators to measure

the same construct. To determine on which dimension the

items in the questionnaire load, an exploratory factor analysis

was performed. Items that were statistically loading to the

common factors were then compared with the relevant theories

(refer to Table 1). After dully considering the theory as well the

statistical outcomes, items that load to factors that cannot be

explained theoretically were dropped and the second level

exploratory factor analysis was performed on the reduced

number of items to determine the factor structure.

Whilst factor loadings indicates the convergence of the items

to the respective factors (of dimensions), the strength of the

convergence i.e how closely all the items in the said factor (or

dimension) clustered together is determined by computing the

Bentler Bonnet Coefficient.

According to Ahire et.al (1996), the convergent validity of the

instrument can be determined using Bentler Bonnet coefficient

(delta), a delta value of 0.90 or greater demonstrates strong

convergent validity.

Table 2 shows the components and the number of items for

the dimensions as well as the Bentler-Bonnet Coefficient for

the effectiveness of the web based assessment system

13234 Nagarajah et al./ Elixir Early Childhood Education 58A (2013)

13230-13234

Table 2 : The convergent validity of the Instrument for measuring

effectiveness of the web based assessment system

Dimension Number of Items

Bentler-Bonnet Coefficient

Effectiveness of the web based assessment system

Factor 1 (Ease of Use ) 4 0.87

Factor 2 (Appropriateness of Content)

4 0.88

Factor 3 (Helpful in children development )

3 0.89

iii Dimensionality:

Dimensionality is a process of evaluating the “belongingness” of

the items to certain dimensions in the construct. For the instrument

to be dimensionally sound, items should only measure the

dimensions that they theoretically belong to. The confirmatory

factor analysis (CFA) is used if the dimension for the construct is

supported by a sound theory and the researcher has a reasonably

good knowledge of the number of dimensions while the

exploratory factor analysis (EFA) is used when the researcher is

uncertain about the relationship between the items and latent

factors (Ahire at.al, 1996).

The construct ‘Effectiveness of the web based assessment

system’ was defined using both existing literature and focus group

interviews, both the EFA and CFA were used. The exploratory

process was used to explore the relationship between the latent

factors (dimensions) and the observed variables (items), while the

CFA was used to confirm the relationship (Sureshacandar et al.,

2002). The principal component analysis was used as the

extraction method for the EFA and the factors were rotated using

the Varimax rotation method with Kaiser normalization. Prior to

that, a reliability test was performed and only items with an index

greater than 0.4 were considered for factor analysis.

The EFA provides a three -factor solution with 68.44% total

variance explained. The Bartlett Test of Sphericity gives a very

small p-value (0.000), indicating that there is a statistical

probability that the correlation matrix has a significant correlation

among at least some of the variables (Hair, Anderson, Tatham &

Black 1995). Furthermore the Kaiser-Meyer-Olkin Measure of

Sampling Adequacy is also very high, 0.871, indicating that the

latent constructs can predict the variability of the responses in the

observed variables. Table 3 shows the result of the factor

analysis.

Table 3 : Factor analysis: Effectiveness of web based assessment system

Rotated Component Matrix

Component

Ease of Use

Appropriateness

of Content

Helpful in

children

development

Item 9 .845

Item 10 .827

Item 11 .751

Item 5 .754

Item 6 .755

Item 7 .812

Item 8 .802

Item 1 .753

Item 2 .435 .772

Item 3 .433 .750

Item 4 .777

Variance

Explained

34.36 21.50 12.58

Extraction Method: Principal Component Analysis.

Rotation Method: Varimax with Kaiser Normalization.

a. Rotation converged in 6 iterations.

Reliability

Reliability refers to the consistency of the instrument. In this

paper, the Cronbach alpha was used to evaluate the

consistency of the responses for items with the corresponding

dimensions. An alpha value of 0.7 to 0.8 is considered

satisfactory for social science researches (Nunally &

Bernstein, 1994). The Alpha values for the various items of the

instruments are shown in Table 4.

13235 Nagarajah et al./ Elixir Early Childhood Education 58A (2013)

13230-13234

CONCLUSION

The literature review has helped to raise the awareness of the

need of preschool assessment as it is able to serve various

purposes including screening for developmental delays,

evaluation of programmes used in the preschools, support for

teachers’ professional development and importance of parental

involvement. Besides being purposeful, assessment tools have

been identified by NPSC, so are the tools valid and reliable? Here

again, the role that the preschool educators play not only in using

the appropriate tools to measure different domains of

development but also correct reporting is important as it could

have adverse effects on their students’ progress.

However, it is felt that an understanding of the developmental

domains of children from birth to eight years old is fundamental in

establishing the rubrics in the assessment tools. The theorists like

Piaget, Vygotsky, Dewey, Erikson and Gardner have identified the

milestones in children’s development which indirectly help to

establish certain “benchmarks” to validate children’s

performances

The National Preschool Standard Curriculum, the informal

assessments and the teacher training are known factors in

establishing quality preschool education. The question is how to

develop a reliable and valid system that can account for students’

holistic development, relevant feedback towards curriculum

effectiveness and last but not least – what encompasses a

preschool assessment system? What is the best was of

implementing the assessment system?

In this research the researchers had provide substantial evidence

where a well design web based assessment system could

possible an option to consider.

REFERENCES

Ackerman, D.J. (2005) Getting Teachers from here to there:

examining issues related to an early care and education

teacher policy. Early Childhood Research and Practice,

Vol. 7, No 1 pp33

Ahire, S.L., Golhar, D.Y., Waller, M.A. (1996), "Development and validation of TQM implementation constructs", Decision Science, Vol. 27 No.1, pp.23-56.

Barnett W. S. (rev. Dec 2004) Better Teachers, Better

Preschools: Student Achievement Linked to Teacher

Qualifications. Preschool Policy Matter, 2. New

Brunswick, NJ: NIEER.

Bowers, F. B. (Nov/Dec 2008) Developing a child assessment

plan: an integral part of program quality, Assessment

and Documentation, Childcare Exchange.

Chang, N. (2010, September 27). Quality education for all: Early

education for children imperative for nation building.

The Star; Focus (F7).

Churchill, G.A. (1979), "A paradigm for developing better measures of marketing constructs", Journal of Marketing Research, pp.64-73.

Epstein, A. S., Schweinhart, L. J., DeBuin-Parecki, A. and Robin,

K. B. (July 2004) Preschool Assessement: A Guide to

developing a balanced approach. Issue 7, New

Brunswick, NJ: NIEER.

Hair, J.F., Anderson, R., Tatham, R., Black, W.C. (1995),

Multivariate Data Analysis with Readings, Prentice Hall International, Englewood Cliffs, NJ.

Gober, S. Y., (2002). Six Simple Ways to Assess Young

Children. Wadsworth Publ. USA.

Guddemi, M., Case, B. J. (Feb 2004) Assessing Young

Children, Assessment Report, Pearson Education.

Layton, C. A. and Lock, R.H. (January 2007)Use authentic

assessment techniques to fulfill the promise of No Child

Left Behind, Intervention in School and Clinic, Vol.42,

No.3, pp169 – 173

McCauley, L. The Developmental Assessment of Young

Children: a practical and theoretical view. Retrieved

16/7/11. priory.com/psch/assessyoung.htm

Puckett, M. B., Black, J. K. (3rd ed. 2008). Meaningful

Assessments of the Young Child: Celebrating

Development and Learning. Pearson Education, NJ

Stipek, D. (June 2005) Accountability come to preschool: Can

we make it work for young children? Phi Delta Kappan

87(10),740-744,747

Sureshchandar, G.S., Rajendran, C., Anantharaman, R.N.

(2002), "Determinants of customer-perceived service

quality: a confirmatory factor analysis approach", Journal

of Service Marketing, Vol. 16 No.1, pp.9-34

Wortham, S.C. (4th ed 2005). Assessment in Early Childhood

Education, Pearson Edu. NJ