Chapter 4.4

Observation Measures for Determining Attitudes and Competencies Towards Technology

Renate Schulz-Zander, Michael Pfeifer, & Andreas Voss

Institute for School Development Research (IFS), University of Dortmund, Germany

Abstract

This chapter provides an insight into how observational measures can contribute for deter-mining attitudes and competencies towards Information and Communication Technology. To get an idea of the capability of observational measures the first part of this chapter outlines the meaning of observational techniques as a tool of research and explains what is important when planning an observational study. For a better understanding in this context, several video- and audio-supported observation techniques are presented, as well as advantages and challenges of observational techniques in general. The second part of this contribution focuses on the methodology and on particular findings of ERIC-referenced empirical studies that researched attitudes and competencies towards IT by using observational techniques. After reviewing those studies and summarizing their findings, this chapter ends with the con-clusions that observational measures as part of a mixed methods research design have the potential to deliver meaningful, unique data and findings. One of the major findings is that at-titudes and competencies toward IT tend to be positive in most of the reviewed studies.

Keywords: observation measures, data triangulation, non-standardized research, attitudes/ competencies towards IT

1. Introduction

During the past quarter century major investments have been undertaken in most OECD

countries to bring computers into classrooms. As a result of this effort a rapid improvement in

student-computer ratios could be seen (see OECD, 2004; 2006). In most OECD countries

primary as well as secondary schools have access to computers and educational software

applications in their classes or in specially facilitated computer classrooms.

In the United States the No Child Left Behind legislation puts forward the need to continue

these efforts in the IT sector and makes substantial funds available through a performance-

based grant program (The White House, 2001). However, the grant program will also shift

emphasis from ensuring access to IT to assessing the impact that technology has on class-

room practices and on how much students learn at school (Dirr, 2003).

Until recent years empirical research on the effects of IT on students’ attitudes and compe-

tencies could not provide a clear picture about the relationship between IT access and stu-

dent academic achievement. The research studies show controversial results: For example

the 2006 OECD PISA report on computer attitudes declares an important effect on the extent

1

of computer use and student performance in mathematics (OECD, 2006). Reanalysis of

these results by the German PISA group showed that these effects are negligible if these

analyses are controlled for context variables in a longitudinal data analysis approach

(Senkbeil & Wittwer, 2006).

In order to address these controversies, alternative research approaches to determine the

impact of IT on classroom practice and student achievement need to be considered. The

challenge is to assess not only the extent of technology in the classroom but also the appro-

priate use of technology to facilitate student learning and to relate these observations to stu-

dent achievement.

One family of approaches is observation techniques. The main advantage of observation

techniques is that research objectives can be studied in their natural field settings (such as

classrooms) and thereby provide a richer understanding of the topic under investigation.

These techniques allow an alignment of classroom observation with the instructional context

and the evaluation goals. According to Dirr (2003) observation protocols – or forms of obser-

vation techniques – serve as valuable tools in order to:

Evaluate the effectiveness of a school program.

Assess the performance of a teacher or a school.

Provide feedback to teachers for professional development.

Conduct basic research on classroom practices.

These functions make observation a useful component of educational research. For exam-

ple, observation protocols can serve as a tool to measure whether the use of IT programs

achieved anticipated changes in teaching practice and how these changes are related to stu-

dent performance.

In this chapter we describe different observation techniques, present the rationale for obser-

vation techniques, and discuss their unique characteristics as well as advantages and chal-

lenges of this research approach. In addition we present examples of how observation tech-

niques can be used to study IT attitudes and competencies and we report a synthesis of find-

ings. We conclude the chapter by discussing how observation research techniques can be

enriched and combined with quantitative research approaches,

2. Observation as an Approach to Researching IT Competencies and Attitudes

In general observation is regarded as a “primary” data assessment technique due to the

closeness to everyday (non-scholarly) techniques in order to acquire information. In compari-

son to this “naive” kind of observation, the systematic form of observation under controlled

2

conditions by means of structured observation criteria. In the following section different types

of classroom observation measurement instruments are introduced and advantages as well

as limitations of these data assessment techniques are discussed.

2.1 Use of Observation Techniques as a Tool of Research

Observation measures in educational research have been used for decades. Substantial re-

search focusing on tracking classroom management and interactions that occur between stu-

dents and teachers has taken place since the 1970s.

Observation measures for determining attitudes and competencies toward technology in-

clude a wide range of approaches, e.g. checklists, inventories, time interval ratings, holistic

ratings, narrative descriptions, and logs. Some of these approaches have built on the prior

work of others, but according to Dirr (2003) there is little evidence of any attempt to bring co-

hesion to these efforts.

Observation techniques allow a systematic, structured data collection process, using well-de-

signed observation record forms. Compared with large-scale summative evaluation tools the

main advantage of these techniques is that data are collected in their natural setting, thereby

providing a richer understanding of the subject under investigation. In the instructional educa-

tional research context these techniques allow a researcher to view and monitor processes

that lead to student comprehension. The data can be used to explain students’ performance.

Observational studies in general go through several stages. The Center for Development In-

formation and Evaluation (1996) named a step-by-step list for observational techniques

which can be used to judge quality. These steps are named in logical order of the data col-

lection and analysis process.

Step 1: Determine the unit of instruction being studied

The first step is to decide which unit of instruction is to be studied. This consideration is es-

pecially important when the protocol involves third-party observers. Will they observe an en-

tire class period, a segment, an entire day, an entire course or program? Different protocols

are used to observe different units of instruction. Because of resource constraints, the obser-

vational approach has to be selective, looking at a few activities, events, or phenomena that

are crucial to the research questions. A good example of this point is the study of Wecker,

Kohnle and Fischer (2007). The research team chose an observation method which focused

on the activities on the computer screen and the users’ spoken comments regarding those

activities (see 3.2).

3

Step 2: Select or develop an observation form

Observation record forms help to standardize the observation process and to ensure that all

important items are covered. They also facilitate aggregation of data gathered from various

sites or by various investigators. Closed response categories (yes or no answers; check

boxes) help to minimize observer variation, and improve the quality of data. If the number of

items in a form is limited then the observation process is more managable. One important

question concerning this step is how the prior work form of other scholars can be integrated

into one’s own research questions.

Step 3: Specify a sample

Once the forms are ready, the next step is to decide where the observations will be carried

out and whether it will be based on one or more schools or classrooms. A single site obser-

vation may be justified if the class or school can be treated as a typical case or if it is unique.

However, single observations should generally be avoided. As a rule, several schools’

classes are needed to obtain a reasonable understanding of a situation (see Stigler & Gal-

limore, 2000). In SITES-M2 (Second Information Technology in Education Study - Module 2)

for example the countries selected one or more of the primary, lower secondary, or upper

secondary levels, with 4 cases for each level (Kozma, 2003). The cases were selected by a

national panel for their “innovativeness” and for what could be learned from them about how

technology is being used to support educational change.

Step 4: Decide on timing

Timing is a crucial point in the observational process, especially when events are to be ob-

served as they occur. Classes normally follow a fixed sequence. Poor timing can distort find-

ings (see Petko, Waldis, Pauli, & Reusser, 2003). Because most studies observe classes

continuously with at least two observers that record their findings independently on a form

and/or by the help of video-technology (see Schaumburg, 2001), the probability of missing

the key-moments when events occur can be decreased.

Step 5: Conduct the field observation

As previously mentioned, an observation team can provide more comprehensive and reliable

data than a single observer (see Schaumburg, 2001). If many classes are to be observed,

non-experts can be trained as observers, especially if observation forms are clear, straight-

forward, and mostly close-ended.

Step 6: Complete forms

The observer should take notes as inconspicuously as possible. The best time for recording

is during observation, however, is not always feasible because it may disturb the situation. In

such cases, recording should take place as soon as possible after observation. For example,

4

McInerney, Mc Inerney, Lawson, and Jacka (1996) used a procedure where the instructor

also took the role as an observer and recorded his observations in a weekly tutorial diary

(see 3.2).

Step 7: Analyze the data

Observation data from close-ended questions can be analyzed using basic procedures such

as univariate and bivariate statistics. Qualitative observations need to be coded in reference

to specific categories with respect to the research questions. Finding these aspects and ar-

ranging them together in a system of codes is an important process. The “cyclical analytical

process” includes viewing, coding, and analyzing the data aimed at transforming the video

images into objective and verifiable information (Jacobs, Kawanaka, & Stigler, 1999). Usually

the coding process is repeated several times because in viewing the videotaped material

new aspects might occur important from the researcher’s perspective (also see Stigler, Gal-

limore, & Hiebert, 2000; Dirr, 2003; Petko, Waldis, Pauli, & Reusser, 2003; Horsley & Walker,

2003). For the coding itself software such as videograph (www.ipn.uni-kiel.de) or ATLAS.ti

(www.atlasti.com) can be used.

Step 8: Check for reliability and validity

Observation techniques are susceptible to error and bias that can affect reliability and valid-

ity. Errors can be minimized by following some of the procedures suggested, such as check-

ing the representativeness of the sample of schools or classes selected; using close-ended,

unambiguous response categories on the observation forms, recording observations

promptly, and using teams of observers at each site (see Wecker, Kohnle, & Fischer, 2007).

2.2 Video- and Audio-supported Observation

In the early 20th-century, one can find the roots of present observation approaches. However,

early 20th century approaches are now supported by high-end technical devices. Video was

originally used as a stenographic recording method (Stevens, 1910 cited by Stigler, Gal-

limore, & Hiebert, 2000). The video supported observation was used more than 40 years ago

for the first time in the context of educational research. Now this research approach receives

much attention (Leinhardt, & McCormick, 1996; Stigler, Gonzales, Kawanaka, Knoll, & Ser-

rano 1999; Aufschnaiter, 2001; Ulewicz & Beatty, 2001; Petko, Waldis, Pauli & Reusser,

2003; Seidel, Prenzel, & Kobarg, 2005). Recording audio data is another observational

method. Depending on the research questions a combined use of videotaping and audio-

recording can be useful since multiple types of data enable the data to be triangulated and

thereby increase confidence in the conclusions drawn.

A recent approach to recording observational data is to record user activities on a computer

screen (Wecker, Kohnle, & Fischer, 2007). This approach can be adequate, for example, for

5

investigating students’ competencies while using a computer. Compared to videotaping

teachers’ or students’ behavior, screen recording captures all desktop activities. It can deliver

more data about the individuals in detail due to the separately recorded data on each com-

puter. In contrast to the observation with a single video camera this approach can deliver

data from every single computer and its user(s). For capturing the activities on a computer

screen, software has to be installed like Snapz Pro (MAC) (www.ambrosiasw.com) or Camta-

sia and SnagIt (PC) (www.techsmith.com). If a microphone is installed near the computer the

software is able to record the subjects’ spoken comments during their activities with the com-

puter.

The latest approach to gaining data on attitudes is the implementation of Face Reader-Soft-

ware (www.noldus.com). This software automatically recognizes specific properties in facial

images, including the following emotional expressions: happy, sad, angry, surprised, scared,

disgusted, and neutral. The emotions are represented as bar graphs and as a continuous

signal. An additional gauge display analyzes how negative or positive the emotion is. This

software is currently not a substitute for a human coder but it might be a good support for

precoding video data in respect of specific aspects. A human coder/researcher can select

such precoded video sequences and analyze them more deeply in the context of the whole

video. The classifications made by the Face Reader can be exported to analytical and data-

base programs for further analyses.

2.3 Advantages and Challenges of Observation Research Techniques

Observational data deliver a much wider spectrum of information, which expands the re-

searcher’s ability to analyze complex human interactions such as those in a classroom set-

ting. When using a video-supported observation approach, the analysis of complex human

interactions can be conducted more easily because the recorded video data can be analyzed

again and again. Through this set of techniques unanticipated ideas and alternative analytic

categories can also be discovered. The same set of video data might provide a focal point for

interdisciplinary collaboration. In addition, data gained from video-supported observation al-

low the integration of qualitative and quantitative methods of analysis (Stigler, Gallimore, &

Hiebert, 2000).

When conducting an observation in a classroom there are also challenges (Dirr, 2003). Each

observer approaches the classroom with his or her own experiences and biases. As a result,

two observers may focus on different aspects of the classroom and thus they may record dif-

ferent phenomena for the same lesson. Also every school environment provides a different

context in which the observations take place. This variation can affect the record-keeping be-

6

haviors of the observers. One or more observers may have an effect on classes e.g., on the

behavior of teachers and students. The observed lessons can vary from the “normal” situa-

tion and thus lead to a falsification of the data. Unless the observers and coders are given

extensive training, reliability among their observed and rated data can be low.

Classroom observations, data recording, and data analysis are very time-consuming. There-

fore the researcher has to carefully consider which data are needed with respect to the re-

search questions.

3. A Synthesis of Empirical Research Results

The U.S. No Child Left Behind Act (The White House, 2001) emphasizes that the use of

technology in the classroom can be a tool to improve academic achievement. Such legisla-

tion emphasizes the fundamental need to gain more knowledge about how integration of IT

in classrooms can be realized. In this respect research on attitudes and competencies to-

ward IT is a major topic for study. Observation techniques may provide a deeper insight into

learner attitudes, compared to standardized questionnaires.

In the following section we present the results of an ERIC search of studies on attitudes and

competencies toward information technologies that used observational techniques. Our ma-

jor goal is to provide insight on what kind of observation techniques exist and how they are

implemented in studies. Furthermore we will give examples of findings attained by the use of

observation measures and advantages that accrued compared to other methodological ap-

proaches. We will synthesize the findings of these studies to put together a picture of what

we can learn from using observation techniques in understanding teachers’ and students’ at-

titudes and competencies towards technology.

We followed Whitley’s (1997) approach to locate studies. As a first step we conducted a

computerized search the ERIC database from their inception through abstracts available as

of June 1994. We conducted the search using the term 'comput* and attitud*', in which '*' is a

wild-card character that institutes a search for any word having the designated stem. As a

second step the reference lists of prior literature reviews were examined for relevant sources.

Finally, we searched the reference lists of articles located by the other means.

The ERIC inquiry results on this issue produced a few results based on the keywords obser-

vation/measurement, qualitative research, participant observation, data collection, and social

research. The search also produced a small number of results for the keywords attitudes,

students and observation. The largest number of hits was reached with the keywords com-

puter-capabilities, attitudes, computer use in combination with a second set of keywords

7

computer attitudes, computer literacy, computer uses in education, that were added to the

search by using OR-operators.

3.1 Review of Researches on Attitudes toward Information TechnologyMany studies have used the “traditional” observation-method, conducted by one or two per-

sons who joined the observation group and filled out observation forms. This approach of in-

vestigating attitudes is still quite common although the influence of new observation tech-

nologies is continuingly growing. A more elaborate way to gain data on attitudes is to use a

triangulation approach. This means the interpretation of the acquired data through observa-

tion methods by involvement of other qualitative data (e.g. interviews) or quantitative data

(e.g. questionnaires) collected in parallel. Several studies used mixed methods in order to

capture the richness, complexity, and interdependence of events, actions, and conditions in

the real classroom. Observation methods were also applied to corroborate interview or sur-

vey data. Combining data from more than one source helps to detect possible biases in sub-

jective reports. By using triangulation for the interpretation of data, the probability of getting

objective, reliable and valid results is much higher than simply using a single approach.

In the following the methodology and findings of selected studies that use a triangulation ap-

proach will be presented. It should be mentioned that in several studies research on attitudes

is just one aspect of the research goals. In many studies research on attitudes is also com-

bined with research on competencies.

One study carried out by Schulz-Zander and Preussler (2005) at the upper secondary level

used a triangulation approach to evaluate a new state program in North Rhine-Westphalia,

which combined self-regulated learning methods with a problem oriented approach to mathe-

matics Data were gained from principal, teacher and student interviews, a student survey

and classroom observations, conducted by two persons and videorecorded.

Students attitudes towards ICT, cooperation, independent learning, academic, computer-re-

lated and mathematic-related self-efficacy were gathered from questionnaires combined with

interview data. The observations were used to support information gained from these data.

The triangulated data indicated increased self-regulated learning in general, caused by the

new developed materials and learning arrangements. In addition, the study showed signifi-

cant gender differences. More male students assessed the digital media as supportive for

their learning processes than female students. Males also preferred ICT for self learning,

and felt mathematics more interesting and better visualized through digital media than fe-

8

males, who preferred traditional media. More females had problems with self-regulated learn-

ing or functioning as a tutor. Females felt overly challenged to master the difficulties, espe-

cially when they did not collaborate with others. Thus, more support was required from the

teacher. Also, significant gender differences related to the academic, mathematic-related and

computer-related self efficacy were found. Males had a significantly higher self-efficacy than

females. High performaning students (mostly males) coped better with the new concept of

mathematics problem orientation, self-regulation, and ICT use than low performing students

did. The findings indicate a cumulative effect for low performing students (see also Preussler

& Schulz-Zander, 2004).

Schaumburg and Issing (2002) carried out a study with a longitudinal pretest-posttest control

group design to investigate the impact of personal mobile computers in the classroom of a

German grammar school (“Gymnasium”) (see also Schaumburg, 2001; 2002). Subjects

covered were mathematics, German, and English as a foreign language. To investigate

changes in teaching strategies and classroom practices, the observational data were col-

lected to confirm data (e.g. attitudes towards ICT, teachers and students role, self efficacy)

gained from teacher and student interviews and student surveys. Researchers observed the

same classes (Grade 7, 8, and 9) and the same teachers in lessons with and without laptop

use. A randomly selected sample of lessons of the laptop classes was videotaped over 2 ½

years. The video-supported observations were conducted by two persons, one who did the

videotaping and a second person who simultaneously conducted a non-technical unstruc-

tured observation of what was happening in the class. For each interval of 5 minutes length,

two trained observers recorded the dominant media use and form of instruction. Schaumburg

and Issing combined their observations with interviews of the teachers from their observed

lessons to corroborate the information they would get from their observational data (triangu-

lation).

Using teacher interview data, Schaumburg (2002) identified five types of teachers, based on

their attitudes toward teaching with laptops: type 1 focused on teacher-centered instruction,

type 2 focused on media literacy/ technics, type 3 focused on curricular/ content, type 4 fo-

cused on didactics and methods, and type 5 supported a constructivist approach. Teachers

of type 4 and 5 composed the minority.

Results gained from questionnaires and interviews were confirmed through observations with

respect to increased independence of students’ work during laptop lessons. But, the study

could not confirm increased collaborative classroom activities in the laptop lessons for all

three cohorts, when compared to lessons without laptops. The same was true for the shift

9

from teacher-centered instruction to student-centered learning through laptop use. The video

analysis showed that the change was less profound than the teacher interviews and student

survey data indicated. The amount of independent student work increased significantly dur-

ing laptop lessons, but for teamwork, pair work, lectures and teacher-guided discussions no

significantly differences were ascertained. Yet, teachers and students felt that the laptops

were valuable tools for teamwork and facilitated collaboration. With respect to the different

types of teachers, Schaumburg found out that only teachers of type 2, 3 and 4 displayed a

change in their teaching style. Type 2 and 4 increased the group work in their classes. Type

2 teachers exhibited a more effective acquisition of computer compentency. Only type 4

teachers practiced collaboration related to content. Solely type 4 could be identified as hav-

ing made a substantial change of their teacher roles. These findings should be considered in

the context of the investigated school type which in general is more focusing an teacher-cen-

tered instruction and independent learning and less on cooperative learning.

Another study using the triangulation approach was conducted by Jaervenoja and Jaervelae

(2005). They came to the conclusion that an observation on its own cannot tell enough about

the students’ attitudes towards a computer-supported inquiry learning setting. The authors

used interviews with the students to support their observational data.

In contrast to the ordinary interpreting-process (see Schaumburg, 2001), Jaervenoja and

Jaervelae used a modified approach. By the help of the Noldus Observer software

(www.noldus.com) they transcribed their video-data as a first step. In a second step they

watched the transcripts again and wrote specific descriptions regarding the students’ emo-

tional and motivational expressions as well as volitional behavior. During the second phase

the researchers were more familiar with the data, and looked again at the videotapes with the

transcriptions they had made during the first phase and wrote specific descriptions of the

students emotional and motivational expressions and their volitional behavior.

The aim of Jaervenoja and Jaervelae was to understand how students describe their emo-

tional experiences and their attitudes in computer-supported collaborative inquiry.

The results show five main sources for student’s attitudes during the computer-supported in-

quiry learning. Self- and context-driven emotions were the most frequent sources of emo-

tional experiences but also task-, performance- and social-driven emotions were identified.

Jaervenoja and Jaervelae conclude that these results are in accordance with the idea that in-

dividuals bring their prior learning experiences and assumptions to the learning situation

(Higgins, 1990; Salonen, Lehtinen, & Olkinuora, 1998).

3.2 Review of Research on Competencies for Information Technology

10

In this section we will give an overview of ERIC-referenced articles concerning the research

on competencies for information technology.

The study of Schaumburg and Issing (2002) and the study of Haeuptle and Reinmann

(2006),both were the same in design (multi-methodological approach), data interpretation (tri-

angulation) and medium of intervention (notebook computer). The study of Haeuptle and

Reinmann (2006), is also a case study, carried out at a German secondary general school

(“Hauptschule”) in Grades 7 to 10. One of their research questions aimed at clarifying how

the implementation of notebook computers into classes in a school can contribute to im-

provement of student competencies. In their study they used the observation approach to

support the information that they received from student interviews. Especially when trying to

gather information about competencies the observational approach can deliver data to make

visible processes of acting and reacting. The study comes to the conclusion that the imple-

mentation of notebooks into the classroom improves the student IT and social competencies.

The students also developed different strategies in working with and using the notebooks.

Furthermore the students showed their willingness to learn more about ICT.

The purpose of the study of Ilomaeki and Rantanen (2007) was to examine the development

of students’ high-level computer skills and compentency (student expertise) in information

and communication technology (ICT). Design (multi-methodological approaches), data inter-

pretation (triangulation) and also the medium of intervention (notebook computers) were the

same as in the previously presented study. In combination with other methods, classroom

observation was used in this study to describe the classroom activities concerning ICT and

student participation in them. This study used a longitudinal design. Data collection (e.g. ob-

servational data, questionnaires) was repeated several times, mostly with a duration of at

least one year in between. Through this approach it became possible to see changes hap-

pening over a longer period, e.g. in the students’ ICT compentency caused by the implemen-

tation of notebook computers.

Similar to the results of Haeuptle and Reinmann (2006), Ilomaeki and Rantanen also came to

the conclusion that student ICT compentency increased through the use of notebooks in the

classroom. But in addition they identified differences in the students’ degree and type of ICT

compentency during their three-years of research. They identified a sub-group of students

with a special technical interest in ICT but also a sub-group that used ICT as a tool for their

own creativity and human interests. Ilomaeki and Rantanen also came to the conclusion that

the degree of student ICT compentency depended on the students’ personal interest and

motivation – for instance the time the individual student spent using the internet at home.

11

McInerney, Mc Inerney, Lawson and Jacka (1996) conducted research on the question of

how a cooperative group setting influences a student’s computer compentency training, in

comparison to a direct instruction setting. The instructor, who was teaching both groups, also

was the observer who provided observational data. After the lessons the instructor recorded

his observational findings in a weekly tutorial diary, which contained several categories of in-

terest. This tutorial diary was analyzed in a qualitative/intuitive way. Findings were supported

by student interviews (triangulation). The study came to the conclusion that an instructional

setting that boosts the development of self-regulation and peer-support reduces the students’

anxiety and increases his or her motivation by enhancing a sense of control and compen-

tency.

Finally we present a technical state-of-the-art observational approach. This research, con-

ducted by Wecker, Kohnle and Fischer (2007), tried to clarify the question of whether higher

computer compentency – more specifically greater procedural computer-related knowledge,

higher familiarity with computers, and higher self-confidence in using a computer – might be

associated with greater acquisition of knowledge. They collected data by using question-

naires in combination with a screen observation. Screen observation means observing all the

activities that occur on the computer screen by using screen capturing-software. Wecker,

Kohnle and Fischer also recorded audio using an integrated microphone in the computer. It

then became possible to interpret what is seen on the screen by combining this data with the

information that the computer users provide by commenting consciously or unconsciously on

their activities. Wecker, Kohnle and Fischer did the transcription/coding of the video/audio-

data with the help of the Videograph-software. This technically-supported observation ap-

proach provides new and innovative possibilities for conducting nonparticipating observation.

The findings of this study do not indicate significant positive relations between procedural

computer-related knowledge or self-confidence in using the computer and knowledge acquis-

ition. Notably students having a greater familiarity with IT achieved significantly less know-

ledge. Wecker, Kohnle and Fischer interpret these findings in the context of the patterns of

media use by different navigation styles adopted by students with high and low familiarity

with computers. Students with high familiarity with IT exhibited more shallow processing

strategies ("browsing") which were less functional for learning.

4 Conclusions

The use of observational methods has changed during recent decades from non systematic

and, non-technical to a more objective and reliable, technically-supported method. Based on

the review of research on attitudes and competencies of the ERIC database it can be stated

12

that most studies used other methodological approaches (e.g., questionnaires or interviews)

as well as observations.

For measuring attitudes and competencies the main value of observational measures is

added capability for clarifying explorative research questions. Observational methods can de-

liver meaningful, unique data. Especially when using observational data in the context of data

triangulation in combination with other methodological approaches (qualitative as well as

quantitative approaches) – as was practiced in all our reviewed studies – observational in-

vestigations are able to provide valuable information about the conditions of the natural field

settings, teaching arrangements, classroom activities and the individuals in detail that help to

make an overall interpretation of the results more objective, reliable and valid.

References

Aufschnaiter, S. v. (2001). Nutzung von Videodaten zur Untersuchung von Lehr-Lern-Prozessen : aktuelle Methoden empirischer pädagogischer Forschung [The use of videodata for the research of learning processes: New methods for empirical educational research]. Münster u.a., Germany: Waxmann.

Center for Development Information and Evaluation. (1996). Performance Monitoring and Evaluation Tips. Using Direct Observation Techniques. Washington, DC: Center for Development Information and Evaluation (CDIE).

Dirr, P. J. (2003). Classroom Observation Protocols: Potential Tools for Measuring the Im-pact of Technology in the Classroom. Policy and Planning Series, Appalachian Tech-nology in Education Consortium.

Haeuptle, E., & Reinmann, G. (2006). Notebooks in der Hauptschule. Eine Einzelfallstudie zur Wirkung des Notebook-Einsatzes auf Unterricht, Lernen und Schule [Laptops in a school for secondary education: a case study of the implications of the use of notebooks on teaching, learning and the school]. Augsburg, Germany: University of Augsburg, Germany. Retrieved June 17, 2007, from http://imb.phil.uni-augsburg.de/imb/system/files/Notebook-Klassen_Abschlussbericht.pdf

Higgins, E. T. (1990). Personality, social psychology, and person-situation relations: Stand-ards, and knowledge activation as a common language. In L. A. Pervin (Ed.), Hand-book of personality: Theory and research (pp. 301-338). New York: Guilford.

Horsley, M., & Walker, R. (2003). Video based classroom observation systems for examining the use and role of textbooks and teaching materials in learning. Bratislava, Czech: IARTEM.

Ilomaki, L., & Rantanen, P. (2007). Intensive use of ICT in school: Developing differences in students' ICT expertise. Computers and Education, 48, 1, 119-136.

Jacobs, J. K., Kawanaka, T., & Stigler, J.W. (1999). Integrating qualitative and quantitative approaches to the analysis of video data on classroom teaching. International Journal of Educational Research, 31, 717-724.

Jaervenoja, H., & Jaervelae, S. (2005). How students describe the sources of their emotional and motivational experiences during the learning process: A qualitative approach. Learning and Instruction, 15, 465-480.

Kozma, R. B. (2003). Technology, innovation, and educational change - A global perspect-ive. A report of the Second Information Technology in Education Study: Module 2. Eugene, OR: International Society of Technology in Education.

Leinhardt, G., & McCormick, A. (1996). Videotape and audiotape recording in educational re-search. In T. Plomp & D. P. Ely (Eds.), International encyclopedia of educational re-search (2nd ed., pp. 568-572). Cambridge, UK: Cambridge University Press.

13

McInerney, V., McInerney, D., Lawson, R., & Jacka, C. (1996). Students' attitudes towards cooperative, self-regulated learning versus teacher directed instruction in a computer training course: A qualitative study. Paper presented at the Annual Meeting of the American Educational Research Association, New York.

Organisation for Economic Cooperation and Development (OECD) (2004). Completing the foundation for lifelong learning. An OECD survey of upper secondary schools. Paris: Organisation for Economic Cooperation and Development.

Organisation for Economic Cooperation and Development (OECD) (2006). Are students ready for a technology-rich world? What PISA studies tell us. Paris: Organisation for Economic Cooperation and Development.

Petko, D., Waldis, M., Pauli, C., & Reusser, K. (2003). Methodologische Überlegungen zur videogestützten Forschung in der Mathematikdidaktik [Methodological considerations for video-supported research in mathematics pedagogy]. Ansätze der TIMSS 1999 Video Studie und ihrer schweizerischen Erweiterung. ZDM, 35, 6, 265-280.

Preussler, A., & Schulz-Zander, R. (2004). Selbstreguliertes Lernen im Mathematikunterricht – Empirische Ergebnisse des Modellversuchs SelMa [Self-regulated learning in teaching mathematics - empirical findings of the SelMa model]. In F. Schumacher (Ed.), Innovativer Unterricht mit neuen Medien. Ergebnisse wissenschaftlicher Begleitung von SEMIK-Einzelprojekten (pp. 119-141). Grünwald: FWU Institut für Film und Bild in Wissenschaft und Unterricht.

Salonen, P., Lehtinen, E., & Olkinuora, E. (1998). Expectations and beyond: The develop-ment of motivation and learning in a classroom context. Advances in Research on Teaching, 7, 111-150.

Schaumburg, H. (2001, November). The impact of mobile computers in the classroom--Res-ults from an ongoing video study: 7. Proceedings of Selected Research and Develop-ment [and] Practice Papers Presented at the National Convention of the Association for Educational Communications and Technology (Vol. 1-2), Atlanta, GA. Schaumburg, H. (2002). Konstruktivistischer Unterricht mit Laptops?Eine Fallstudie zum Einfluss mobiler Computer auf die Methodik des Unterrichts[Constructivist learning with laptops: a case study on the impact of mobile computers on the teaching methods?]. Retrieved June 17, 2007, from http://www.diss.fu-ber-lin.de/2003/63/

Schaumburg, H., & Issing, L.J. (2002). Lernen mit Laptops: Ergebnisse einer Evaluationsstudie [Learning with laptops: Findings of an evaluation study]. Gütersloh, Verl: Bertelsmann Stiftung.

Schulz-Zander, R., & Preussler, A. (2004). Selbstreguliertes und kooperatives Lernen mit digitalen Medien – Ergebnisse der SITE-Studie und der SelMa-Evaluation [Self-regulative and collaborative learning with digital media - findings of the SITE study and the SelMa evaluation. In B. Bachmair, P. Diepold & C. de Witt (Eds.), Jahrbuch Medienpädagogik 4 (pp. 211-228). Wiesbaden: VS Verlag für Sozialwissenschaften.

Seidel, T., Prenzel, M., & Kobarg, M. (Eds.). (2005). How to run a video study. Technical re-port of the IPN video study. New York, München, Berlin: Waxmann.

Senkbeil, M., & Wittwer, J. (2006). Do computers influence the development of mathematical performance? In PISA 2003. Untersuchungen zur Kompetenzentwicklung im Verlauf eines Schuljahres. PISA-Konsortium. Deutschland: Münster, Waxmann.

Stigler, J. W., & Gallimore, R. (2000). Using video surveys to compare classrooms and teaching across cultures: Examples and lessons from the TIMSS video studies. Educational Psychologist, 35, 2, 87-100.

Stigler, J. W., Gallimore, R., & Hiebert, J. (2000). Using video surveys to compare classrooms and teaching across cultures: Examples and lessons from the TIMSS video Studies. Educational Psychologist, 35, 87-100.

Stigler, J. W., Gonzales, P., Kawanaka, T., Knoll, S., & Serrano, A. (1999). The TIMSS 1995 videotape classroom study: Methods and findings from an exploratory research pro-ject on eighth-grade mathematics instruction in Germany, Japan, and the United States. (NCES 1999-074), U.S. Department of Education. Washington, DC: National Center for Education Statistics.     

14

The White House. (2001). No child left behind: Transforming the federal role in education so that no child is left behind, Washington, DC.

Ulewicz, M., & Beatty, A. (2001). The power of video technology in international comparative research in education. Washington, DC: National Academy Press.

Wecker, C., Kohnle, C., & Fischer, F. (2007). Computer literacy and inquiry learning: When geeks learn less. Journal of Computer Assisted Learning, 23, 2, 133-144.

Whitley, B. E., Jr. (1997). Gender differences in computer-related attitudes and behavior: A meta-analysis. Computers in Human Behavior, 13, 1, 1-22.

15