on some sources of metacomprehension

Scand. J . Psychol22, 17-25, 1981

On some sources of metacomprehension

YVONNE WAERN SUSANNE ASKWALL

University of Stockholm, Sweden

Waern, Y. & Askwall, S.: On some sources of metacomprehension. Scandinavian Journal of Psychology, 1981,22, 17-25. Metacomprehension is defined as readers’ evaluations of their performance on a task, involving inferences derived from a text. In two studies metacomprehension was related to characteristics of the inferences required, in terms of 1) amount of information to be kept in long-term memory (Study I ) , 2) amount of information to be kept in short-term memory (Study 2) and 3) existence of negatives (Study 2). In the first study, 41 psychology students read a text and afterwards judged 1) the correctness of a set of pragmatic inference statements, and 2) their confidence in being correct or the difficulty ofjudging each statement. In the second study, 81 high-school students read a text and simultaneously judged 1) the correctness of a set of logical inference statements and 2) their confidence in being correct and the difficulty of judging each statement. In both studies, metacomprehension was not significantly correlated with actual performance. The results indicate that one important source of metacomprehension consists in information processing load. In Study 1, long- term memory requirements represent this load, in Study 2, the existence of negatives. Yvonne Waern, Department of Psychology, University of Stockholm, Box 6706, S-11385 Stockholm, Sweden.

How are people able to tell that they understand a message? Such a judgment implies not only that they process the message per se, but that they also process their own processing of it in some way. How are people able to tell that they don’t understand a message? They will then have to attempt to process the message and detect that they have failed. Comprehension is a double-decker bus with “sheer” comprehension processes on the first floor and comprehension checking on the second.

Exploration of human cognition’s second floor began about ten years ago (Flavell, 1970). Since then, it has attracted enough interest to merit a name of its own, viz. “metacognition”. There are some excellent reviews of the topic (Flavell, 1977; Brown, 1978), so this paper will not attempt to make such a review. Instead we will extend previous research by elaborating on one particular aspect of metacognition, which arises from a twofold partition of the metacognitive problem.

The first partition delineates the observing process from the directing process. Cognitive proces-

2-81 1945

ses have to be adapted to task requirements as well as to the person’s limitations and capacities. Thus metacognitive processes must involve observations of the environment and the effects of the cognitive processes as well as directions to the cognitive processes as to how to proceed. This paper deals with the observing aspect, not the directing.

The second partition refers to the type of question posed in the research enterprise. Previous research has asked “how well”-questions. For example, interest has been directed toward “how well” people can predict or postdict their own learning results. Such studies have demonstrated metacognitive abilities and disabilities. A natural follow-up of how-well results is a how-question: How can people know when they understand and when they don’t? This is the question posed by the present studies.

The “how well” questions and their empirical data form a background for the present studies. Previous research has studied the relation between people’s actual performance and their own percep- tion of this performance. It can be expected (and

Scctnd. J . Ps?c.hol. 22

18 Y . Waern and S . Askwall

Table 1. Text with number of sentence inserted The number was not originally there

On Creative thinking

be adequately assessed, whereas in others it cannot, we pose the question: Do resultant differences in metacognitive judgments arise from differences in task characteristics? This ‘paper addresses the issue in two exploratory studies in the context of a 1. Creative thinking has always to emanate from some

2 .

3.

4.

5 . 6.

I .

8.

9.

10.

1 1 .

12.

13.

14.

15.

16.

material. The material of creative thinking consists of human experiences. Human experiences are not a copy of reality but a transformation of it. This depends on the fact that man does not passively receive his experiences, but shapes them himself from his actions in the environment. We can see this in the little child as well as in the adult. The little child pokes everything. The adult makes thinking experiments. By actively manipulating the environment the little child develops his world knowledge as well as the researcher develops human world knowledge by ex- perimenting. Actions and their effects do not become incorporated as wholes in the human experience. Instead they are analyzed as features which consitute the elements of the human knowledge. The more extensive the analyses are made, the more complex the world knowledge will become. Actions and their effects can be connected, compared and classified on basis of their features. In this way a structure is obtained which allows a simple description of the world knowledge. According as the descriptions of features and struc- tures are built up inside a person, he will be able to predict what kind of effect a certain action ought to have in a certain situation. If the action leads to an expected result, the world knowledge will not be changed. If the action on the other hand does not lead to the

comprehension task.

STUDY 1

The purpose of the first study was to study metacomprehension in relation to a connected, natural text. In order to study the relation between kind of item and metacomprehension, the comprehension task was divided into different sets of items.

Method Materials. A text about creative thinking representing a summary of a particular view about creative thinking was used. AnEnglish translation ofthe text is given in Table 1 .

In order to study comprehension, a set of statements were constructed. The statements represent different kinds of inferences and can be regarded to differ from one another in the following ways: (1) Some inferences are implied by the text, others are contradictory or irrelevant to the text. (2) Some inferences cover a single sentence while others require integration of several sentences. For- ty-four statements were constructed. Examples of the statements are presented in Table 2, together with their classification, i.e. true, false or irrelevant, and number of sentences covered.

expected result, the world knowledge of the person will become unstable. This difference between expected and obtained result is the starting point of creative thinking.

Subjects. Forty-one first-year psychology students participated in the investigation. Although they were volun-

it should be noted that participation in a number of experiments is a course requirement.

Procedure. The subjects were met in groups of about

investigations bare this out), that the relation obtained differs from subject to subject in line with the cognitive task posed.

Differences in subjects’ performance have primarily been studied as developmental differences, and have repeatedly shown that young children are less able t o assess their own performance than older children (Brown, 1978; Flavell & Wellman, 1977).

Differences between tasks can be seen in studies of adults, where it h a s b e e n found that adults a re adequate a t estimating their own memory performance, but are notoriously bad at estimating their own judgmental capacity. An excellent analysis of the judgmental problem is presented by Einhorn & Hogarth (1978).

Finding that the performance in some tasks can

Suind. J . Pswbol. 22

ten. In the first step they were asked to read the text in order

to acquaint themselves with its content. The verbal instruction stressed skimming rather than close reading. A 2.5 minute time limit was set. Both the reading instruction and the time limit were chosen to avoid overlearning of the text. We wanted to get a fair range of comprehension scores, i.e. false answers as well as correct ones. Accord- ing to pre-trials, 2.5 minutes would allow enough time for all subjects to read through the text at least once.

In the second step, the subjects were instructed to re- produce the content of the text in their own words, as detailed as possible. The verbal instruction stressed detailed reproduction but not verbatim recall. No time limit was posed.

In the third step of the study, the students were re- quested in a written instruction to indicate whether or not the author of the text presented in step one would agree or disagree with the statements, constructed to assess comprehension. The statements were presented in different random order for each student, employing cards upon which each statement was written. The students were instructed to base their judgments of the author’s opinion

Table 2. Some examples of statements used in the comprehension test together with their classification

Statement

Number of True/false/ referred to irrelevant sentences

38.

15.

20.

17.

37.

35.

An individual needs to be able to analyse his experiences in order to be able to use them creatively True Experience is a necessary condition for creative thinking True An action which does not lead to a desired result will not be attached to anything in the conception of the world and thus will not change it An action which leads to a desired result in a certain situation will be connected stronger to this situation, which partly will change the conception of the world False A strong motivation is a prerequisite for creative thinking Irrelevant A thinking must solve new problems in order to be called creative Irrelevant

False

on the text, which was not available. Twenty-two of the students were also asked to rate the difficulty of judging the opinion of the author for each statement in turn, by choosing one of the three following alternatives: I had (a) no difficulty, (b) some difficulty and (c) great deal of difficulty in judging the opinion of the author. The remaining 19 students were asked to assess how confident they were about having given the right answer to the statement. The students were asked to indicate their confidence by choosing one of the four following alternatives: I am (a) certain, (b) nearly certain, ( c ) somewhat uncertain and ( d ) very uncertain that I have given the correct answer to the statement. No time limit was set for this part of the study.

Results At this point, only results concerning comprehension and difficulty/confidence ratings will be presented. Free recall will be treated elsewhere. The difficulty ratings and confidence ratings were transformed as follows: No difficulty= 1. Some difficulty=2. Great difficulty=3. Very uncertain= 1. Somewhat uncertain=2. Nearly certain=3. Cer- tain=4.

Metacomprehension 19

The relation between actual and perceived performance. Before going into detail, one can ask, how well actual comprehension and metacomprehension correspond. This correspondence was calculated as product moment correlations over items between the average difficulty o r confidence rating for a single item and the frequency of subjects answering that particular item correctly. These correlations were 0.07 for difficulty, 0.17 for confidence. None of these is significant ( n = 4 4 ) . These low correlations indicate that metacomprehension is not adequate.

Metacomprehension in true and false items. The next step consisted of analyzing metacomprehension inadequacies. One possible source of difficulty consists of the truth status of items. I t has previous- ly been found that false alarms present an important source of error in similar comprehension tasks (Waern, 1977; Schultz & Waern, 1978). Therefore, the comprehension items were divided into true and false or irrelevant items respectively (see Table 2). The authors agreed completely on this division. The proportions correct and average metacomprehension ratings for true and false items respectively are presented in Table 3. There are n o significant differences between the averages, neither in terms of performance, nor in terms of metacomprehension judgments.

The product moment correlations between comprehension and metacomprehension were calculated separately over true items and over false or irrelevant items. The results are given in Table 4.

Only one significant correlation was obtained, thus item truth status cannot be proposed as a source of metacomprehension in this study.

Metacomprehension and number of sentences to be integrated. As was mentioned in the methods

Table 3. Proportion correct and mean difficulty (MDR) and confidence (MCR) ratings for true and false or irrelevant items respectively

Group A" Group Bb

pro- pro- portion portion

Type of item MDR correct MCR correct

True' 1.78 .76 3.03 .66 False or irrelevantd 1.75 .60 3.13 .63

N(subjects)=22. * N(subjects= 19. cn(items)= 15. n (items)=29.

20 Y. Waern and S . Askwall

Table 4 . Product moment correlations between mean ratings of difJiculty and confidence respectively and number of correct anSWerSfor true and

Relation between difficulty and confidence. As was mentioned above, feelings of difficulty and confidence should not be related to each other in a

false or irrelevant items one-to-one ratio. Items may, of course be perceived to be easy, where confidence is high. Likewise items may be perceived to be difficult, where confidence is low. There can also be cases in which items are perceived to be difficult by the subject,

Type of item

True False itemsb Type of rating items“ and irrelevant

Difficulty .06 .05 Confidence .69* .03

n=15. ’ n = 2 9 . * p<O.Ol.

section, the comprehension items were judged to require different amounts of information in order to be integrated. In this study, we will call this requirement the “coverage” of an item. The coverage was obtained by analyzing the text together with the comprehension item, and calculating the number of sentences which had to be considered in order to answer the comprehension item correctly. This calculation was straight-forward and took no overall associations into account. Thus, the authors to a great extent agreed in judging the coverage of each comprehension item. Those disagreements which did arise were solved through discussion. The same calculation of coverage was also attempted for false items. Here however, it was found, that a false item could be rejected on different grounds, either by considering several sentences or by considering only one or a few sentences. The effect of coverage was therefore calculated for true items only.

The product moment correlations between coverage and average difficulty as well as between coverage and confidence were calculated on all true items (n=15) and are presented in Table 5 .

It is obvious that coverage has something to do with metacomprehension. At the same time it is seen that comprehension is not necessarily affected by coverage. Only the group that rated confidence showed a significant correlation between coverage and actual performance. With these results, it must therefore be concluded that coverage may affect metacomprehension, but not necessarily actual per-

who is highly confident of having overcome the difficulty. Thus correlation between difficulty and certainty should be attenuated as difficulties are overcome. The relation between difficulty and confidence was studied here by calculating the product moment correlation over items between the two groups of subjects. Hence a correlation of -0.67 was obtained. Considering the fact that this figure includes random errors, due to different subjects giving the two kinds of ratings, the figure must be considered to indicate a high relation between difficulty and confidence. Thus the case may be that difficulty and confidence are perceived as contrasts, and that difficulties are seldom overcome, or detected to have been overcome.

Discussion

This exploratory study into the field of metacomprehension has shown that the subjects were unable to adequately assess their own performance. Reasons for their inadequacy were pursued by ask- ing what information was used in their metacomprehension judgment.

The information for metacornprehension can be obtained in different ways. Experienced readers may recognize text characteristics and questions that cause difficulty. They can base their confidence and difficulty ratings on prior experiences. In addition, the question processing gives rise to ex-

Table 5. Product moment correlations between number of correct answers, mean d$ficulty and confidence rating and number of sentences covered by each true item (“coverage”)

formance in the comprehension task. I t can also be the case, of course, that difficulty and confidence Relation Difficulty Confidence ratings indicate different types of metacomprehension. If a difficulty is perceived but overcome, difi-

Group

CoveragelMetacomorehen- I

culty may not relate to actual performance. How- sion” .55* -.62** .64** ever, confidence must be related to performance, if Coverage’Performancen - ‘ I 9

metacomprehension is adequate. a n=15. * ~ 4 0 . 0 5 . ** p<O.OI.

Scond. J . Psycho/ . 22


Table 6. The text aboutfish in Tarktarktisk

About f ish in Tarktarktisk There are many large fish in the northern parts of Tarktarktisk, which attract researchers from all over the world. All the large fish are iceblue, a color that renders them almost invisible. This is probably one of the reasons why they were discovered so late. All the iceblue fish are longnosed, a fact which is significant from the standpoint of their survival. With the help of their long snouts, these fish can search for food in the many clefts and deep holes in the area. All the longsnouted fish are algae-eating fishes. They are especially well adapted to the prevailing climate and food circumstances, since many areas in Tarktarktisk are rich of algae.

All algae-eating fishes are deep-sea fishes. Deep-sea fishes mate only once a year, in the deep waters where they can peacefully spawn. All the deepsea fishes are pairfish. Pairfish take turns in watching the spawn. While one of the pair guards the spawn, the othp- collects food. All pairfishes migrate. The migrating fishes can be seen moving southward in April. By that time, the researchers also leave Tarktarktisk homeward bound to their laboratories.

periences which can be used as a basis for the ratings. The present results indicate that current experiences were used. This interpretation is based on the significant correlation between coverage and metacomprehension. Coverage must be assessed (or experienced) for each item in this particular situation. The coverage can be seen as a n indication of memory requirements brought about by each item. Information from the text has t o be remem- bered (long-term memory requirement) as well as integrated into a single inference (short-term memory requirement). One source of metacomprehension thus seems to be inherent in memory requirements, both long-term and short-term. The present study cannot answer the question of whether it is the difficulty to retrieve information which serves as source for metacomprehension, or the difficulty to hold information in short-term memory. Study 2 will attempt t o separate these two sources of information.

It is clear that this identified source of information is not sufficient to lead to adequate metacomprehension. The present study may lack refine- ments that could point t o possible weaknesses in metacomprehension. Study 2 will try a finer grain.

STUDY 2

The purpose of the second study was to refine both the method and procedure used in Study 1 in order

to obtain more precise answers. First, the subjec- tivity in describing the items was eliminated by using a logical inference task. This made it possible to explore in detail the different kinds of logical inferences along with their metacomprehension. Secondly, in the first study some interesting relations between memory requirements and metacomprehension arose, for which either difficulties in long-term memory retrieval or short-term memory limitations could be responsible. Study 2 facilitates the interpretation by eliminating long-term memory requirements. Third, the search for some reason behind inadequacy of metacomprehension will be pursued by comparing single items.

Method Materials. Two meaningful, but fictional texts were constructed. They were identical in their structure, but dif- fered in content. One of the texts dealt with birds in Baranda, the other with fish in Tarktarktisk. The textstructure consisted of a series of propositions about objects (birds and fish respectively). All the propositions had the following form: All X are Y. For example: All big fishes are ice-blue. Each proposition was connected to the next proposition in the series, by means of the second attribute mentioned, i.e.: All Y are Z. For example: All ice-blue fishes have long noses. In this way a structure of successive class inclusions was obtained. The text about fish in Tarktarktisk is presented in its entirety in Table 6.

Forty test-items were constructed in relation to the two texts. Thirty-two of these referred to the fish text and eight to the bird text. The test-items covered eight different logical forms. Two of these forms were logically correct while six were logically incorrect. The reason why less correct than incorrect items were constructed was that a subjectively even distribution of true and false items was pursued. Furthermore, test-items vaned in terms of number of text propositions to which they referred. Each item referred to at least one and at most four propositions. Griggs (1974) has shown that the first and the last proposition in a series similar to those that were constructed here are treated differently compared to the propositions in

Table 7. Main structure of texts and items belong- ing to the text aboutfish in Tarktarktisk

Main structure Item Main structure of text type Status of items

All 0 are P A AlIPareQ B All Q are R C All R are S D All S are T E AlITareU F

G H

True True False False False False False False

Q thus R Not R. Thus not Q R. Thus Q Not Q. Thus not R Not Q. Thus R R. Thus not Q Not R. Thus Q Q. Thus not R

Scmd. J . Psycho/. 22

22 Y . Waern and S . Askwall

Table 8. Percent correct answers (76) and mean d$ficulty (MDR) and mean confidence ratings (MCR) for each item, separating groups and number of propositions referred to

~~ ~

Group A"

1 prop. 3 prop. Item type % MDR MCR % MDR MCR

loo 73 32 50 98 93 95 95

1.4 4.5 1.9 3.9 1.7 4.1 2.2 3.6 1.8 4.0 1.8 4.1 1.8 3.9 1.6 4.1

82 50 32 57 loo 95 98 98

1.7 4.2 2.1 3.8 1.8 4.0 2.1 3.7 1.8 4.0 1.7 4.0 1.8 4.1 1.7 4.0

Group B

2 prop. 4 prop.

% MDR MCR % MDR MCR

94 1.5 4.1 100 1.8 4.2 88 2.2 3.4 67 2.3 3.4 18 1.9 3.9 12 1.7 4.0 39 2.3 3.4 36 2.3 3.4 97 2.2 3.6 88 1.8 3.6 97 2.0 3.7 loo 2.1 3.7 94 2.3 3.4 94 2.1 3.7 97 2.1 3.6 91 2.1 3.7

between them. Therefore the first and the last proposition in the series were excluded in the construction of the test-items. All the test-items were built up according to the following pattern: a statement about a fact followed by an inference. For example: This specimen is taken from a longnosed fish. That fish must have been ice-blue.

The test-items were divided into two sets of 24 items each (set A and set B). Set A consisted of the eight items that referred to the Baranda birdtext and 16 belonged to the Tarktarktisk fish text. Half of the latter referred to one single proposition in the text while the other half to three propositions. Set B consisted of the eight items about the Baranda bird text plus 16 items that referred to the Tarktarktisk fish text. Half of these items referred to two propositions in the text while the remainder to four propositions. In Table 7 the main text pattern as well as the test-items that were constructed in relation to the text about fish are presented.

Subjects. Eighty-one students from a senior highschool in Stockholm participated as subjects in the study. They were between the ages of 16 and 18, with approximate an equal distribution of males and females.

Procedure. The students were met in groups of about 20 at a time. The data was collected during regular school hours under the supervision of one experimenter. The data collection began with a general introduction about the purpose of the study, followed by more specific instruc- tions, which were both verbal and written.

The texts were said to be observations made by two researchers during their expeditions in two different fictional countries. It was pointed out that the texts were fictional. The test-items introduced were said to be con- siderations made by the two researchers after they re- turned from their trips. For each item the students were asked to perform the following:

1. Judge whether or not the inference drawn by the researcher was true or false in relation to the content of the text. The students were told to judge an inference as true, if it followed logically from the text or if it para- phrased the text. The students were instructed to judge an inference as false, if it did not follow logically from the text or if it stated something that could not be said in relation to the content of the text.

Scrind. J . Psycho/. 22

2. Rate the degree of difficulty in judging the inference. The students were asked to indicate the difficulty by marking one of five alternatives stating: I found it ( a ) very difficult, ( 6 ) difficult, ( c ) somewhat difficult, ( d ) easy and ( e ) very easy to judge the inference.

3. Rate how confident they were about having judged the inference correctly. The students were asked to indicate their confidence by marking one of five alternatives stating: I am ( a ) very certain, ( b ) certain, ( c ) somewhat uncertain, ( d ) uncertain and ( e ) very uncertain that I have judged the inference correctly.

The two texts together with the test-items were held in separate envelopes. Each item was written on a separate card. The different test-items were presented in a different random order for each student. The order in which the two ratings were performed was also randomized between subjects. Set A of the test material was distributed to 47 of the students (Group A) and set B to the remaining 34 students (Group B) that took part in the study.

The students were asked to start to work with the text about birds in Baranda. The main function of the Baranda text was to acquaint the students with the material of the study. While working with the bird text, the students were free to ask questions.

A 45-minute time limit for the data collection was set, during which the students were free to work at their own pace. If students completed the task before the 45-minutes were over, they were asked to complete a short questionnaire. The primary function of the questionnaire was to

Table 9. Product moment correlations over items between number of correct answers and average dijjficulty and confidence ratings

~~~ ~

Difficulty rating Confidence rating group group

A B A B

-.49** -.08 .42* - .07

a n=16. * p < O . I O . **p<0.05.


Table 10. Results of analyses of variance on logical forms

Group Aa Group Bb Dependent variable F F

Number of correct answers 34.14** 45.53** Difficulty rating 7.12** 6.70** Confidence rating 5.61** 6.94**

Note: Df for group A: 71301, group B: 71231. “n=44 . bn=34. **p<O.O1.

keep everybody busy and thus avoid unnecessary distur- bance.

Results The data from three subjects had to be discarded, because they did not complete the task.

The average metacomprehension judgments and the proportion of correct answers for each item are presented in Table 8. Each group is presented separately, as it could not be expected that the ratings are made on comparable scales. The difficulty and confidence ratings were transformed as follows: Very easy= 1 , easy=2, somewhat difficult=3, difficult=4, very dCicult=S. Very uncertain= 1, uncertain= 1, uncertain=f, somewhat uncertain=3, certain=4, very certain=5. This table gives the necessary background information to interpret the analyses of variance to follow later.

The relation between actual and perceived performance. The product moment correlations between proportion of correct answers and average difficulty and confidence ratings respectively are presented in Table 9.

There is no consistent trend in the correlations, one group showing significant relationship, the other not. The nonconsistent trend in the correlations may be explained by some skewed distribu- tions of the dependent variables. In both groups, the distribution of percent correct is negatively skewed, as is seen in Table 8. In group B, the difficulty and confidence ratings are also skewed, whereas in group A, these ratings are more sym- metrically distributed. Instead of relating the metacomprehension ratings to performance, therefore, it may pay more to analyze the sources of variation, both with respect to performance and metacomprehension. Such an analysis was performed here by separating different kinds of items.

Logical forms. The effect of logical form was

investigated by calculating analyses of variance for each group and in each of the following dependent variables: performance, difficulty and confidence ratings. The different amounts of information to be integrated were combined in these analyses. The F-values are presented in Table 10.

Table 10 shows that the differences between logical forms are significant for both groups in all three dependent variables. In order to analyze the source of variation somewhat closer, the items representing different logical forms were divided into two groups, one represented by items lacking negatives and the other group, representing items containing one or two negatives. Items A and C belong to the “positive” group, items B, D, E, F, G, H belong to the “negative” group. Testing the difference between the means in the positive and the negative groups, using Scheffe’s method of contrasting means, it was found that all contrasts were significantly different at the 0.05 level. The positives were correctly answered to a lesser degree than the negatives (cf. Table 8). At the same time, the positive items are subjectively easier and lead to greater confidence than the negative items (cf. Table 8). There seems to exist a source of information here, which is responsible for adequate as well as in- adequate metacomprehension. Negative items are sometimes objectively more difficult than positive items. The “sometimes” qualification in this study depends upon which negative and which positive forms are compared. Forms B and D (logically equivalent to the modus tollendo tollens and deny- ing the antecedent) result in more errors than form A (logically equivalent to modus ponens). The “obvious” contradictory forms E-H are however answered correctly to about the same extent as form A. Here, then, there is one possible source of inadequacy for metacomprehension: negatives may signal a difficulty which is not at hand (or very

Table I I . Product moment correlations over items between mean difficulty (MDR) and confidence ratings (MCR) within and between groups

Between groupso Within groups”

MDRvs. MCRvs. MDRvs. MDRvs. MDR MCR MCR MCR

.71* .74* - .93* - .92*

“n=16. *p<O.Ol.

Scnnd. J . Psrc.hoi. 22

24 Y . Waern and S. Asliwall

easily overcome). Also, it is seen from Table 8 that the objectively most difficult item, i.e. the one least correctly answered, is the positive “C”. This item corresponds to the logical form of confirming the antecedent from an affirmed consequent. This kind of error is extremely common in human reasoning. It is interesting to note that subjects perceive very little difficulty with this form. This, then, is a second source of inadequacy for metacomprehension: not to perceive a possible logical error as a problem.

Metacomprehension and number of propositions to be integrated. As in Study 1, the effect of having to integrate more or less information was studied. An analysis of variance was performed for each group, testing the difference between one and three and two and four propositions to be integrated, for each dependent variable (performance, difficulty and confidence ratings). None of these differences were significant, which indicates that amount of information to be integrated has no effect, either on performance, or on metacomprehension. This result conflicts with expectations and with Study 1 . Several explanations may be put forth. First, the amount of information to be integrated did not differ as much in this study as in the previous one. Thus, the requirements may not have been sufficiently strong to affect performance or experience. Sec- ond, in this study all material to be integrated was available to the subjects since they had access to the text. No long-term memory requirement was therefore posed. It can thus be suggested that sources for metacomprehension relate to long-term memory failures rather than short-term memory load.

Relation between d$ficulty and confidence. The present study allows for a separate calculation of interrater reliability and relation between difficulty and confidence. The product moment correlations between the two groups with regard to difficulty and confidence respectively was calculated by tak- ing the average confidence and difficulty rating over individuals in each group for each item and correlat- ing them by pairing items of corresponding form. The product moment correlations are thus based upon 16 items. The correlations are presented in Table 11, as well as the correlation between confidence and difficulty ratings over items in each of the groups.

It is evident that the subjects have given almost the opposite ratings when rating difficulty and confidence. Also, the reliability is satisfactory, consid-

ering the fact that raters as well as content of items (different number of sentences integrated) were var- ied.

GENERAL DISCUSSION

Both the present studies indicate that metacomprehension is not adequate in a task requiring inferences. Both studies also show that the adequacy of metacomprehension is related to characteristics of the comprehension items.

It must be suggested then that metacomprehension is not based upon performance. This is self- evident, since the subjects cannot assess their performance directly. Instead, metacomprehension seems to be based upon experiences evoked by the processing of the items. This proposal is supported by two kinds of results: the difference between positive and negative items in Study 2, and the effect of amount of information to be integrated in Study 1.

Both these results indicate that the more processing an item requires, the more difficult the subjects perceive the item, and the less confident they are in being correct. Statements containing negations require more processing than corresponding positive statements, as has been shown in studies using latencies as dependent variables, see e.g. Clark (1974). Also, when the amount of information to be integrated exceeds the amount of information possible to keep in short-term memory, more processing is required (compare for instance the times necessary to perform mentally the multiplications 4 by 3 and 43 by 24). It can also be suggested that the less available the information in long-term memory, the more processing will be required in retrieving it (by for instance reconstructing lost information).

If this assumption as to the relation between processing requirement and perceived difficulty is correct, then it would follow that items which do not require extensive processing are perceived as being easy. The two items perceived as being easiest are the “modus ponens” and the “affirming the consequent” items. There is some evidence that these two logical forms also are the ones which are most quickly answered (compared to the others used in the present study), see e.g. Evans (1977). Furth- ermore, the differences in memory loads support the assumption of effect of processing requirements. In Study 2, differences between memory loads are less than in Study 1 in two ways: amount

Scnnd. J . Psycho(. 22

Metacornprehension 25

of information to be integrated did not vary as much, and no long-term memory requirements were posed. Thus, even though a non-obtained difference is worth less than a significant difference, the overall result pattern supports the following conclusion: Assessment of own performance in a situation that does not allow for direct observation of it, is based upon experience of processing requirements in the task at hand.

If this conclusion is correct, it has educational consequence. There are two types of inadequacies that might impair efficient performance in a learning and comprehension task: to work too much with a nonproblematic material, and to work too little with a problematic material. The first effects the effi- ciency, the second the possibility of obtaining correct answers. The first lesson to be learned is that these types of inadequacies can be diagnosed. An investigation like Study 2 may easily be performed in the classroom, and be used as a diagnostic in- strument. Pupils might well differ in terms of which kind of inadequacies they show. The second step consists of devising treatment for the various inadequacies. The first inadequacy may be remedied by changing the material: it is often possible to find or construct materials that do not tempt to overdo- ing. The second inadequacy must be remedied by working with the pupil. A task perceived to be easy must be rendered problematic. The pupil must be made to see the hidden difficulties. Here the field is open for creative teachers.

The investigation has been supported by a grant from the Swedish Council for Research in the Humanities and So- cial Sciences. Data collection and treatment have been performed by Susanne Askwall. Study 2 is based upon a M.A. thesis by Susanne Askwall.

REFERENCES Brown, A. L. Knowing when, where, and how to re-

member: A problem of metacognition. In R. Glaser (Ed.), Advances in instructional psychology, Vol. 1 , pp. 77-165. Hillsdale, New Jersey: Lawrence Erlbaum Ass, 1978.

Clark, H. H. Semantics and comprehension. In T. A. Sebeok (Ed.), Current trends in linguistics, Vol. 12, pp. 1291-1428. The Hague, Pans: Mouton, 1974.

Einhorn, H. J., & Hogarth, R. M. Confidence in judge- ment: Persistence of the illusion of validity. Psychological Review, 1978,85, 395916.

Evans, J . St. B. T. Linguistic factors in reasoning. Quar- terly Journal of Experimental Psychology, 1977, 29, 297-306.

Flavell, J. H. Developmental studies of mediated memory. In H. W. Reese & L. P. Lipsitt (Eds.), Advances in child development and behavior, Vol. 5 , pp. 181- 211. New York: Academic Press, 1970.

Flavell, J. H., & Wellman, H. M. Metamemory. In R. V. Kail, Jr. & J. W. Hagen (Eds.), Perspectives on the development of memory and cognition, pp. 3-33. Hills- dale, New Jersey: Lawrence Erlbaum Ass, 1977.

Griggs, R. A. Logical errors in comprehending set inclu- sion relations in meaningful text. [Indiana Mathemati- cal Psychology Program, Report No . 74-71, 1975.

Schultz, C. B., & Waern, Y. World knowledge and the recall and comprehension of text.[Reportfrom the De- partment of Psychology, University of Stockholm, No. 5331, 1978.

Waern, Y. Comprehension and belief structure. Scan- dinavian Journal of Psychology, 1977,18,266-274.

Srand. J . Psyrhol. 22

on some sources of metacomprehension

Documents