the editing process in writing: a performance study of more skilled …€¦ · the composing...

The Editing Process in Writing: A Performance Study of More Skilled and Less Skilled CollegeWritersAuthor(s): Glynda HullReviewed work(s):Source: Research in the Teaching of English, Vol. 21, No. 1 (Feb., 1987), pp. 8-29Published by: National Council of Teachers of EnglishStable URL: http://www.jstor.org/stable/40171099 .Accessed: 30/11/2011 16:31

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

National Council of Teachers of English is collaborating with JSTOR to digitize, preserve and extend access toResearch in the Teaching of English.

http://www.jstor.org

http://www.jstor.org/action/showPublisher?publisherCode=ncte

http://www.jstor.org/stable/40171099?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp

The Editing Process in Writing: A Performance Study of More Skilled and Less Skilled College Writers

Glynda Hull, University of California, Berkeley

Abstract. Two groups of college writers (more skilled and less skilled editors) corrected and commented upon the sentence-level errors in two tasks (a self- written essay and three essays written by others), under two conditions (no feedback and feedback on location of error). Analyses of students' corrections showed that, while the more skilled writers almost always corrected more errors than the less skilled, the two groups performed similarly on the self-written essays where neither corrected many errors at all. Both groups performed better on the standard essays and better with feedback. Analyses of students' protocols showed that three strategies which were used for correcting errors (consulting, intuiting, and comprehending) varied with task and condition.

Error in writing has been a sore topic for a long time. Over a hundred years ago, Harvard President Charles W. Eliot complained that "bad spelling, incorrectness as well as inelegance of expression in writing, [and] ignorance of the simplest rules ... of punctuation are far from rare among young men of eighteen" (Cited in Hook, 1979, p.8). More recent complaints take a different tack, focussing most often on the unprofitability of too great a concern for correctness, particularly when such a concern involves the teaching of grammar and an attendant neglect of the teaching of composition skills (see, for example, Braddock, Lloyd-Jones, & Schoer, 1963; Petrosky, 1977; Perl, 1979; Ponsot & Deen, 1982). Such complaints serve the good purpose of broadening conceptions of what should be taught in the name of writing. At the same time, we know that some writers have great difficulty with sentence-level error, difficulty which does not go away by itself and which needs the attention of teachers and researchers. The National Assessment of Educational Progress (1980), for example, has identified two distinct populations of seventeen-year-olds based on an analysis of writing mechanics: one which "appears to have a general, though imperfect, grasp of written language" and another which "appears to be virtually lost" (p. 44).

The most convincing demonstration of the difficulty that basic writers have with error is Mina Shaughnessy's Errors and Expectations (1977). This

This research was carried out at the University of Pittsburgh as a doctoral thesis under the direction of William L. Smith.

Research in the Teaching of English, Vol. 21, No. 1, February 1987

8

The Editing Process in Writing 9

rich interpretive taxonomy has little to do with giving students up for lost, however, for Shaughnessy was interested in allowing researchers and teachers to see that there are other causes for error in writing besides ignorance and carelessness. Like researchers studying child language acquisition and second language learning, she, and Kroll and Schafer (1978) after her, demonstrated that one can identify patterns of error which often reveal the consistent application of erroneous rules or buggy procedures and which imply faulty logic and misconceptions - but a logic and an intention none- theless. They showed, then, that the processes of making errors and learning to correct them can be interpreted within a theory of how language learning occurs. This kind of research gave teachers a new perspective on error, for it pointed the way to cognitive diagnosis: examine a text, talk to a student, determine the reason for a type of error. (But see Ney, 1986, for a critique of this approach.) It also gave researchers a new agenda: using errors to understand the development of and constraints on writing ability.

Some scholarship on error in the writing of young adults has followed Shaughnessy's lead, taking as its aim to trace errors to their sources, with oral language being a predominant candidate. (See, for example, Epes, 1985, and the review by Morrow, 1985.) Other researchers have located the sources of sentence-level error in the limits of the human information-processing system. Most notably, Daiute's (1981) work demonstrates how constraints imposed by short term memory may lead to errors and also impede error detection and correction. Others have been interested in characterizing the editing process, taking as their purpose to provide a description of what happens after a student makes an error and returns to his or her text as an editor in order to detect and correct it. Bartholomae's (1980) case study of an inexperienced editor is an example of this research direction, as is the work of Lees (in press). The attitudes that inform this current research on error thus differ both from the traditional impatience with errorful writing and the more recent tendency to minimize its relative importance and thereby its gravity.

Purpose and Design

In the present study, my aim was to extend current research on error by investigating some of the variables that could affect the editing process. The emphasis was not, then, on discovering sources of errors, but on adding to what we know about the processes of detecting and correcting them. My approach was to study editing performance by comparing different groups of writers, under different conditions, on different tasks.

I chose to study two groups of college writers who differed by virtue of their writing experience and editing skill. By juxtaposing the performance of writers who made few errors with writers who made considerably more, I hoped to determine some of the knowledge and skills that one group had

10 Research in the leaching of English

in contrast to the other. By comparing their performance when editing their own writing and someone else's, I intended to test whether editing varies as a function of task. One would intuitively expect performance to be superior when the task is to edit another's writing, and there is some experimental evidence to support this as well (Bartlett, 1982). By comparing writers' performance under two conditions, feedback on error location or no such feedback, I planned to test whether focussed re-reading serves as a catalyst for

editing. My expectation was that it would help. The composing process is

usually represented as including a review component in which writers cycle through their texts in order to re-read, and this re-reading can trigger editing or revising. If, however, a writer reads a text and senses no disso- nance, even in a vague or amorphous way, the review process ceases to be

self-regulatory. One way to facilitate error detection is to provide feedback on error location in order to foster self-corrective feedback. Thus, the

design of the study allowed a comparison of the performance of different

groups of writers, on different tasks, under different conditions. "Performance" as far as error correction goes is usually taken to mean

frequency counts: the errors in a text are tabulated and categorized accord-

ing to a descriptive taxonomy taken from or based upon a handbook. I, too, intended to conduct frequency counts based on textual analyses, but saw a need as well to develop a new taxonomy for error that could also serve as a scheme for categorizing writers' comments about their corrections. The usual assumption about sentence-level error is that such data are easily obtainable and easily categorized (e.g., Mellon, 1975), and that traditional taxonomies of error are therefore sufficiently descriptive systems. However, evidence is beginning to appear to the contrary.

Scholars argue the impossibility of determining definitively a writer's intended text, even when that intended text is a student's essay and the reader is a teacher who wants only to construct a correct version of it. Bartholomae (1980), for example, claims that classifying an error is an

interpretive act: "for any idiosyncratic sentence . . . there are often a variety of possible reconstructions, depending on the reader's sense of the larger meaning of which this individual sentence is only a part, but also depending upon the reader's ability to predict how this writer puts sentences together" (p. 265). There is also evidence of error's phenomenology. Williams (1981) demonstrated that readers find errors where they expect to find them, as in student essays, and fail to see them where they do not expect them, as in

scholarly articles. There is evidence that errors are not equal in terms of how they affect readers and writers. Nystrand (1982) has shown how all errors are not equal in terms of a reader's comprehension, and also how the same error (in the case he mentions, the omission of a comma) can

change in communicative importance depending on context. And Bartlett (1982) demonstrated that errors like ambiguous pronouns which result from

The Editing Process in Writing 1 1

"privileged knowledge," or the information a writer brings to a text that a reader is not a party to, are more difficult for children to detect and correct than problems less likely to be so masked, like left-out words. It seems, then, that correcting some kinds of errors requires different strategies than cor-

recting others. (See also Braddock, Lloyd-Jones, & Schoer, 1963, and Hull, 1985.)

Taken together, such studies provide an interesting collection of converg- ing evidence that error and editing are not so simple as has been commonly assumed. If labelling an error is an interpretive act, then all readers will not acknowledge the same errors in the same texts. If errors vary in detect- ability dependent on a reader's relation to a text, then traditional descriptions of error will have little predictive value for determining which errors will be noticed and which will be missed. If the degree to which an error interferes with communication can change given different linguistic con- texts, then no static taxonomy can be used to rank serious errors as opposed to non-serious ones. If detecting some kinds of errors draws upon different

cognitive strategies than detecting other kinds, and these "kinds" don't cor-

respond to traditional error descriptions, then handbooks and proofreading guides have failed to present errors in ways that are psychologically salient.

For the present study, rather than use a taxonomy based on traditional handbook types as has been customary in research on error, I tried to devise a way to categorize errors that would capture what is salient to writers about errors when they read a text in order to edit it. This taxonomy groups errors

according to the operations a writer performs in order to make a correction:

1. consulting - a writer calls upon his knowledge of the conventions of written language;

2. intuiting - a writer relies upon how a text sounds; 3. comprehending - a writer detects something wrong with the meaning of

the text.

(In the above definitions, and throughout my discussion, I use correct and correction as inclusive terms, to cover all of the processes involved in error correction - detection, or recognizing that something is wrong; diagnosis, or

deciding the nature of the problem; and alteration, or actually making a

change on paper. I do not attempt to separate these sub-processes in the

present study.) The consulting category turns upon a writer's being able to call up a rule.

For example, a writer must know a rule for how to use apostrophes in order to show possession in order to correct the error in this mans hat. This cate-

gory includes all those errors which are errors simply because they violate

agreed upon conventions of written English. Most spelling errors fall into this category, as do many punctuation mistakes and usage errors. Many

1 2 Research in the leaching of English

consulting errors are located within a constituent; that is, they occur within a phrase rather than across a sentence or sentences.

The intuiting category includes many syntax errors, like Shaughnessy's (1977) blurred patterns. Usually, the meaning of a text is clear despite the presence of this kind of error, yet a writer/reader has a sense that something is amiss in the sentence, usually by its sound. The correction of intuiting errors could proceed by reference to rules of syntax but will likely not, unless a writer has unusual linguistic knowledge. And they occur within sentences or clauses rather than within constituents.

An example of the comprehending category is Bartlett's (1982) referential

ambiguity, as in this sentence she reports by a fifth grader: One day three boys went ice skating. He was showing off and he didn't see . . . (p.355). In order to correct such an error, a writer must recognize that the text's meaning is faulty, that another reader without "privileged knowledge" would not understand the text or would misinterpret it. Errors of comprehension often require a writer/reader to process more than a sentence of text; that is, they are

usually "supersentential." Thus, the first important characteristic of the taxonomy devised for the

present research is that errors are grouped by the operation required to correct them. A second characteristic is that it is context-dependent. Although it is possible to say that most spelling errors are consulting errors and most syntax errors fall into the intuiting category, some errors will cross

category boundaries according to their context. For example, the omission of a comma after an introductory element would seem to be a violation of a writing convention. But in Nystrand's (1982) example, "By the time we finished our dinner was ready" (p. 65), the error impedes comprehension, and thus it would be categorized in this instance as a comprehending error. (See Hull, 1983, for a complete listing of the error taxonomy used for the

present study, and Hull, in press, for a lengthier discussion of the issues

governing the creation of any error taxonomy.) If these categories are distinct, one would expect to see differences in

error correction rates as a function of each. One would also expect there to be behavioral differences in terms of how writers describe their detection/ correction activities. One would expect, for example, a writer to cite a rule

("you always put commas after quotation marks") when correcting a con-

sulting error. It should be possible, if writers comment upon their corrections as they make them, to categorize many of their comments as

consulting, intuiting, or comprehending strategies. This scheme for cate-

gorizing errors ex post facto is also a scheme, then, for categorizing acts of

editing. To summarize, the present study was designed to determine how writers

who differ in editing performance respond to operationally-defined cate-

gories of errors in different kinds of written texts. Two groups of writers (skilled and less skilled editors) corrected and commented upon three kinds


of error (consulting, intuiting, and comprehending) in two tasks (a self- written essay and three essays written by others) under two conditions (feedback on error location and no feedback).

Methodology

Participants

Participants were chosen from two populations of undergraduates who differed in their error-correction performance, yet were similar in age and educational level:

1. students enrolled in basic writing classes at an urban university who wrote two timed, impromptu essays (a placement essay administered as part of admission to the university, and a diagnostic essay written the first week of class in basic writing as a check on the placement essay), both of which contained intuiting, comprehending, and consulting errors.

2. students enrolled in composition classes other than basic writing at the same university, who wrote one timed, impromptu essay (the diagnostic essay) which contained none of the above-mentioned types of error.

The samples drawn from these two populations consisted of 13 less skilled and 11 more skilled writers. From approximately 2,000 placement essays, a prospective sample of 150 less skilled writers was identified. Of these 150 students, 53 registered to take basic writing the term during which the research was conducted. Of these 53 students, 17 wrote diagnostic essays the first week of term which contained instances of the three kinds of error that were present in their placement exams. Fifteen of these students agreed to participate in the study, and 13 completed all of the tasks. The sample of 12 more skilled writers was taken from a pool of approximately 250 students enrolled in 12 upper-level writing classes. From this pool, 17 students were identified who had written error-free essays, 16 agreed to participate in the study, and 12 completed it.

Test Instruments

Two text instruments were used: self- written essays and standard essays. The essay topic for the self-written essay directed the writers to describe a personal experience and to generalize on the basis of that experience. This describe/generalize format had been used regularly to structure placement essays required for college admission. Specifically, the assignment read:

All of us have, at some time in our lives, taken a risk in order to accom- plish a goal. I'd like you to begin your essay by writing about a time when you have taken such a risk. Explain why you took it and what the


results were. Then, based on what you've written, draw some conclusions about what is involved, in general, when people take risks.

The standard essays (see Table 1) were adapted from essays that origi- nally had been written by incoming freshmen as placement exams, on similarly structured describe/generalize topics. All sentence level errors were corrected in the three standard texts, and minor adjustments were made to ensure that each essay was the same length, approximately 250 words or one and one-fourth typed pages when double-spaced. This length was chosen purposefully; it is more than could appear on one typed page and thus made the texts appear more like a realistic editing exercise and less like a textbook drill.

Three kinds of errors were planted in each text: one consulting error, one intuiting error, and one comprehending error. Each error type was placed in one of the essays approximately 50 words from the beginning of the text. The two errors not occurring initially appeared in variable posi- tions to ensure that writers who found all three errors in one essay would not be reinforced for looking in the same places for the same errors in

subsequent ones. Since each category of error would be represented once in each standard essay and thus be presented to each writer a total of three times, it would have been possible to vary the specific type of error used to

represent the three general categories. However, to standardize the tasks, the same error type was used to represent the categories in all standard essays.

The error chosen to represent the consulting category was a variety of spelling error and can be illustrated by the phrase, this geniuses word (Table 1, line 10). Correction requires the application of a rule for showing singular possession. The error chosen to represent the intuiting category was what Mina Shaughnessy (1977) called a blurred pattern as in this example: / think that by adding to the confusion was the fact that Dr. Lesgpld has an inferiority complex (Table 1, lines 14 and 15). This kind of error presumably occurs when a writer begins to compose one sentence pattern, but loses track of its syntax and merges it halfway through with another sentence pattern. To correct it, a writer would not be likely to call up an explicit rule but would "hear"

something wrong in the sentence. The error chosen to represent the com-

prehending category was a misplaced however and can be illustrated with the

following sentences: [Dr. Lesgold] was not smart enough to set up the electronic balance the school had bought for him a year before. Dr. Lesgpld was able to devise a way to deprive our senior class of the highlight of senior year. However, every year the biology class had gone to the anotomy lab at Pitt to see the medical students work on cadavers

(Table 1, lines 17-23). The third sentence does not logically follow the second unless the however is deleted or moved to the second sentence. To correct the mistake a writer thus must do more, one expects, than listen to determine whether a sentence "sounds right" or to watch for violated conventions; he or she must pay attention to meaning as it develops between sentences.


Table 1

Example of a Standard Essay

A CONSULTING error appears in line 10; an INTUITING error appears in lines 14 and 15; a COMPREHENDING error appears in line 21.

1 "Very bad" is too mild a rating for the man who was my 2 senior organic chemistry teacher, Dr. Ivan Lesgold. I would 3 guess his age to be about sixty, but his theories had to have 4 been much older than his age. This German horror could have 5 gone to school in the Dark Ages. 6 Although Dr. Lesgold considered himself a scientist, he 7 seemed to me to have tunnel vision. Our organic chemistry book, 8 for example, said that insulin consisted of fifty-one amino acids. 9 Lesgold said there were only forty-eight. The omniscient Doctor

10 had spoken, and our class grudgingly accepted this geniuses word. 1 1 Dr. Lesgold's class was a study in confusion. He would start 12 at the back of a chapter, skip to the middle, and then to the 13 front. (Chronological order must be too easy for powerful brains.) 14 I think that by adding to the confusion was the fact that Dr. 15 Lesgold had an inferiority complex: he was always trying to prove 16 he was right. The Doctor claimed that he could solve everything 17 that ailed the world if he just had the right equipment. But for 18 some unknown reason he was not smart enough to set up the 19 electronic balance the school had bought for him a year before. 20 Dr. Lesgold was able to devise a way to deprive our senior 21 class of the highlight of senior year. However, every year the 22 biology class had gone to the anatomy lab at Pitt to see the 23 medical students work on cadavers. My senior year would have been 24 no exception, but Dr. Lesgold scheduled the trip on Senior Skip 25 Day, knowing all too well none of the senior class would attend. 26 To be fair, I have to admit that Dr. Lesgold always thought 27 he was doing the right thing. Even though most of me thoroughly 28 dislikes the guy, a small portion of me has to admire the man for 29 his dedication in teaching so long in a place where he apparently 30 was not appreciated by anyone. 31 I believe that good teaching involves developing pupils' 32 ideas, not force- feeding teachers' ideas to the students. In the 33 "Lesgoldian" method of teaching, there was no willingness to 34 compromise or to listen to the blasphemous students who dared to 35 challenge the edicts of Ivan. Teachers should try to expand their 36 pupils' horizons; instead, Lesgold only narrowed ours.

Test Procedure

For both tasks, each writer was instructed to 1) read the text silently until he or she reached an error or something that needed changing, 2) make the change or correction on the page, and 3) explain aloud why he or she made the change or correction. At this point, the experimenter was free to

16 Research in the Teaching of English

intervene by asking questions in order to clarify the writer's explanation or to ask for further explanation. When writers miscorrected - that is, when

they identified as an error something that was correct, or when their attempt to correct an error did not succeed - they were still allowed to make whatever

change they had begun. The error correction tasks were counterbalanced for each group. For half

of the students in each group, the standard essays comprised the first task and one session of the study. For the other half, the first task consisted of

writing an essay and then correcting it, and the standard essays were administered in the second session. Each session took place in a small conference room equipped with a table and chairs.

The standard essays were presented in random order to each writer in one session. This session began with the writer reading a set of written instructions on how the session would proceed. These instructions had been

piloted to determine the kind of language that would lead students to make

editing changes rather than revisions. Each writer was encouraged to question anything that seemed unclear in the instructions. The experimenter sat across from the writer, noting his or her editing changes on a copy of the standard essays.

For the self-written essay, each writer was provided written instructions, a tablet of paper, and a multi-ink pen. The instruction sheet illustrated how each draft was to be written in a different color of ink, and how a thin line should be drawn through deletions or changes. Writers were encouraged to

spend as much time as they needed in order to write the best essay that

they could. They were also encouraged to write as they normally would.

They were not, that is, to feel compelled to produce several drafts if this were not the way they usually composed. After these instructions, the writers were left alone in the conference room to compose their essays. As previously arranged, they signalled when they had written their last draft but before

they had read over it for the last time. A copy was made of their final draft, and the writers read a set of written instructions identical to those provided for the editing of the standard essays. After reading the instructions, the students corrected their own essays using the same procedure as for cor-

recting the standard ones. The editing tasks were carried out under two conditions - feedback and

no feedback. Prior to the feedback conditions, the locations of errors missed

during editing were highlighted with a yellow marker. The amount of text that was highlighted varied according to error type, but was never less than one sentence and never more than five lines. Armed with this information on locus, students tried again. Every error missed by the experienced writers was covered in this session, but because the inexperienced writers missed

many more errors, usually only one error from each of the three categories was presented to them. The conditions were purposefully not counterbal-


anced (the non-feedback condition always occurring first) in order to determine which errors writers could correct entirely on their own.

These procedures allowed for the collection of two kinds of data: (1) written responses to error and (2) oral comments on those responses.

Data Analysis

For the self-written essays, the errors that writers made and corrected were identified by two raters (interrater reliability was .90). These errors were then categorized as consulting, intuiting, or comprehending (interrater reliability for this classification, calculated for 20 percent of the errors, was .94.). Similar procedures were carried out for the standard essays. (Means and standard deviations for errors made are provided in Table 2.)

For the protocol analysis, tapes were transcribed, and the typed tran- scriptions were divided into "episodes," each consisting of the comments the writer made about one error or textual problem. The episodes were then categorized according to whether the writer seemed to be consulting, intuiting, or comprehending in order to make a textual change.

Results

Analysis of Errors Corrected

Results will be presented for the total number of errors corrected and for the number of errors corrected in each of the three error categories (consulting, intuiting, and comprehending).

Table 2

Mean Errors Made in Self- Written Task By Groups and Categories

Total Consulting Intuiting Comprehending Errors

Less Skilled Writers M 21 3 3 27

(n = 13) SD 13 4 3 19

More Skilled Writers M 6 1 1 8

(w = 11) SD 9 1 1 4

1 8 Research in the leaching cf English

Total Errors Corrected

The proportion of total errors was analyzed by means of a three-way ANOVA, repeated measures on two factors, with level of skill as the between

groups factor and task and conditions as the within groups factors. All three main effects were significant. (See Table 3 for means and standard deviations.)

The groups differed in overall performance, with the more skilled group correcting more errors than the less skilled, independent of tasks or conditions (F = 10.69; df = 1,22; p < .005). The writers performed differently on the tasks, correcting significantly more errors in the standard essays than in the self-written ones (F = 70.19; df = 1,22; p < .001). And they performed differently as a function of condition, correcting significantly more errors in the feedback condition than in the non-feedback condition (F = 27.39; df = 1,22; p < .001).

In addition to these main effects, the analysis showed two significant interactions. Both groups corrected more errors on the standard tasks. However, the groups differed in their rate of increase, the more skilled writers improving more on the standard tasks than the less skilled, as shown

by an F-test for simple effects (F = 18.22; df = 1,22; p < .001). The other

significant interaction involved all three factors - groups, tasks, and conditions (F = 27.95; df = 1,22; p < .001). Both groups improved their rate of error correction on both tasks as a result of feedback. However, the more skilled group differed significantly from the less skilled on the standard task, non-feedback condition as indicated by an F-test for simple effects (F = 23.54; df = 1,22; p < .001), where they corrected significantly more errors than the less skilled group. They did not, however, differ from the less skilled on the immediate task in either condition or the standard task in the feedback condition.

Table 3

Mean Percentage of Total Errors Corrected by Groups, Tasks, and Conditions

Self-Written Essays Standard Essays No Feedback Feedback No Feedback Feedback


(w =13) SD 9 22 13 18


(w = 11) SD 12 28 19 25


Error Categories

Some of the more skilled writers did not make any errors in some of the

categories on the self-written task, and some of them corrected all of the errors in the standard essays during the non-feedback condition. This resulted in a number of empty cells. Consequently, the data on the number of errors that writers corrected in each of the three error categories were not tested for statistical significance and are presented only in terms of the mean percentage corrected.

The more skilled writers averaged a correction rate of 49 percent for

consulting errors; the less skilled, 23 percent. A greater percentage of con-

sulting errors was corrected on the standard task than on the self-written - 47 percent compared to 24 percent - and as a function of feedback - 53

percent compared to 18 percent. For intuiting errors, the more skilled writers

averaged a correction rate of 56 percent; the less skilled, 38 percent. More

intuiting errors were corrected in the feedback condition - 58 percent as

opposed to 36 percent. But performance did not seem to differ as a function of task, where the correction rates were 48 percent for the self-written task and 49 percent for the standard task. For comprehending errors, performance favored the less skilled group slightly, with more skilled writers averaging 31 percent and less skilled averaging 35 percent. As a function of task, writers averaged a 52 percent correction rate on the standard essays, but

only 14 percent on the self-written essays. As a function of condition, writers

averaged a 40 percent correction rate with feedback, but only 26 percent without. (See Table 4 for means and standard deviations.)

Protocol Analysis

The protocol data yield a total of 1,024 episodes - 565 for the less skilled writers and 459 for the more skilled. These data included any comments writers made about textual changes that they wanted to make, whether or not those changes had to do with actual errors. Of the total, 177 (17 percent) were excluded from further analysis either because students did not ver- balize anything or because their comments could not be categorized. Of the 177 discarded episodes, 92 (52 percent) belonged to less skilled writers, and 85 (48 percent) to more skilled writers. The remaining episodes were cate-

gorized according to whether the students made textual changes on the basis of "consulting" or calling upon a knowledge of rules or written language conventions; by "intuiting" or relying upon a sense that something might be wrong; or by "comprehending" or attending to semantic discrepancies. If writers used more than one strategy in correcting an error, the strategy which finally led to a textual change was counted. Table 5 contains examples of the categories.

The proportion of total episodes for each of the three strategies was

analyzed by means of a three-way ANOVA, repeated measures on two fac-


Table 4

Mean Percentage of Consulting, Intuiting, and Comprehending Errors Corrected by Groups, Tasks, and Conditions

Self-Written Essays Standard Essays No Feedback Feedback No Feedback Feedback

S S S S bC £ bf) g bO § bO £

I ?| iff! iff! iff! 8 I § §|i 8 1 I 8 | §

LESS SKILLED Af 8 25 10 32 63 22 5 18 46 45 46 62 SD 10 32 17 22 48 17 12 22 29 25 24 44 n 13 11 13 13 9 13 13 13 13 13 13 11

MORE SKILLED M 7 36 0 48 63 25 53 66 47 86 60 62 SD 11 43 0 33 48 42 36 25 33 24 46 2 n 11 6 7 11 4 6 12 12 12 7 19 11

tors, with level of skill as the between groups factor and task and conditions as the within groups factors.

For the consulting category, there were no overall differences in how the

groups performed. (See Table 6 for means and standard deviations.) Forty percent of the time, less skilled writers relied upon rules to guide their textual changes, while more skilled writers did so 48 percent of the time. There was, however, a significant difference in performance as a function of task (F = 6.54; df = 1,22; p < .025). The students consulted more fre-

quently on the self-written essay (52 percent) than on the standard essays (38 percent). Performance also differed significantly within subjects as a function of condition (F = 15.34; df = 1,22; p < .001). When given feedback, students increased the rate at which they relied upon rules, from 35 to 54 percent.

The first of two significant interactions is task by conditions (F = 379.0; df = 1,22; p < .001). An F-test for simple effects showed there was no significant difference between the percentage of consulting done on the standard and self-written tasks in the non-feedback condition. However, with feedback, students relied upon rules significantly more on the self-written tasks than on the standard (F = 427.9; df = 1,22; p < .001). The second interaction is groups by tasks by conditions. F-tests for simple effects indicated a significant difference (F = 34.31; df = 1,22; p < .001) between the


Table 5

Samples of Consulting, Intuiting, and Comprehending Strategies Taken From Protocols

CONSULTING STRATEGY

Less Skilled Writer, Self- Written Task, No Feedback

Writer: Down here, "the money the Pirates offered!' Investigator: Uh-hum? W: "The Pirates" maybe "had." I: Yeah? W: Could I put "had" in there? I: Why would you do that? W: Cause "offered" had "ed" on it.

Less Skilled Writer, Standard Task, No Feedback

Writer: In line 7, does a comma belong there? Investigator: Let's see. Well, how do you tell when a comma belongs and when it doesn't? W: Well, like if you have a dependent and independent sentence? I: \feah, so which is that? W: Well, the first one would be dependent. I: Uh-hum. W: And the second one's independent. I: So would you leave the comma in there? W: Yeah. I: So when you have a dependent one, in front of an independent one . . . W: \feah, you leave the comma in there then.

INTUITING STRATEGY


Writer: Uh, maybe line 23. Investigator: \feah? W: Uh, "had gone"? I: Uh-hum? W: I don't know if that sounds right or not? But I think maybe uh, "However,

every year the biology class went to the gross anatomy lab"? I: Okay. W: I don't - you know - it sounds a little better.

More Skilled Writer, Standard Task, No Feedback

Writer: That "by" on line 13- Investigator: Um-hum? W: Well, I could simply just delete that. I: Uh-hum? W: (reading) "I think that adding to the confusion was the fact that Dr. Lesgold had an

inferiority complex." I: \feah. How long did it take you in the sentence before you realized that you

needed to take out the "by"? W: It just, I saw it immediately. I: Uh-hum.


Table 5 (Continued)

W: Um - that "by" sort of sets up, um - this is funny. I know that automatically it should not be there. But telling you why is going to be difficult. . . . It's going to take the reason some time to surface.

COMPREHENDING STRATEGY

More Skilled Writer, Self-Written Task, Feedback

Writer: (reading) "Many people feel that life is more exciting and satisfying if you're milling to take a chance now and then." Switch the "is" to "can be." Investigator: Okay. W: It's a different sense if you have "can be" there.


Writer: Line 27, that ain't supposed to be "bun" is it? Investigator: Is it? W: No, what's "bin"? That means "special bun" or what? I: Maybe it is "bun"? (reading) "Put the good sections of the good leaves in a special

bun" instead of "in a special bin"? W: A "bin" most bins are kind've big, I guess. I: For putting lettuce into, you mean. W: Uh-huh. Could be ... they both make sense, I guess.

groups on the self-written test, feedback condition. Less skilled writers increased their rate for the consulting strategy on this task and condition more than did the more skilled writers.

For the intuiting strategy, performance of the two groups did not differ, nor did performance differ as a function of task or condition. (See Table 7 for means and standard deviations.) However, one interaction was significant: tasks by conditions (F = 8.95; df = 1,22; p < .025). An F-test for

Table 6

Mean Percentage Consulting Strategy Was Used by Groups, Tasks, and Conditions

Self- Written Essays Standard Essays No Feedback Feedback No Feedback Feedback


(n = 13) SD 24 24 28 27


(n = 11) SD 34 32 16 19


Table 7

Mean Percentage Intuiting Strategy Was Used by Groups, Tasks, and Conditions



(n = 13) SD 25 20 15 27 More Skilled Writers M 29 9 29 29

(n = 11) SD 31 20 13 18

simple effects showed that feedback increased the proportion of intuiting done on the standard essays, but decreased the proportion done on the self- written essays (F = 8.09; df = 1,46; p < .005).

For the comprehending strategy, there was no main effect for groups, but performance within subjects did differ significantly as a function of conditions (F = 7.38; df = 1,22; p < .025). Students decreased their use of the comprehending strategy when given feedback. Performance within subjects also differed as a function of tasks (F = 7.78; df = 1,22; p < .025). Students used the comprehending strategy significantly more on the standard tasks than on the self-written essay. (See Table 8 for means and standard deviations.) One significant interaction occurred, groups by conditions (F = 7.41 ; df = 1,22; p < 0.25). As confirmed by an F-test for simple effects (F = 17.7; df = 1,46; p < .001), less skilled writers used the comprehending strategy proportionately more in the non-feedback condition than did the experts. However, no such differences existed when feedback was provided.

Discussion

This study compared the performance of more skilled and less skilled college writers as they attempted to correct and comment upon the errors in a piece of their own writing and in essays written by other students, with feedback on the location of the errors and without it. Performance was determined through textual analyses by tabulating the total number of errors corrected and the percentage of errors corrected in each of three categories (consulting, intuiting, and comprehending), and also through protocol analyses by tabulating the percentage of protocol episodes in each of three categories (consulting, intuiting, and comprehending). The results


Table 8

Mean Percentage Comprehending Strategy Was Used

by Groups, Tasks, and Conditions



(n =13) SD 26 18 27 38


(n = 11) SD 29 21 14 23

of the analyses showed that, while the more skilled writers almost always corrected more errors than the less skilled, the two groups performed sim-

ilarly on the self-written essays, where they corrected many fewer errors than on the standard essays. Both groups performed better on the standard

essays and better with feedback. These findings also seemed to apply, in the main, to the three operationally-defined error categories, but it should be noted that no statistical tests were performed on the data for error cate-

gories. Analyses of students' protocols showed that the three strategies which were used for making textual changes (consulting, intuiting, and comprehending) varied with task and condition. The consulting strategy was used more frequently in the feedback condition, particularly by the less skilled writers. The intuiting strategy increased during the feedback condition on the standard essays, but decreased on the self-written essays. The comprehending strategy generally decreased with feedback but increased on the standard tasks. Less skilled writers tended to use the comprehending strat-

egy more frequently in the non-feedback condition. One of the goals of the research was to describe the editing skills of more

skilled as compared to less skilled college writers. In general, the data showed the more skilled writers out-performing the less skilled by detecting more total errors. Although both groups corrected more errors on the standard tasks, the more skilled group improved at a higher rate. And although both groups did better as a function of feedback, the more skilled group did better still on the standard task, non-feedback condition. For the error

categories, the more skilled writers seemed to do particularly well compared to the less skilled writers at correcting errors in the consulting category. Less skilled writers matched the performance of the other group on only one task, under one condition: they corrected a few more errors in the comprehending category, on the standard essays, no-feedback condition. Two other


differences for the less skilled writers emerged from the strategy data. They employed the comprehending strategy more during the non-feedback condition, and they increased their use of the consulting strategy on the self- written essay, feedback condition more than did the more skilled writers.

One interesting part of these findings is the less skilled writers' tendency, when left on their own, to focus on meaning. That is, they detected more errors in the comprehending category than the more skilled writers on one task, and they used the comprehending strategy more than the other group in non-feedback situations. If these less skilled writers are less skilled readers too, one might expect them to focus primarily on decoding or on reading for gist even when engaged in a proofreading task. And since the majority of sentence-level errors are not meaning-related, students will naturally fail to see many mistakes. This reminder of the centrality of meaning-making in reading and writing tasks might, then, partially account for some students' poor performance on editing tasks, and it might also temper the

impatience such performance often arouses. The findings do not necessarily imply, however, that the students' focus

on meaning was entirely praiseworthy. In editing their own essays, these writers often expressed a concern for making a text literally true or accurate. One writer said, "here in the fourth line, I'd write to lose weight to -wrestle at a lower weight class, instead of to drop weight. Not many people might understand what that means." And another: "Down here, seeing the chance to gain some recognition. I'd put, seeing the chance to possibly gain some recognition. To make it more accurate." On the standard essays, the less skilled writers made comments which suggested a preoccupation with determining what the orig- inal authors of the essays must have meant to say or should have said. One student objected to a sentence on how a teacher had narrowed his students' horizons by noting: "I think they learned more than they think they did. . . . I don't see how he could narrow your horizons." While alterations of the sort the students suggested could be said to clarify a text, they are not the kinds of improvements one usually associated with meaning-making, nor would they significantly improve an essay's quality. It is important to note, too, that in making these changes, the students were correcting what they believed were errors, things that had to be altered in order for the text to be correct. The suggestion, then, is that these less skilled writers had not yet learned to distinguish between matters of necessity and matters of choice, and that their confusion is reflected in the emphasis they placed on using the comprehending strategy.

One of the curious findings of the study was that the less skilled writers bested the more skilled on correcting comprehending errors (misplaced however's) on the standard tasks. This might be taken as evidence of one

group's skill at detecting errors in meaning and the other's oversight or lesser skill. There is another explanation, however. One less skilled writer commented after he had read a standard essay, "I tell you, there's just a lot


of however's in here." He pointed to sentences and read, "However, however. . . it's just like words thrown in. I try to limit however's myself." And he crossed out two however's from the standard essay. Other less skilled editors corrected the misplaced however's by means of idiosyncratic rules, such as "never end a sentence with however." These comments both call into question the find-

ings for that category and illustrate that writers sometimes edit by means of buggy rules - rules that are not very apparent from textual analyses alone.

Given the more skilled group's better performance with consulting errors, and the less skilled group's use of idiosyncratic rules, one might suppose that a likely reason for the skilled group's better performance overall was their mastery of editing prescriptions - of well-honed rules for grammar and print-code conventions. There was some evidence from the

protocols that the more skilled writers knew and readily applied grammat- ical terminology and rules and that the less skilled writers knew less such

knowledge or did not apply it. Only two less skilled writers corrected the

consulting error, a possessive mistake, in the standard essays, and these two corrected it inadvertently, by rewording the sentence. Experts, on the other hand, seemed to have internalized this rule as well as a good many others. But there was also evidence that the more skilled group sometimes applied idiosyncratic rules. One writer, for example, justified shifting person from one to you because "one is like everybody in the world and you is like you're talking to someone about someone." It would be an oversimplification, then, to think that the better editors always operated by conventional rules.

Both more skilled and less skilled writers did better overall at error correction when they worked on a text that was not their own. In contrast, both

groups did very poorly when they attempted to edit their own writing, the main difference between the groups being that the more skilled writers made fewer errors to begin with, and so had fewer to correct. In particular, the groups were able to correct more consulting and comprehending errors when those errors did not occur in their own writing. These findings, like Bartlett's (1982), might be interpreted to mean that writers' knowledge of their own intentions can mask error detection. Alternately, one could argue that the findings simply illustrate that the errors on the standard essays happened to be errors that the writers knew, while the errors in their own

writing were of course errors that they did not know. This argument picks up speed if one believes that the errors chosen for the standard tasks are not really tokens representing a category, but individual error types. Even if

they do share a common category, the errors a particular writer makes might be more difficult for that writer than errors of the same type that another writer makes. With either interpretation, however, the implication is that there will be some errors that writers will have difficulty seeing, even upon close proofreading. This makes it reasonable, one could argue, to advise


writers to turn to other readers at some point during the writing process for help detecting those mistakes that will otherwise remain stubbornly invisible, or to cease to expect completely error-free papers.

The study shows that one clear way to increase writers' success at detect-

ing errors in both their own writing and in essays by others is to provide them feedback on an error's location. This technique improved both overall and category-specific error detection and was particularly helpful in facili-

tating the correction of consulting errors. It may be, then, that the crucial variable in editing is recognizing the existence of an error (cf. Flower et al., 1986). Ideally, error correction is a self-contained, cybernetic process, where

reading one's writing results in information on performance or the recognition that one has made an error. With that knowledge, editing can proceed, and then reading starts again until another error is detected.

What feedback on location seems to do is facilitate self-monitoring and

thereby make self-correction possible. Since feedback on location narrows the range one must consider, it thereby limits the possible errors that can occur as well as the possible responses that one can make to them. Both

groups, for example, decreased their use of the comprehending strategy and increased their use of the consulting strategy in the feedback condition. Since most errors are consulting errors, and since many of these occur within a sentence rather than across sentences, this adaptation worked well. Instead of attending to meaning, writers attended to rules and thereby increased their correction rates. It is worth pondering, though, this apparent ease with which students were influenced to consult rules rather than attend to meaning. Perhaps this result says as much about students' notions of what a researcher/English teacher expects to hear when he or she points out a line containing an error as it does about the salutary effects of the feedback

technique. In any pedagogy designed to deal with sentence-level error, there is

another variable to consider besides condition and task, and that is the nature of sentence-level error. It has become standard practice to think of such errors only in terms of the categories provided by handbooks of grammar and usage. Typically, we distinguish errors according to whether they are mistakes of various sorts in punctuation or spelling or grammar or

usage. Yet such a classification may fail to capture the salient features of these errors, particularly from the perspective of the writer who must correct them. The taxonomy used in the present study had the virtue of

describing errors operationally, according to the strategies writers would draw upon to correct them. The fact that writers were able to correct more errors in some categories than in others could be taken to mean that the

taxonomy has some psychological validity. At present the taxonomy also has the disadvantage of consisting of very

broad categories that beg to be sub-divided. The argument could be made


that other large groupings of error would result in equally different correction rates, and arbitrarily so. The scheme is also hard to apply: there are judgment calls to be made concerning which category an error fits into given its occurrence in a particular context. But here the taxonomy suffers a common fate: applying any taxonomy requires one to interpret a text, to judge a writer's intentions and to imagine his or her intended text. Thinking of errors in terms of consulting, intuiting, and comprehending categories may seem hard to do just because we are accustomed to thinking of errors according to different schemata - according to arbitrary ones, perhaps.

References

Bartholomae, D. (1980). The study of error. College Composition and Communication, 31, 253-269.

Bartlett, E.J. (1982). Learning to revise: Some component processes. In M. Nystrand (Ed.), What writers know: The language, process, and structure of written discourse (pp. 345-363). New York: Academic Press.

Braddock, R., Lloyd-Jones, R., 8c Schoer, L. (1963). Research in Written Composition. Urbana: National Council of Teachers of English.

Daiute, C.A., (1981). Psycholinguistic foundations of the writing process. Research in the Teaching of English, 15, 5-22.

Epes, M. (1985). Tracing errors to their sources: A study of the encoding processes of adult basic writers. Journal of Basic Writing, 4, 4-33.

Flower, L., Hayes, J.R., Carey, L., Shriver, K., 8c Stratman, J. (1986). Detection, diagnosis, and the strategies of revision. College Composition and Communication, 37, 16-55.

Hook, J.N. (1979). A long way together: A personal view of NCTE's first sixty-seven years. Urbana, 111.: National Council of Teachers of English.

Hull, G. (1983). The editing process in writing: A performance study of experts and novices. Unpublished doctoral dissertation, University of Pittsburgh.

Hull, G. (in press). Constructing taxonomies for error (or can stray dogs be mer- maids?). In T. Enos, A sourcebook for basic writing teachers. New York: Random House.

Hull, G. (1985). Research on error and correction. In B. W. McClelland 8c T. R. Donovan (Eds.), Perspectives on research and scholarship in composition (pp. 162-184). New York: Modern Language Association.

Kroll, B.M., 8c Schafer, J.C. (1978). Error analysis and the teaching of composition. College Composition and Communication, 29, 243-48.

Lees, E. (in press). Error-hunting, fish and game: Proofreading as reading. In T. Enos, A sourcebook for basic writing teachers. New York: Random House.

Mellon, J.C. (1975). National assessment and the teaching of English. Urbana, IL: National Council of Teachers of English.

Morrow, D.H. (1985). Dialect interference in writing: Another critical view. Research in the Teaching of English, 19, 154-180.

National Assessment of Educational Progress. (1980). Writing achievement, 1969-79: Results from the third national writing assessment. Volume 1. 17 -Year-Olds (Report No. 10- W-01). Denver: National Assessment of Educational Progress.

Ney, J.W. (1986). Error analysis, theories of language, and the teaching of writing. Written Communication, 3, 15-29.


Nystrand, M. (1982). An analysis of error in written communication. In M. Nystrand (Ed.), What writers know (pp. 57-74). New York: Academic Press.

Perl, S. (1979). The composing processes of unskilled college writers. Research in the Teaching of English, 13, 317-336.

Petrosky, A.R. (1977). Grammar instruction: What we know. English Journal, 66, 86-88.

Pbnsot, M., & Deen, R. (1982). Beat not the poor desk. Montdair, NT: Boynton/Cook. Shaughnessy, M. (1977). Errors and expectations: A guide for the teacher of basic writing.

New York: Oxford University Press. Williams, J. (1981). The phenomenology of error. College Composition and Commu-

nication, 32, 152-168.

Conference on Writing Assessment

The National Testing Network in Writing, The New Jersey Department of

Higher Education, and The City University of New York announce the FIFTH ANNUAL CONFERENCE ON WRITING ASSESSMENT on April 5, 6, and 7 in Atlantic City, New Jersey. This national conference is for educators, administrators, and assessment personnel and will be devoted to critical issues in assessing writing in elementary, secondary, and postsecond- ary settings. Discussion topics will include theories and models of writing assessment, assessing writing across the curriculum, the impact of testing on minority and ESL students, computer applications in writing assessment, and current research on writing assessment.

For information and registration, please write Dr. Mary Ellen Byrne, New

Jersey Department of Higher Education, 225 West State Street, Trenton, New Jersey 08625 or call her at (609) 987-1962.

the editing process in writing: a performance study of more skilled …€¦ · the composing...

Documents