31810984.pdf

8/14/2019 31810984.pdf

1/19

AUDITING: A JOURNAL OF PRACTICE & THEORYVol. 27, No. 1May 2008pp. 151168

Audit Documentation Methods:A Path Model of Cognitive Processing,

Memory, and Performance

Elizabeth A. Payne and Robert J. Ramsay

SUMMARY:One result of Public Company Accounting Oversight Board Auditing Stan-

dard No. 3 Audit Documentation (AS No. 3) (PCAOB 2004) is that audit firms are using

more detailed audit workpapers and fewer summary memos to document their pro-

cedures. We explore implications of the choice of these two documentation methods.

Specifically, we develop and test a model showing how different forms of documen-

tation affect the cognitive processes auditors use during audit testing, and how differ-

ences in cognitive processing affect auditors memory and two performance measures

(error detection and recognition of a potentially fraudulent pattern in the evidence).

The results of path analysis show auditors who prepare detailed workpapers spend

more total time on the task, and total time is associated with identifying more errors

and better pattern recognition. Auditors who prepare summary memos examine the

evidence items a greater number of times, which is associated with better pattern

recognition and increased memory. The overall result is that each method enhances

cognitive processing differently and the manner of cognitive processing affects per-

formance measures differently. Documentation by summary memos is more efficientand results in better memory of the evidence. Documentation by detailed workpapers

results in better error detection for one type of error and permits a self-review of doc-

umented evidence by the preparer, which enhances pattern recognition. Finally,

documentation by detailed workpapers provides evidence that may potentially enhance

pattern recognition at the reviewer level.

Keywords:audit documentation; memory; fraud; pattern recognition; error detection.

Data Availability: Data are available from the first author upon request.

INTRODUCTION

This study examines the implication of auditors use of two different documentationmethods, summary memos or detailed workpapers. Summary memos typically sum-marize the sample source and selection criteria, audit procedures performed, and

Elizabeth A. Payne is an Assistant Professor at the University of Louisville and Robert J.

Ramsay is an Associate Professor at the University of Kentucky.

We are grateful for helpful comments given on earlier versions of this paper from Michael Bamber, Rick Tubbs,Mary Curtis, Jackie Hammersley, Dan Stone, Cathy Miller, Associate Editor Ken Trotman, and anonymous re-viewers. We also wish to thank participants at the 2005 AAA Auditing Mid-Year Meeting and the 2005 AAAAnnual Meeting, workshop participants at the Universities of Mississippi and Louisville, as well as three partici-pating firms for their support.

Submitted: July 2006Accepted: September 2007

8/14/2019 31810984.pdf

2/19

152 Payne and Ramsay

conclusions reached without providing any details of the individual evidence items exam-ined, whereas detailed workpapers do provide all of the specific information for each sampleitem such as names, dates, amounts, etc. We develop and test a model in which we firstexamine how using different methods of documentation affect the manner in which auditors

cognitively process evidence. In the next step of the model, we examine how differencesin cognitive processing affect memory and performance. We also test for the possibility ofdirect effects of documentation on memory and performance.

Audit documentation has recently received considerable attention by standard settingbodies. The AICPAs Auditing Standards Board has released two documentation standards,and The Public Company Accounting Oversight Board (PCAOB) recently adopted AuditingStandard No. 3Audit Documentation(AS No. 3) (PCAOB 2004) as one of its first pro-nouncements to improve audit quality and enhance public confidence. In releasing AS No.3, the PCAOB discusses how documentation affects the quality of work performed. Forexample, the release accompanying the standard comments that the need to document theprocedures performed, the evidence obtained, and the conclusions reached demands a dis-

ciplined approach to planning and performing the engagement (PCAOB 2004). Moreover,in testimony before the PCAOB Documentation Round Table, Lynn Turner, former ChiefAccountant of the SEC commented, when you ... ask people to document their work, theybehave in a different fashion than if they dont have to document their work (PCAOB2003).

Previously, auditing firms used a reduced level of documentation for certain audit testsin order to increase efficiency and reduce litigation risks (Rich et al. 1997). Such docu-mentation often consisted of only a summary memo explaining overall procedures andfindings; details relating to audit evidence often were provided only for exceptions noted(Rich et al. 1997). The alternative is to prepare a detailed workpaper listing the specifics

of all items examined, procedures performed for each item, and results for each item. Basedon our numerous discussions with practicing auditors, one result of AS No. 3 is that firmshave generally returned to using more detailed workpapers, although summary memos arestill used in some cases. As one practitioner noted, we may find that a memo is an efficientand effective way of documenting the work completed. We would then attach the memoto the appropriate step in the database and ensure that the results section of the databasereflected our work completed in the memo.

Studies in auditing have examined how different types of documentation affect auditorsjudgments. For example, studies have compared the use of flowcharts, narratives, and ques-tionnaires while developing an understanding of internal controls (Purvis 1989; Bierstakerand Brody 2001). Agoglia et al. (2003) find that the format of justification memos affects

fraud likelihood judgments of preparers and reviewers. Specifically, requiring memos thatdetail positive and negative evidence on a series of components of the decision resulted indocumentation of more data and a more positive assessment of control environments thansupporting or balanced memos. Other studies have examined the effect of workpaper con-tent on auditor judgment (e.g., Tan and Trotman 2003; Tan and Yip-Ow 2001; Yip-Ow andTan 2000), the effect of audit approach on documentation (e.g., Ricchiute 1999; Bierstakeret al. 2006; Bierstaker and Wright 2004), and the effect of audit judgment on documentation(Shankar and Tan 2006). We extend this work by examining the effects of documentation oneffectiveness and efficiency of evidence gathering procedures. That is, we examine thePCAOBs conjecture that the anticipated form of documentation can affect the performance

of the audit procedures (to be documented).In our study, staff and senior level auditors participated in a computerized case studyinvolving testing internal controls for write-offs of accounts receivable The results of path

8/14/2019 31810984.pdf

3/19

Audit Documentation Methods 153

analysis show that different documentation methods lead to auditors taking different cog-nitive processing routes to achieve performance. Using detailed workpapers increases thetotal amount of time spent on the task compared to summary memos. Time spent is posi-tively related to both error detection and recognition of a pattern that may indicate fraud.

Summary memo documentation results in examining evidence items a greater number oftimes than detailed workpapers. Examining evidence more times increases memory for theevidence and increases pattern recognition.

Overall, the results show that the method of documentation affects the manner of cog-nitive processing, and the manner of cognitive processing has differential effects on per-formance and memory. The results also show that preparing summary memos is associatedwith better memory and preparing detailed documentation is associated with greater detec-tion of an authorization error (but not a posting error, which almost all of the subjectsdetected). As expected, the results show that summary memos are more efficient. Therewas no difference in pattern recognition between the two groups; however, a preparersthorough self-review of evidence documented in detailed workpapers increased the pre-

parers ability to identify patterns in the evidence that may be beneficial in fraud detection.Finding that documentation method affects efficiency, cognitive processes, memory, and

effectiveness for some errors can inform practitioners and standards setters as they choosemethods of documentation. Audit researchers can also consider the effects of cognitiveprocessing and of error type when further examining audit documentation. Finally, our datasuggest that documentation by detailed workpapers may enhance pattern recognition (butnot error detection) at thereviewerlevelan implication that can be examined in futurestudies.

The next section of this paper provides a theoretical foundation and develops testablehypotheses. Following this is the method section which describes the participants and ex-

perimental task. Results are presented next, and the paper concludes with a discussion ofthe results, including implications for practice and limitations of the research.

THEORY AND MODEL DEVELOPMENTThe overall research question addressed in this study has two parts: (1) Does the type

of audit documentation used by auditors affect their cognitive processing, and (2) Do dif-ferences in cognitive processing affect auditors memory and performance? In the next twosections, we develop a model (shown in Figure 1) to explore these research questions andpropose testable hypotheses. As shown in Figure 1, our approach is to examine the cognitiveprocessing route auditors take as a result of the documentation method used. We do notmake any predictions of direct effects of documentation on memory or performance because

there is no theoretical basis for such. However, we discuss the possibility of direct effectsfollowing the model development.

Documentation and Cognitive Processing

We first examine how summary memo and detailed workpaper documentation methodsdiffer in their effect on the audit process. Gathering evidence involves actions such asexamining, vouching, tracing, observing, scanning, and recalculating (Louwers et al. 2007).These actions are performed using client documents such as invoices, purchase orders,master lists, etc. Regardless of the documentation method, the basic audit procedures arethe same.

However, the documentation method affects the audit process in the following manner.Auditors completing detailed audit workpapers not only examine source documents, theyalso enter details of the source documents into a worksheet which may include names

8/14/2019 31810984.pdf

4/19


FIGURE 1

Model of the Effects of Audit Documentation on Internal Control Testing

H4 +

H7 +

H8 +

H +

H5 +

H3 +

H2

H1 +

# ITEMS

TIME

# EXC

MEM

PATTERN

Factors Affecting

Processing

Outcome MeasuresType of

Documentation

METHOD

Variable Definitions:METHOD (0 Summary, 1 Detailed);

# ITEMS# of times evidence items are examined;TIMEtotal amount of time spent examining the evidence items (in minutes);MEM# of correct memory test responses (08 range);

PATTERNrecognition of a pattern in the evidence (1 yes, 0 no); and# EXC# of exceptions identified (02 range).

dates, amounts, indications of client authorization, etc. Auditors preparing summary memosoften do not document such specifics (Rich et al. 1997). They examine the client documentsand may only stop to document exceptions. The two documentation methods examined inthis study are likely to affect the cognitive processing of auditors during the audit task, butin different ways.

Cognitive processing is often viewed as a hierarchy of processing stages (Craik andLockhart 1972). At a shallow level of processing, encoding or basic recognition of thestimulus takes place. Sustained or deeper processing of the stimulus results in matching,pattern recognition, semantic comprehension, and eventual integration of the stimulus withprior knowledge. Deeper processing also leads to enhanced memory and is thought to resultfrom the amount of attention devoted to the stimulus, its compatibility with the analyzingstructures (or existing schema), and the amount of time available for processing (Craik andLockhart 1972). Moreover, the Spreading Activation Theory of semantic processing positsthat The longer a concept is continuously processed (either by reading, hearing, or re-hearsing it), the stronger the links between memory nodes will be (Collins and Loftus1975, 411).

These theories taken together indicate that both processing time and number of viewsof information are associated with deeper cognitive processing. However, deeper processing

only occurs if a greater number of presentations of the stimulus result in deeper analysis;The effects of repeated presentation depend on whether the repeated stimulus is merely

8/14/2019 31810984.pdf

5/19


processed to the same level or encoded differently on its further presentations (Craik andLockhart 1972, 126).

How, then, might cognitive processing differ due to documentation methods? We ex-amine proxies for these two theoretical aspects of cognitive processing (processing time

andnumber of views of evidence). The respective proxies are (1) the total amount of timespent examining the evidence items and (2) the total number of times evidence items areexamined.

Auditors preparing documentation by detailed workpapers during evidence gatheringwill spend a greater amount of time during the process than those preparing summarymemos. Logically, it should take an auditor a greater amount of time to physically documentthe evidence, compared to documenting errors only. Therefore, cognitive processing isenhanced with this documentation method. Alternatively, auditors preparing summarymemos may examine the evidence items more than once as they proceed through the testof controls because a lack of documented details may necessitate going back over evidenceitems. For instance, examination of a specific document may cause the auditor to review

previously inspected documents that appear similar to determine whether they are from thesame vendor/client, or because they look suspicious in some way, etc. Additionally, manyaudit procedures involve scanning the evidence in order to identify patterns or unusualitems. Therefore, examining the evidence items a greater number of times is more likelyto occur when using a summary form of documentation and will also enhance cognitiveprocessing.1 These predictions are formally stated as follows:

H1: Documentation by detailed workpapers results in more time spent on theaudit task than documentation by summary memos.

H2: Documentation by summary memos results in a greater number of timesaudit evidence items are examined than documentation by detailedworkpapers.

Cognitive Processing, Auditor Performance, and Memory

In the previous section, we made predictions about the effects of different documen-tation methods on different aspects of auditors cognitive processing during tests of controls.The next step is to examine how such processing affects performance and memory. Ourstudy captures two performance measures. The first performance measure involves errordetection. In a test of controls setting, error detection is essentially a mechanical processwhere one item is compared to another in accordance with the audit program instructions.Auditors apply procedures such as tracing, vouching, comparing documents for specificitems, reviewing the evidence for unusual items, etc. Auditors must examine the evidencelong enough to detect the errors. Therefore, processing time should be positively associatedwith error detection because auditors who take enough time to carefully make the com-parisons are more likely to detect errors than those who quickly move through the evidence.Examining evidence more times also improves the likelihood that an error will be detected.Reviewing the evidence additional times will likely provide auditors with the opportunityto find exceptions they had missed previously if encoding is enhanced during the repeatviewing of the evidence (Craik and Lockhart 1972). This discussion leads to the followingpredictions:

1 Viewing the evidence additional times will certainly lead to increased time spent on the task. Therefore, numberof views will be controlled for while testing the expected time differences due to documentation in H1

8/14/2019 31810984.pdf

6/19


H3: The amount of time spent on the audit task is positively related to errordetection.

H4: The number of times the evidence items are examined is positively relatedto error detection.

Our second measure of performance is pattern recognition. In auditing settings, patternrecognition is the ability to process patterns or configurations of stimuli that are impor-tant to subsequent judgments or decisions (Hammersley 2006, 311) and is used in nu-merous audit judgments including evidence evaluation (Hammersley 2006). The results ofprior audit research suggest that pattern recognition is a difficult task for auditors (Bedardand Biggs 1991), and deeper cognitive processing is required in order to recognize patterns(Craik and Lockhart 1972). Therefore, the enhanced cognitive processing associated withboth proxies should be positively associated with pattern recognition, so we make thefollowing predictions:

H5: The amount of time spent on the audit task is positively related to recognitionof a pattern in the evidence.

H6: The number of times audit evidence items are examined is positively relatedto recognition of a pattern in the evidence.

Our last outcome measure is that of memory of the audit evidence. We include memoryalong with our performance measures because the accuracy of long-term memory is acritical precursor to the accuracy of audit judgments and decisions (Sprinkle and Tubbs1998; Libby and Luft 1993; Moeckel and Plumlee 1989). Stronger memory traces are a

byproduct of deeper cognitive processing (Craik and Lockhart 1972). Therefore, both proc-essing measures should be related to better memory of audit evidence.2

H7: The amount of time spent on the audit task is positively related to memoryof the audit evidence.

H8: The number of times audit evidence items are examined is positively relatedto memory of the audit evidence.

We have hypothesized that preparation of summary memos increases the number of

times evidence items are examined, and preparation of detailed workpapers increases timespent on the audit task. Both of these measures of cognitive processing are expected toimprove performance and memory. Another interesting question is whether one measure ofcognitive processing is superior to the other. If so, we would expect one method of docu-mentation to result in greater performance and memory. However, theory does notrankthefactors affecting cognitive processing but merely asserts that time and number of viewslead to deeper processing (Craik and Lockhart 1972). Therefore, we have no theoreticalbasis to predict a direct effect of documentation on performance or memory. However,discovering differences in performance and memory as a result of the cognitive processingfocus (enhanced processing time or number of views of evidence items) has important

2 We do not expect memory to be related to the two performance measures because the memory test in this studywas designed to measure memory of general audit evidence, not specific errors. Furthermore, the embeddederrors are unrelated to the pattern so these two performance measures should not be related

8/14/2019 31810984.pdf

7/19


implications to the practice of auditing. For example, when auditors view evidence itemsmultiple times, will successive views result in higher levels of processing or same-levelprocessing? Thus, we propose the following nondirectional research question:

RQ1: Does the method of documentation have a direct effect on audit perform-ance or auditors memory?

Cognitive Processing after Documentation

Pattern recognition is a difficult task for auditors, and deeper cognitive processing whileperforming the audit procedures is expected to enhance performance. In addition, a com-pleted, detailed audit workpaperprovides opportunity for enhanced cognitive processingbeyond preparation of the workpaper. To the extent that preparers perform a self-review oftheir workpapers, we should notice improvement in pattern recognition beyond that relatedto documenting the evidence if auditors encode the evidence differently during this addi-tional examination (Craik and Lockhart 1972). Thus, we expect the amount of time spent

by the preparer reviewing detailed workpapers upon completion is positively related torecognition of the pattern.3

H9: The amount of time spent reviewing detailed workpapers upon completionis positively related to recognition of a pattern in the evidence.

RESEARCH METHODParticipants

A total of 286 staff and senior level auditors from three of the (then) Big 5 accountingfirms participated in the experiment. Responses from 35 of the participants were not usabledue to failure to complete the task or loss of data because of premature exit from thecomputer program (responses were saved to an output file upon final exit), leaving 251complete responses. Of these complete responses, 16 participants were deemed to havefailed/disregarded the documentation manipulation, and 27 were deemed to lack an under-standing of the task (see Manipulation Check section following). Three participants fellinto both of these categories, resulting in an additional 40 discards, and a final sample sizeof 211.

Materials and Procedures

The experiment was administered during the firms training sessions. After a briefintroduction by the researchers, computer diskettes containing the different versions of the

experiment were randomly distributed to the participants. The auditors completed the ex-periment on their own computers, in the presence of the researchers and firm instructors,and returned the diskettes to the researchers upon completion. No time limits were imposed.Actual total time for the entire experiment ranged from 30 to 60 minutes.

The case study was developed specifically for this study.4 The study began with back-ground information related to a hypothetical client and an introduction to the experimental

3 The self-review variable of H9 is not included in the model because it relates only to review ofdetailedworkpapers. We will test H9 separately from the model. We do not examine self-review in the memo-form ofdocumentation because memos require only that exceptions be documented (not details of all evidence items).Therefore, reviewing a memo does not allow the auditor to see the entireevidence set again; the auditor canonly review the exceptions already noted. Thus, cognitive processing is not likely enhanced.

4 The experiment was pilot tested with undergraduate auditing students, doctoral students, and accounting faculty.Minor changes were made after pilot testing to improve comprehension of questions and reduce the amount oftime required to complete the task

8/14/2019 31810984.pdf

8/19


task.5 An audit program (modified version from Ricchiute 1998) was provided next (seethe Appendix), followed by the test of controls task screen that included either a detailedworkpaper (spreadsheet) or summary memo to be completed, as well as the sample items,authorization memos, and accounts receivable subsidiary ledgers needed to perform the test

of controls. These items were accessible one-at-a-time via the drop-down menus on thecomputer screen. For example, participants would click on the SAMPLE box. A drop-downlist of the 20 sample items would appear and the participants would then select one ofthe 20 sample items to view from the drop-down list. Then they would click on theAUTHORIZATION MEMOS box and select the corresponding memo from this drop-downlist. Next, they would click on the ACCOUNTS RECEIVABLE SUBSIDIARY LEDGERbox and select the corresponding ledger item from this drop-down list. The informationpresented to the participants in the two documentation groups was identical. At any timewhile completing the test of controls, the participants could refer back to the audit programby clicking on the button labeled AUDIT INSTRUCTIONS. The final step in the auditprogram required the participants to review all of the write-off entries for unusual items;

this step serves to prompt participants to self-review their work.Upon completion of the test of controls, participants were provided with a text box and

prompted to identify anything unusual they had noticed while performing the test of controlsthat should be investigated further. They were then given a distracter task, memory test,and exit questionnaire to conclude the experiment. The computer program required partic-ipants to complete each response box on a current screen in order to proceed to the nextscreen so there were no missing responses.

Measurement of Variables

Two methods of documentation were manipulated by providing subjects with either (1)

a detailed workpaper (spreadsheet) in which the format was already established such thatparticipants had to fill in all required fields and add necessary tick-marks for all of thesample items, or (2) a summary memo (text box) stating Please describe below the pro-cedures you performed, and explain in full detail any exceptions noted. Two proxies fordepth of processing were obtained. The computer program recorded the amount of timeparticipants spent examining each piece of audit evidence; these amounts were summed toprovide a total amount of time (in minutes) spent on the test of controls (TIME).6 A countwas also made of the number of times participants selected the evidence items (# ITEMS).This measure includes the initial examination of each piece of evidence as well as repeatexaminations, summed together for all sample items. Participants correct responses to aneight-item recognition memory test were summed for a measure of memory accuracy(MEM).

5 Following the introduction, participants were informed that their work could be selected for review upon com-pletion of the study in order to enhance motivation and effort. For purposes of another study (Payne et al. 2007),two different review methods were described, followed by a forced manipulation check. All statistics have beencalculated both with and without inclusion of this variable. Type of review does not change the significance ofthe results shown in this study. In addition, we tested a documentation method by review method interactionand it was not significant for any of the variables (direct effects) or any of the paths in the model.

6 BecauseTIMEis a total measure of time (consistent between documentation groups) used in the path modelfor all participants, it does include the self-review time for detailed workpaper preparers discussed below. Forpurposes of a separate statistical analysis to test the effects of self-review time (H9), self-review time is laterseparated from workpaper preparation time.Because participants could have one document from each of the three types of documents open at the sametime (i.e., the sample write-off item, authorization memo, and accounts receivable subsidiary ledger all for oneparticular customer), thetotaltime is notactualtime for the task. An approximation of actual time would bethe total time shown in Table 1 divided by 3

8/14/2019 31810984.pdf

9/19


Two performance measures were obtained. Embedded in the evidence were two excep-tions, a posting error, and a missing authorization. Additionally, a pattern was embeddedin the sample that could potentially indicate employee fraud via circumvention of controls.The pattern relates to the fact that the internal control policy required write-offs greater

than $500 be approved (as stated in the audit program); yet five sample items were justunder this approval threshold. Numerous items just below an authorization level are acommonly recognized red flag in fraud investigation (ACFE 2006; McNeal 2006).

Participants workpapers, memos, and the text box requesting unusual items needingfurther investigation were all reviewed and coded by the researcher and a separate raterblind to the experimental hypotheses.7 Coding identified the number of exceptions (# EXC)correctly identified (02 range) and whether the pattern (PATTERN) was identified (yes orno).

To test the effect of preparer self-review time on recognition of the pattern, a proxyfor time spent reviewing the completed workpaper was obtained. Self-review time wasmeasured as the time interval starting when the participant clicked on the last item of

evidence and ending when the participant left the task screen.

RESULTSManipulation Check

Sixteen participants provided with the memo-form of documentation ignored the in-structions toexplain in full detail any exceptions notedand had actually written detailedinformation abouteverysample item. These 16 participants were deemed to have ignoredor misunderstood the manipulation and were discarded. Additionally, 27 participants didnot understand the $500 threshold for approval of write-offs and listed every write-off

without approval as an exception. Responses for these participants were also discarded.

8

Demographics

Demographic measures collected in the exit questionnaire indicate the participants werean average age of 25.1 years, had performed the experimental task in actual practice anaverage of 2.4 times, and had an average of 1.5 years of audit experience. The participantsdeemed the experimental task to be realistic (average 6.9 on a 010 point scale), andindicated they put forth a good deal of effort while completing the experiment (average 7.7on a 010 point scale). The final sample includes 107 female and 104 male participants.There were no significant differences in demographic variables between experimental

groups.

Descriptive Statistics

Descriptive statistics for each of the variables included in the predicted model areshown by experimental group in Table 1. Auditors who prepare summary memos examineevidence items a greater number of times, spend less time, and have a better memory of

7 Both the researcher and the separate rater were aware of the treatment since both examined spreadsheets andmemos. The researcher was aware of the hypotheses but the separate rater was not. Both independently examinedall 251 documents and upon comparison there was onlyonediscrepancy, which was resolved (one of the ratershad simply overlooked a tick-mark denoting an error).

8 We examined both general audit experience and task-specific experience to determine whether experience playeda role in the participants understanding of memo-form documentation or authorization thresholds. Experiencewas not significant in either case

8/14/2019 31810984.pdf

10/19


TABLE 1

Descriptive Statistics by Method of Documentation

ActualRange

Means (Std. Dev.)

(n 104)Summary

Memo

(n 107)Detailed

Workpaperp-value*

(Two-Tailed)

# ITEMS 49202 89.7 (28.4) 74.9 (21.0) .00

TIME 24123 63 (24) 72 (16) .00

MEM 28 6.7 (1.2) 6.3 (1.3) .02

PATTERN 0, 1 36% 31% .47

# EXC 02 1.7 (.5) 1.8 (.4) .14

POST 0, 1 .93 (.25) .93 (.26) .83

AUTH 0, 1 .81 (.40) .91 (.29) .04

* Reported p-values are for T-tests; nonparametric tests produce the same results.Variable Definitions:# ITEMS# of times evidence items are examined;

TIMEtotal amount of time spent examining the evidence items (in minutes);MEM# of correct memory test responses (08 range);

PATTERNrecognition of a pattern in the evidence (1 yes, 0 no);# EXC# of exceptions identified (02 range);POSTposting error identified (0 no, 1 yes); and

AUTHauthorization error identified (0 no, 1 yes).

the evidence than auditors who prepare detailed workpapers. There are no significant dif-ferences in performance between the groups for pattern recognition or for total errors de-

tected. However, detailed workpaper preparers were more successful at detecting one typeof error (authorization). Pearson correlation coefficients are also provided for these variablesin Table 2. The t-tests and correlations show support for many of the predicted relationships.However, path analysis was used to test the predicted model so that both direct and indirecteffects could be observed and measured. Path analysis examines the effect of a predictorvariable on an outcome variable, while controlling for other predictors.9

Path Analysis

In testing each path, nonsignificant standardized partial regression coefficients wereremoved from the model (Pedhazur 1982). All possible paths were tested in order to test

our research question relating to direct effects of documentation. In addition, we include acontrol for the correlation between the two cognitive processing variables; examining anitem a greater number of times will naturally increase the amount of total time spent.However, by controlling for the increase in total time due to number of views, we canmeasure the effect of increased total time as a result of spending more time with eachindividual piece of evidence. The resulting model includes only significant paths and isshown in Figure 2.10

9 Path analysis is the preferred statistical method because there are no latent variables included in the model thatwould call for structural equation modeling (Raykov and Marcoulides 2000). However, structural equationanalyses were also performed using EQS in order to measure the model fit. Various measures indicated a goodmodel fit (e.g., the Root Mean Square Error of Approximation .048). The SEM results support those obtainedthrough path analysis.

10 Additional analysis of the model using only the participants who examined 100 percent of the evidence items(184 out of 211) resulted in the same significant paths as the full model

8/14/2019 31810984.pdf

11/19


TABLE 2

Pearson Correlation Coefficients

DOC # ITEMS TIME MEM PATTERN # EXC POST AUTH

DOC 1.00# ITEMS .29** 1.00

TIME .22** .27** 1.00

MEM .16* .23** .06 1.00

PATTERN .05 .16* .22** .02 1.00

# EXC .10 .15* .23** .12 .07 1.00

POST .02 .11 .14* .02 .12 .62** 1.00

AUTH .14* .11 .18** .17* .00 .82** .05 1.00

*, ** significant at the .05 level and .01 levels, respectively, two-tailed.

Variable Definitions:

DOCform of documentation (1 detailed, 0 summary);# ITEMS# of times evidence items are examined;

TIMEtotal amount of time spent examining the evidence items (in minutes);MEM# of correct memory test responses (08 range);

PATTERNrecognition of a pattern in the evidence (1 yes, 0 no);# EXC# of exceptions identified (02 range);POSTposting error identified (0 no, 1 yes); and

AUTHauthorization error identified (0 no, 1 yes).

FIGURE 2

Results of Path Analysis

[H6] .11 (.06)

[H8] .23 (.00)

.36 (.00)

[H3] .19 (.00)

[H5] .22 (.00)[H1].32 (.00)

[H2] .29 (.00)# ITEMS

TIME

PATTERN

MEM

# EXC

METHOD

Standardized Coefficients (p-values, 1-tailed) shown for each path.Variable Definitions:METHOD (0 Summary, 1 Detailed);

# ITEMS# of times evidence items are examined;TIMEtotal amount of time spent examining the evidence items (in minutes);MEM# of correct memory test responses (08 range);

PATTERNrecognition of a pattern in the evidence (1 yes, 0 no); and# EXC# of exceptions identified (02 range).

The expectations related to the first link in the model are both supported (H1 and H2).

The method of documentation had a direct effect on both depth-of-processing measures.Participants preparing a detailed workpaper spent more time on the task (p .00, one-tailed) and participants preparing the summary memos examined the evidence items a

8/14/2019 31810984.pdf

12/19


greater number of times (p .00, one-tailed). The number of times evidence items areexamined was also positively related to the total time spent on the task (p .00, one-tailed).

We expected time to be positively associated with error detection. The relationship is

significant (p

.00, one-tailed) and provides support for H3. However, the number of timesevidence items are examined was not significantly related to error detection. Thus, H4 wasnot supported. As expected, both measures of cognitive processing were positively asso-ciated with recognition of the pattern. The amount of time was significant (p .00, one-tailed) providing support for H5 and the number of times evidence items are examined wasmarginally significant (p .06, one-tailed) providing some support for H6. Only the numberof times evidence items are examined significantly (p .00, one-tailed) improved mem-ory of the evidence, supporting H8. We expected time to be positively associated withmemory but the relationship was not significant; H7 is not supported.

Theoretically, there is no distinction made between the two measures of cognitive proc-essing, leading us to question whether performance or memory would be more improved

as a result of additional processing time or repeated viewing of the evidence. The pathanalysis shown in Figure 2 controls for the strong correlation between these processingmeasures (p .00) and overall, shows that neither processing measure is superior. Instead,it appears general memory of audit evidence is improved from repeated viewing, and dif-ferent performance measures are affected by different methods of cognitive processing.Both time and repeated viewing of evidence improve pattern recognition, while only in-creased time improves error detection. In sum, the method of documentation affects theprocess that participants use to perform the test of controls via spendingmore timethrough-out the task (detailed workpaper) orviewing the evidence items a greater number of times(summary memo). Deeper processing is positively associated with performance.

We posed a research question to examine whether the form of documentation wouldhave a direct effect on performance or memory. As noted in Tables 1 and 2, memory issignificantly greater for the memo group (p .02), and the path model shows this is dueto repeated viewing of the evidence. That is, repeated viewing mediates the relationshipbetween documentation and memory. The direct effect of documentation on pattern rec-ognition is not significant (p .47, two-tailed). Because the direct effect of documentationon error detection is marginally significant (p .14, two-tailed), we further examined thetwo embedded errors individually. Results (shown in Table 1) show that preparing detailedworkpapers is related to greater error detection for the authorization error (p .04, two-tailed), but not the posting error (p .83, two-tailed). In the total sample, only 15 partic-ipants did not identify the posting error compared to 30 who did not identify the authori-

zation error. These results suggest that for this particular internal controls task, identifyingproper authorization was somewhat more difficult than identifying proper posting, and doc-umentation by detailed workpapers helped with the more difficult identification.

In our final hypothesis, we examine the relationship between the amount of time spentinself-reviewof completed workpapers and recognition of the pattern. The logistic regres-sion results are shown in Table 3. These results support our expectation since self-reviewtime significantly increases the likelihood of identifying the pattern (p .04, one-tailed),after controlling for the effects of documentation time. In light of the significance of reviewtime identified in these analyses, we repeated the path analysis for all participants using atotal time measure that does not include review time (i.e., total time up to the point where

the participants clicked on the last piece of evidence). Both documentation time and review

8/14/2019 31810984.pdf

13/19


TABLE 3

Logistic Regression Results: Pattern Recognition

Logistic Regression Model:

Pattern 0 1D 2R .Model Chi-square 8.065; p-value .018 (two-tailed).

Parameter Estimate Chi-Square p-value (one-tailed)

Constant 3.006 7.830 .00

D .000 2.951 .04

R .004 3.138 .04

D total amount of time spent documenting the evidence items (in minutes).R total amount of time spent reviewing the completed detailed workpaper (in minutes).

time were still significant using these measures (p .04 and p .04, respectively, one-tailed). As expected,post-hocanalyses show review time does not enhance pattern recog-nition for the memo documentation group (p .37, two-tailed).

Additional AnalysesPattern Recognition

Successful pattern recognition requires auditors to acquire and combine all necessarycues, whereas fixating on nonimportant cues or surface features of the task may impairrecognition (Bedard and Biggs 1991). A review of the specific evidence items making upthe pattern shows that only five participants from each documentation group failed to ex-amine all of the items. We further found no significant correlations between time spent ornumber of views of nonrelevant sample items (indicative of fixation) and pattern recogni-tion, all of which suggests that failure to recognize the pattern was likely due to an inabilityto combine the cues properly.11

Documentation and the Review ProcessFinally, we considered how the form of documentation might potentially affect the

review process because previous studies have found documentation can affect reviewersjudgments (e.g., Tan and Trotman 2003; Tan and Yip-Ow 2001; Yip-Ow and Tan 2000).Specifically, in our study, we wanted to know whether the evidence documented would besufficient for a reviewer to potentially identify errors or patterns missed by the preparers.

In the case of summary memo documentation, the answer is no because the auditorsonly documented exceptions and other problems they found. Documentation by detailedworkpapers would not help the reviewer identify the participants missed exceptions for theposting error because in every case the missed exception was incorrectly tick-marked toindicate it was okay. Also, documentation by detailed workpapers would not provide muchhelp for a reviewer to identify the missed authorization error because in most cases partic-ipants incorrectly tick-marked this exception as well. In four cases, however, the error wasincorrectly tick-marked but the authorized dollar amount was omitted (a tickmark was

11 In addition, there were no significant differences between those who recognized the pattern and those who didnot due to demographics (firm, age, gender, task experience, general audit experience, and position in the firm).Furthermore, a review of the participants information search patterns did not reveal any unusual patterns asalmost every participant proceeded through the evidence in the order it was presented and performed the stepsof the task simultaneously for each sample item

8/14/2019 31810984.pdf

14/19


placed in the column rather than the dollar amount). It is possible that a review commentinstructing the preparer to add the authorized dollar amount might lead the preparer to findthe mistake. For pattern recognition, however, documentation by detailed workpapers wouldlikely be beneficial at the review stage. In all but five cases, the correct dollar amounts

were documented on the workpaper. For the five cases where dollar amounts were missing,again, a review comment instructing the preparer to add these amounts would make theworkpaper complete so that the reviewer could potentially notice the pattern for these aswell.

DISCUSSIONOur purpose in this study was to examine how different methods of workpaper docu-

mentation might affect the cognitive processing used by auditors, and whether processingdifferences are related to memory and performance. We find that the method of documen-tation does directly affect processing. Documenting full details of all evidence items ex-amined increases cognitive processing through the amount of time spent on the task. Pre-

paring a summary memo increases cognitive processing through examining the evidenceitems additional times. Therefore, cognitive processing is enhanced with bothmethods ofdocumentation. However, these differential enhancements to processing have differentialeffects on memory and performance. More time spent on the task improves both error de-tection and recognition of a potentially fraudulent pattern in the evidence, but does notimprove memory of the audit evidence. Examining the evidence items additional times im-proves pattern recognition and memory, but not error detection.

In our experimental setting, the number of times the evidence items are examined wasnot positively associated with error detection as predicted. Upon review of the informationsearch patterns of participants, we believe this may be because examining evidence items

a greater number of times does not necessarily result inretracing the evidence. In otherwords, if an error is missed the first time through, it may not be detected later unlessbothitems of comparison are simultaneously reviewed again so that a second trace can beperformed. Another predicted relationship that was not supported was that between timeand memory. It is not clear why we did not find this relationship to be significant. Sincetime is positively related to performance, it appears to be an acceptable proxy for cognitiveprocessing. Therefore, time should also be related to memory. However, Mayer (1984) notesthat sometimes deeper levels of processing actually result in reduced retention of factsbecause the details are changed as they are integrated with prior knowledge.

Examination of the effects of documentation methods on detection of individual errorsshowed detailed documentation was significantly related to error identification for an au-

thorization error but not a posting error. Perhaps having to document the specific step forthe specific item results in closer attention paid. This suggests that when examining auditorerror detection, researchers must consider carefully the nature of each error. The amountof cognitive processing necessary for successful error detection likely depends upon thetype of error.

This study has several implications for audit practice. First, although auditors whoprepare detailed workpapers may tend to examine evidence items fewer times, they maybe able to enhance their memory of the evidence through number of views. Second, ourresults also highlight the importance of self-review. When auditors prepare detailed work-papers, thorough self-reviews of evidence in the workpapers upon completion increase

the preparers ability to identify patterns in the evidence that may be beneficial in fraud

8/14/2019 31810984.pdf

15/19


detection. Finally, our additional analyses also show that documentation by detailed work-papers provides the evidence that may potentially enhance effectiveness at the reviewerlevel in terms of pattern recognition, but not error detection.

The instrument used in this study was developed before AS No. 3 was issued. Thus

the documentation requirements in the experiment do not meet the specific requirementsof the Standard, which are Documentation ... should include identification of the itemsinspected ... by indicating the source from which the items were selected and the specificselection criteria (PCAOB 2004, para. 10) because we did not ask participants in the memogroup to indicate the source and selection criteria. However, we do not believe this infor-mation would have affected our conclusions. Additionally, we only embedded two errorsin the task to help increase realism. As most participants were able to identify both errors,a ceiling effect may have reduced our ability to find a significant direct effect of documen-tation on performance. Since our results do show differences in error detection betweendocumentation groups for one of the exceptions, it seems that more research is needed toidentify certain types of errors that are harder to detect than others and whether documen-

tation methods affect error detection differently based on difficulty.No time limits were imposed in this experiment; time limits may affect the results and

should be examined in future research. This is very interesting because it is not clear whatwould happen. Would time pressure cause participants to forego repeated viewing of theevidence, to work faster (more so in the detailed workpaper group) or to forego self-reviewof detailed workpapers? Finally, the memory test used in this study was given soon aftercompletion of the test of controls, though a distracter task was given in between.

APPENDIXAUDIT PROGRAM

8/14/2019 31810984.pdf

16/19


TEST OF CONTROLS TASK SCREENWORKPAPER GROUP

TEST OF CONTROLS TASK SCREENMEMO GROUP

8/14/2019 31810984.pdf

17/19

8/14/2019 31810984.pdf

18/19


Pedhazur, E. J. 1982.Multiple Regression in Behavioral ResearchExplanation and Prediction. 2ndedition. New York, NY: CBS College Publishing.

Public Company Accounting Oversight Board (PCAOB). 2003. Audit Documentation Roundtable.September 29. Washington, D.C.: PCAOB.

. 2004.Audit Documentation and Amendment to Interim Auditing Standards.Auditing Standard

No. 3. Release No. 2004006. June 9. Washington, D.C.: PCAOB.Purvis, S. E. C. 1989. The effect of audit documentation format on data collection. Accounting,

Organizations and Society14: 551563.Raykov, T., and G. A. Marcoulides. 2000. A First Course in Structural Equation Modeling. Mahwah,

NJ: Lawrence Erlbaum Associates.Ricchiute, D. N. 1998.Auditing and Assurance Services. 5th edition. Cincinnati, OH: South-Western

College Publishing.. 1999. The effect of audit seniors decisions on working paper documentation and on partners

decisions.Accounting, Organizations and Society24 (2): 155171.Rich, J. S., I. Solomon, and K. T. Trotman. 1997. Multi-auditor judgment/decision making research:

A decade later.Journal of Accounting Literature 16: 86126.

Shankar, P. J., and H. T. Tan. 2006. Determinants of audit preparers workpaper justifications.TheAccounting Review81 (2): 473497.

Sprinkle, G. B., and R. M. Tubbs. 1998. The effects of audit risk and information importance onauditor memory during working paper review. The Accounting Review73 (4): 475502.

Tan, H. T., and J. Yip-Ow. 2001. Are reviewers judgments influenced by memo structure and con-clusions documented in audit workpapers? Contemporary Accounting Research 18 (4): 663678.

, and K. T. Trotman. 2003. Reviewers responses to anticipated stylization attempts by pre-parers of audit workpapers.The Accounting Review 78 (2): 581604.

Yip-Ow, J., and H. T. Tan. 2000. Effects of the preparers justification on the reviewers hypothesisgeneration and judgment in analytical procedures. Accounting, Organizations and Society 25(2): 203215.

8/14/2019 31810984.pdf

19/19

31810984.pdf

Documents