an audit partner-led field intervention in ...an audit partner-led field intervention in fraud...
TRANSCRIPT
AN AUDIT PARTNER-LED FIELD INTERVENTION IN FRAUD BRAINSTORMING
Sean Dennis
PhD Candidate
University of Wisconsin – Madison School of Business
Karla M. Johnstone
EY Professor
University of Wisconsin – Madison School of Business
975 University Avenue
Madison, WI 53706
June 1, 2014
We thank participants at research workshops at the University of Wisconsin-Madison, University
of Notre Dame, University of Missouri – Columbia, University of Connecticut, and NEBARS.
We express particular appreciation to comments from Jean Bedard, Jere Francis, Jeremy Griffin,
Brian Mayhew, Nate Newton, Dave Ricchiute, Terry Warfield, and Arnie Wright. We thank the
sponsoring audit firm for monetary support of this project, and we thank leadership and
participants at two other audit firms. Johnstone acknowledges support through her professorship
with EY, as well as the Andersen Center for Financial Reporting at the University of Wisconsin
School of Business. Dennis acknowledges financial support from the Accounting Doctoral
Scholars Program. Johnstone also acknowledges helpful comments of Noel Harding and Ken
Trotman during her Visiting Professorial Fellow sabbatical at the University of New South
Wales. Finally, we express our appreciation to Eric Condie and Amy Tegeler, who served as
diligent coders of qualitative data.
AN AUDIT PARTNER-LED FIELD INTERVENTION IN FRAUD BRAINSTORMING
SUMMARY: In a field experiment, we manipulate guidance to audit partners through an
intervention intended to affect the approach they take in leading fraud brainstorming sessions for
actual audit engagements. We examine how this audit partner-led field intervention is associated
with processes and outcomes of these sessions. We predict and find associations between the
intervention and some, but not all, of our process and outcome measures. Analyses of
quantitative data suggest the intervention improves processes and outcomes related to fraud risk
factor identification, but not those related to planned fraud risk responses. Analyses of qualitative
data reveal associations between the intervention and attributes of planned fraud risk responses,
but not the types of fraud risk factors that were identified. This suggests that while the fraud risk
profile of clients in the field does not differ by experimental condition, the audit responses to
these circumstances do differ by experimental condition.
Keywords: Audit planning, field experiment, fraud brainstorming, professional skepticism.
AN AUDIT PARTNER-LED FIELD INTERVENTION IN FRAUD BRAINSTORMING
I. INTRODUCTION
Although audits are not necessarily designed to detect fraud, regulators and users continue to
demand quality in auditors’ fraud detection capabilities (see e.g., Carmichael 2004; Hammersley
2011). Professional standards require auditors to conduct a fraud brainstorming session as part of
each audit (AICPA 2002b), illustrating the importance of this task. The PCAOB has criticized
auditors with regard to the quality of brainstorming and has expressed concern that auditors lack
appropriate professional skepticism in this task (PCAOB 2007, 4). Prior research reports
considerable variation in the extent to which auditors emphasize skepticism and deploy resources
in brainstorming (Bellovary and Johnstone 2007; Brazel et al. 2010). Recent research illustrates
the important role that the audit partner plays in brainstorming (e.g., Carpenter and Reimers
2013; Gissel 2013). The purpose of this study is to examine how an audit partner-led field
intervention affects the processes and outcomes of fraud brainstorming sessions.
The term “tone at the top” emerged in the field of auditing to describe organizational
leadership with respect to internal control over financial reporting. COSO (2011, p. 27) notes
that leadership affects the control environment in that “management and the board of directors or
equivalent oversight body are expected to lead by example”. COSO (2011, p. 28) goes on to state
that “tone is impacted by the personal conduct of management…”. We refine this focus to
investigate the role of audit partner engagement-level leadership. We develop an intervention
intended to improve partners’ leadership by facilitating a fraud brainstorming session that
emphasizes productive interpersonal interaction, motivates effectiveness and efficiency, and
promotes professional skepticism. The audit partner-led field intervention (hereafter, “the
2
intervention”) is a memo to the audit partner with specific, actionable suggestions that include
the following instructions:1
Emphasize the session as a training
opportunity.
Discuss the importance of effective
and efficient fraud brainstorming.
Discuss the importance of professional
skepticism targeted at specific accounts
with a potentially higher level of fraud
risk.
Emphasize both effectiveness and
efficiency to promote an
appropriately calibrated response to
fraud risk.
Discuss the importance of professional
skepticism in general throughout the audit.
Prior research provides evidence relevant to the role of the audit partner in fraud
brainstorming sessions using hypothetical cases in laboratory settings. In an early study,
Carpenter (2004) finds that teams spend more time brainstorming when the audit partner
emphasizes effectiveness as opposed to efficiency. Carpenter and Reimers (2013) find that fraud
risk assessments are higher when the partner emphasizes professional skepticism and that fraud-
related audit procedures are only responsive to fraud risk when the partner emphasizes
professional skepticism. Additionally, Gissel (2013) finds that audit staff are more willing to
share private information related to fraud risks during brainstorming when the audit partner
creates an atmosphere that is viewed as “psychologically safe,” which leads to more accurate
fraud risk assessments. We extend this research by investigating whether an intervention can
affect the approach partners take in leading an actual brainstorming session for an audit
engagement in the field and how that intervention affects brainstorming processes and outcomes.
We conducted the field experiment in conjunction with, and immediately following, the
fraud brainstorming sessions of a sample of 77 audit engagements conducted from July 2013
1 AU 316, Consideration of Fraud in a Financial Statement Audit (formerly SAS No. 99) does not explicitly require
audit partners to perform all of these actions (AICPA 2002b). We developed these instruction in collaboration with
senior leadership at the sponsoring audit firm. These individuals have significant experience with audit firm
methodology and training. The specific content of the intervention was designed to emphasize best practices in a
concise manner that facilitates effective implementation of professional standards and the firm’s methodology.
3
through January 2014 at three audit firms (two Big 4 firms and one international firm). Each
engagement team in the sample was randomly assigned to either the treatment condition (N = 37
partners) or the control condition (N = 40 partners). Each partner in the study received a memo
informing him/her that his/her audit engagement will be involved in a brainstorming research
study; the memo in the treatment condition also included the intervention. After the
brainstorming sessions occurred, a designated audit firm contact person notified the audit
manager and senior on each engagement that their engagement was involved in the study. The
manager and the senior each received a survey to complete individually. The survey measures
audit managers’ and seniors’ perceptions of the topics partners discussed and the issues partners
emphasized during brainstorming, the processes of the session, and the outcomes of the session,
along with descriptive details about the sessions. We received 75 and 73 completed surveys from
managers and seniors, respectively. We expect the intervention to be positively associated with
the quality of brainstorming processes and outcomes.2
The results show that audit managers’ and seniors’ perceptions of the topics partners
discussed and the issues they emphasized during fraud brainstorming differ in certain respects by
experimental condition. In the treatment condition, there is a greater likelihood that the audit
partner discussed his/her prior experiences with fraud during brainstorming, a greater likelihood
that the partner addressed the issues of effectiveness and efficiency to promote an appropriately
calibrated response to fraud risk, and a greater likelihood that the partner discussed the
importance of professional skepticism with respect to specific accounts on the engagement that
have a higher level of fraud risk, compared to in the control condition. However, in both the
2 Note that some partners may exhibit the behaviors described in the intervention even if they are in the control
condition. Therefore, it is an empirical question as to whether there will be significant differences across
experimental conditions in manager/senior perceptions of the approach audit partners take during brainstorming with
respect to topics they discuss and the issues they emphasize.
4
treatment and control conditions, there were about equal levels of the extent to which the partner
emphasized brainstorming as a training/professional development opportunity, about equal levels
of the extent to which the partner discussed the importance of effective and efficient
brainstorming in general, and about an equal likelihood that the partner discussed the importance
of professional skepticism in general throughout the audit. We interpret this as evidence that
partners incorporate these latter three discussion topics and issues of emphasis in brainstorming
sessions in practice as a matter of professional routine.
The multivariate hypothesis-testing results reveal the intervention is associated with
several fraud brainstorming processes. Consistent with expectations, the intervention is
associated with greater increases in self-assessed manager and senior professional skepticism
both in general and with respect to specific accounts with a higher level of fraud risk. Also
consistent with expectations, the intervention is associated with a greater extent of discussion
about how management might perpetrate fraud, as well as longer brainstorming sessions (by
about ten minutes on average); however, we find no association between the intervention and the
extent of discussion about audit responses to fraud risk.
We also analyze quantitative and qualitative measures of fraud risk identification
outcomes and fraud risk response outcomes. Overall, inferences related to fraud brainstorming
outcomes complement the inferences related to brainstorming processes. The intervention is
associated with quantitative measures of fraud risk factor identification outcomes, but not
qualitative measures of fraud risk factor identification outcomes. Specifically, we predict and
find that the intervention is associated with the identification of more fraud risk factors overall
and more new fraud risk factors; this complements the positive association between the
intervention and the extent of discussion about how management might perpetrate fraud.
5
However, we find no association between the intervention and the percentages of fraud risk
factors that relate to revenue recognition or management override of controls, respectively.
Therefore, the fraud risk profile of clients in the field does not differ by experimental condition.
Inferences concerning fraud risk response outcome measures similarly complement the
inferences from the analyses of process measures. Specifically, we do not find any of the
predicted associations between the intervention and quantitative measures related to planned
audit procedures, including no association between the number of planned procedures, the
number of new planned procedures, or the number of planned procedures intended to incorporate
an element of unpredictability to respond to fraud risk. Also in contrast with expectations, the
intervention is associated with a lower likelihood of tailoring the audit plan to the current year
audit by eliminating fraud risk responses used in prior years. The lack of predicted associations
between the intervention and quantitative measures of planned audit procedures result
complements the lack of an association between the intervention and the extent of discussion
during brainstorming about audit responses to fraud risk. These results are unexpected, but
consistent with prior literature that shows auditors often have difficulty linking evaluations of
fraud risk with appropriate fraud risk responses (see, e.g., Hammersley 2011), and with
substantive test modifications in other audit tasks (Mauldin and Wolfe 2014).
Interestingly, we do find associations between the intervention and qualitative
characteristics of fraud risk response outcomes. Specifically, the intervention is positively and
marginally significantly associated with percentage of fraud risk responses that relate to the
nature of planned audit procedures. Additionally, the intervention is negatively associated with
the percentages of fraud risk responses that relate to the extent and timing, respectively, of
planned audit procedures. These results suggest that, compared to engagement teams in the
6
control condition, engagement teams in the treatment condition focused more on what was being
done to respond to fraud risk and less on how much was being done and when it was being done.
Results for control variables reveal several notable insights. Higher levels of audit partner
experience on the client are associated with decreases in self-assessed manager and senior
professional skepticism during the brainstorming session, less discussion about how management
may perpetrate fraud, the identification of fewer new fraud risk factors during the session, more
discussion about potential management override of controls, and a reduced willingness to tailor
the audit plan by eliminating fraud risk responses used in the prior year audit. Thus, there
appears to be some inertia with respect to longer audit partner tenure on an engagement that
yields “stickiness” in some processes and outcomes of brainstorming sessions, which has
interesting implications for research on partner tenure (e.g., Bedard and Johnstone 2010). Results
also reveal greater manager experience on the client is associated with more discussion about
how management might perpetrate fraud and more discussion about audit responses to fraud
risks, suggesting possible benefits to continuity at the manager level. Similarly, the engagement
team’s level of expertise on the client is associated with more discussion about how management
might perpetrate fraud and more discussion about audit responses to fraud risks.
Our results extend prior research that shows the benefits of interactive brainstorming
(e.g., Carpenter 2004; Carpenter 2007; Hoffman and Zimbelman 2009; Lynch et al. 2009; and
Carpenter et al. 2011), auditing-specific interventions intended to facilitate brainstorming (e.g.,
Carpenter 2004; Hoffman and Zimbelman 2009; Lynch et al. 2009; Trotman et al. 2009;
Carpenter and Reimers 2013; and Gissel 2013), and brainstorming session quality (Brazel et al.
2010). This study also extends prior research that seeks to understand, and ultimately mitigate,
the potential drawbacks of brainstorming in interactive groups (e.g., Chen et al. 2013).
7
A particularly important contribution of this study is that no prior research in this area
uses a field experiment design in which an intervention manipulates the behavior of audit
partners while performing an actual audit engagement in the field. The managers and seniors
participating in our study report a mean of 30 months of experience on the clients involved in
this field experiment. These individuals likely develop rich bodies of client-related knowledge
through this experience that they can attend to throughout fraud brainstorming sessions.
Moreover, institutions in practice create powerful incentives for auditors to take brainstorming
seriously. These factors enable enhanced experimental realism in the current study, relative to
prior fraud brainstorming studies (Swieringa and Weick 1982).
Taken together, our results extend prior research and provide important insights to
practice by showing that a field intervention can improve the approach audit partners take in
leading fraud brainstorming sessions in the field and that the use of such an intervention is
associated with improvements in some important brainstorming processes and outcomes. We
therefore recommend that regulators and practitioners offer explicit guidance on connecting
brainstorming sessions to audit program design, such as through simple, actionable interventions.
This paper proceeds as follows. Section 2 provides the literature review and hypotheses. Section
3 describes the method. Section 4 reports results and Section 5 concludes.
II. LITERATURE REVIEW AND HYPOTHESES
Non-experimental Field Research in Fraud Brainstorming
Two prior studies using field data report evidence of considerable variation in fraud
brainstorming processes and outcomes. Bellovary and Johnstone (2007) interviewed 22 auditors
at all personnel levels at seven audit firms (including all Big 4 firms) to provide descriptive
evidence about how auditors conduct brainstorming sessions. They note variability in the level of
8
contribution to the session among participants, formatting of the session, use of audit firm
guidance, and leadership. Additionally, they note that auditors sometimes view the sessions as
training opportunities for inexperienced team members. Brazel et al. (2010) conduct a field
survey using 179 audit engagements at all four Big 4 firms and one international firm to develop
a measure of brainstorming session quality. They find considerable quality variation in practice.
In addition, their results show that quality brainstorming improves the relationship between fraud
risk factors and fraud risk assessments.
Experimental Research in Fraud Brainstorming
Table 1 provides a summary of experimental research on fraud brainstorming. Each of
these experiments (except our own) uses a hypothetical case, often based on a fraud that occurred
in practice. Panel A categorizes this research based upon differences in experimental designs,
while Panel B summarizes dependent measures. Panel A illustrates the variation in brainstorming
research examining the guidance that participants receive, from no specific mention of
“brainstorming” per se, to brainstorming without explicit guidance, to brainstorming with
auditing-specific interventions intended to facilitate the process.3
INSERT TABLE 1 HERE
Two studies examine associations between general fraud brainstorming guidance and
brainstorming outcomes. Trotman et al. (2009) compare three different forms of group
discussion using 111 experienced auditors: (1) interacting in a team that receives no mention of
the term “brainstorming” and no explicit guidelines, (2) interacting in a team that receives
3 Several prior experimental studies compare fraud brainstorming processes and outcomes using different
communication formats, such as individuals working independently, individuals working in nominal groups, and
individuals working in interacting groups (e.g., Carpenter 2004; Carptenter 2007; Hoffman and Zimbleman 2009;
Lynch et al. 2009; Carpenter et al. 2011; Chen et al. 2013). In the current discussion, we focus on findings related to
guidance provided in fraud brainstorming situations, including auditing-specific interventions intended to facilitate
brainstorming. We do not manipulate interaction format; as discussed subsequently, approximately 22 percent of the
participants in our study reported using a formal interaction format and we control for the use of a formal format in
our hypothesis-testing models.
9
explicit brainstorming guidelines based on Osborn (1957), and (3) interacting in a team that
receives “pre-mortem” instructions modeled after those in Klein (1999).4 The results show that
teams brainstorming with explicit guidelines and pre-mortem teams listed more potential frauds
(quantity) and more “expert-identified frauds” (quality), compared to teams that receive no
explicit brainstorming guidelines.
Additionally, Hoffman and Zimbelman (2009) provide evidence related to whether
brainstorming helps auditors change both the nature and extent of their audit work. In an
experiment with 91 audit managers, the authors manipulate whether or not auditors work alone
with no mention of the word “ brainstorm” or in a three-person team with specific instructions
related to brainstorming. The results show that team brainstorming is associated with effective
modifications to a list of standard audit procedures for accounts receivable. Collectively, these
two studies illustrate that brainstorming guidelines can improve fraud brainstorming outcomes.
In an early paper examining fraud brainstorming, Carpenter (2004) tests interventions
related to audit partner emphasis on efficiency versus effectiveness. In an experiment with 240
auditors (80 managers, 80 seniors, and 80 staff), she finds that when the audit partner emphasizes
efficiency (as opposed to effectiveness) audit teams spend less time in fraud brainstorming (a
process measure). Carpenter and Reimers (2013) extend Carpenter (2004) by examining the
association between audit partner emphasis on professional skepticism and brainstorming
outcomes. Using 80 audit managers, they manipulate partner emphasis on professional
skepticism in general during brainstorming (high or low). The results show fraud risk
assessments are higher when the partner emphasizes professional skepticism. In addition, the
4 Pre-mortem instructions direct the auditors to assume a backward-thinking perspective. Participants receiving the
pre-mortem manipulation in Trotman et al. (2009) are told to imagine the following scenario: “It is months into the
future, the audit has already been completed, and no material fraud was uncovered. However, it has just been
announced in the press that there has been a material financial reporting fraud at the company. The manager and
partner have given you no details of the nature of the fraud” (p. 1121).
10
results show auditor choice of fraud-related audit procedures is only responsive to fraud risk
assessments when fraud is present and the partner emphasizes professional skepticism.
Gissel (2013) examines the effect of psychological safety, as created by the audit partner,
in fraud brainstorming.5 In an experiment with 67 staff and senior auditors, Gissel (2013)
manipulates psychological safety (safe or unsafe) and the presence of a general professional
skepticism intervention using videos of a partner and a manager in a simulated brainstorming
session. The results show auditors are more willing to share private information related to fraud
risks during a brainstorming session when the audit partner’s behavior creates an atmosphere that
is viewed by staff and seniors as psychologically safe. Greater willingness to share private fraud
information then is associated with more accurate fraud risk assessments. In contrast to
expectations, the general professional skepticism intervention does not affect willingness to share
information or fraud risk assessments.
Two other studies use experimental settings to examine fraud brainstorming interventions
that are not specific to audit partner behaviors. Hoffman and Zimbleman (2009) find that an
intervention intended to prompt strategic reasoning is associated with effective modifications to
the standard audit procedures.6 Additionally, Lynch et al. (2009) examine the effect of a content
facilitation intervention, which consists of prompts that address issues emphasized in AU 316. In
an experiment with 188 auditing students, they find a content facilitation intervention is
associated with the identification of a larger number of relevant fraud risk factors.
The Intervention
We extend prior studies that employ interventions intended to aid the efficacy of fraud
brainstorming by employing an intervention that attempts to influence the approach audit
5 Gissel defines psychological safety as “a sense of being able to show and employ self without fear of negative
consequences to self-image, status, or career.” 6 Hoffman and Zimbleman (2009) do not find a significant interaction between their two manipulations.
11
partners take in leading an actual fraud brainstorming session for an audit engagement in the
field. In order for the intervention to be effective, the following conditions must be satisfied: (1)
the instructions in the intervention must relate to audit partner behaviors where there is room for
improvement in practice, and (2) members of the audit engagement team must respond to the
behaviors of the audit partner in the experimental condition.
Practitioner-oriented articles suggest the auditing profession has a collective interest in
improving fraud detection (e.g., Landis et al. 2008; Wood and Pickerd 2011). The PCAOB has
criticized audit teams with respect to how they conduct brainstorming sessions, particularly with
respect to a perceived lack of professional skepticism (PCAOB 2007), and prior research
provides some descriptive evidence consistent with the PCAOB’s views (e.g., Bellovary and
Johnstone 2007; Brazel et al. 2010). Collectively, these factors suggest room for improvement in
the way auditors conduct fraud brainstorming; if the elements in the intervention relate to such
improvement opportunities, then the first condition will be satisfied.
Regarding the second condition, prior research suggests managers and seniors attend and
respond to differences in partner behaviors during fraud brainstorming (Carpenter 2004;
Carpenter and Reimers 2013; Gissel 2013). In prior studies, manipulations related to audit
partner behaviors during brainstorming are delivered directly to subordinates via experimental
instruments. In contrast, we manipulate audit partner behavior in the field during brainstorming
sessions on real engagements and measure subordinates’ perceptions of audit partner behavior
subsequent to the sessions. It seems reasonable to expect that these subordinates will respond to
partner behavior. Still, it is an empirical question as to whether the intervention will successfully
manipulate the behaviors that audit partners exhibit in brainstorming sessions, as proxied by
engagement team members perceptions of the topics partners discuss and issues partners
12
emphasize. We therefore include a series of questions in the survey to determine whether
managers and seniors perceive the audit partner behaviors referenced in the intervention
differently, depending on experimental condition. We use the managers’ and seniors’ responses
to these questions to perform a manipulation check and to provide evidence on the behaviors that
audit partners exhibit in fraud brainstorming sessions as a matter of professional practice.
Fraud Brainstorming Processes
AU 316, Consideration of Fraud in a Financial Statement Audit (formerly SAS No. 99),
discusses the importance of maintaining professional skepticism throughout the audit (AICPA
2002b). Notably, this standard discusses professional skepticism in general throughout the audit
and professional skepticism with respect to accounts with a higher level of fraud risk. Given the
importance of professional skepticism in fraud detection and following Carpenter (2004), we
expect that increases in professional skepticism, both in general and with respect to accounts
with a higher level of fraud risk, represent high quality processes in fraud brainstorming sessions.
Specifically, we argue that increases in professional skepticism during brainstorming sessions
facilitate improved discussion as sessions progress.
We also investigate process-oriented dependent measures related to the extent of certain
discussions during the fraud brainstorming session. In order for auditors to respond to fraud risk
effectively and in a way that complies with the applicable audit standards, they must both
identify relevant fraud risks and develop effective audit responses to those risks. Prior research
on brainstorming suggests that quantity drives quality in terms of idea generation (e.g., Trotman
et al. 2009; Osborn 1957). Therefore, we argue that more extensive discussion about how
management might perpetrate fraud and more extensive discussion about potential audit
responses to fraud represent high quality brainstorming processes.
13
A recent working paper provides a rich contextual understanding for how interventions
can facilitate improvements in fraud brainstorming processes and outcomes. Chen et al. (2013)
focus on three potential drawbacks of interactions in brainstorming: production blocking,
evaluation apprehension, and social loafing. They argue that the electronic brainstorming setting
minimizes production blocking (due to a lack of interruptions) and evaluation apprehension (due
to anonymity), thereby enabling a more precise examination of the effect of social loafing on
brainstorming outcomes. In an experiment with 111 audit seniors and managers, they find
evidence consistent with social loafing of seniors driving the differences between nominal
groups and interacting teams in fraud hypothesis generation. Notably for the current study, these
findings suggest audit partners can improve brainstorming outcomes by mitigating potential
drawbacks in brainstorming, such as social loafing, through their leadership during the session.
The limited prior research on associations between fraud brainstorming processes and
audit-specific brainstorming interventions provides mixed evidence regarding the effectiveness
of these interventions. On the one hand, Carpenter (2004) finds the predicted association
between the time spent brainstorming and an intervention emphasizing either effectiveness or
efficiency. Additionally, Gissel (2013) finds the predicted association between participants’
willingness to share private information and a psychological safety manipulation, but not a
professional skepticism-related intervention.
In field settings, partners can exert significant influence over audit activities. Notably for
the current study, partners can focus and re-direct discussions during fraud brainstorming
sessions. Partners can also take measures to address issues of evaluation apprehension and social
loafing by seniors (e.g., Kerr and Tindale 2004; Chen et al. 2013). Live interactions in the field
enable a dynamic implementation of the intervention; such an experimental manipulation would
14
be prohibitively difficult or costly in a laboratory setting. These factors contribute to the
enhanced experimental realism in the current study (e.g., Swierenga and Weick 1982) and
increase the likelihood of associations between the intervention and fraud brainstorming
processes. Following this discussion, we posit the following hypothesis:
Hypothesis 1. The intervention is positively associated with indicators of the quality of
fraud brainstorming session processes.
Fraud Brainstorming Outcomes
We consider brainstorming outcomes in terms of both fraud risk factors and fraud risk
responses, consistent with prior research (e.g., Carpenter 2004; Carpenter 2007; Lynch et al.
2009; Trotman et al. 2009; Carpenter et al. 2011; Carpenter and Reimers 2013; and Chen et al.
2013). We also separately identify the number of fraud risk responses intended to incorporate an
element of unpredictability in the audit as a fraud brainstorming outcome. Further, since all
engagements in our sample are continuing clients, the quantity of fraud risk factors and fraud risk
responses may be similar to that from the prior year. We therefore expect that the number of new
fraud risk factors and new fraud risk responses represent brainstorming outcomes that will be
positively associated with the intervention. This approach is consistent with Hoffman and
Zimbleman (2009), who analyze audit program modifications as a brainstorming outcome.
The PCAOB has expressed concerns about “mechanical” implementation of AU 316
(PCAOB 2007). If audit teams implement this guidance mechanically, then the audit plan will
lack appropriate tailoring. Moreover, if audit teams do not critically evaluate fraud audit
programs each year, then fraud audit programs will become inappropriate as conditions change.
We therefore expect that the elimination of fraud risk responses that had been used in prior year
15
audits (i.e., tailoring the fraud audit program to the current year audit) represents a brainstorming
outcome that will be positively associated with the intervention.
Prior research that investigates associations between interventions and fraud
brainstorming outcomes generally finds different inferences related to fraud risk factors, as
compared to fraud risk responses (see, e.g., Hammersley 2011). Several prior studies find
associations between interventions and either improved fraud risk factor identification or
improved fraud risk assessments, which in turn reflect improved risk factor identification (e.g.,
Lynch et al. 2009; Carpenter and Reimers 2013; Gissel 2013). However, a growing number of
studies suggest auditors have difficulty linking evaluations of fraud risk with appropriate fraud
risk responses (e.g., Asare and Wright 2004; Mock and Turner 2005; Hammersley et al. 2011).
Hammersley (2011, 118), in particular, notes “effective changes to planned procedures in
response to perceptions of increased fraud risk is a joint test of whether [1] the fraud risk factor
identified is useful for this purpose, [2] auditors recognize that the procedures should be
modified, [3] auditors know which procedures should be modified, and [4] auditors know how to
modify the procedure appropriately.” If the intervention does not promote all four conditions
necessary for audit teams to make effective modifications to planned procedures, then the
intervention will not be associated with the brainstorming outcome variables. Moreover, one
dimension of the Brazel et al. (2010) measure of brainstorming quality that is particularly
relevant to the relationship between fraud risk assessments and fraud risk responses is the extent
of discussion about audit responses to fraud risk. If we find no association between the
intervention and this process variable, then it is possible that we will correspondingly find no
association between the intervention and the fraud risk response-related outcome variables.
16
On the other hand, there are two important reasons to expect associations between the
intervention and fraud risk response-related outcome variables in the current field experiment.
First, we include instruction points and illustrative examples in the intervention that are relevant
to each of the elements identified by Hammersley (2011). If the intervention helps engagement
teams satisfy these necessary conditions, then the likelihood of associations between the
intervention and fraud risk response-related outcomes will increase. Additionally, Brazel et al.
(2010) find that high quality brainstorming, measured using process variables, improves the
relationships between fraud risk factors and fraud risk assessments and between fraud risk
assessments and fraud risk responses. Therefore, if the intervention improves fraud
brainstorming processes, as predicted in H1, then the intervention will also improve fraud
brainstorming outcomes (e.g., Gissel 2013). Following this, we posit the following hypothesis:
Hypothesis 2. The intervention is positively associated with indicators of the quality of
fraud brainstorming session outcomes.
In addition to the preceding quantitative measures of fraud brainstorming outcomes, we
analyze qualitative characteristics of fraud risk factors and fraud risk responses. Specifically, we
examine whether the intervention is associated with the identification of higher percentages of
risk factors related to revenue recognition and management override of controls, respectively (as
these two areas receive particularly direct emphasis in AU 316). We perform these analyses to
provide insight on whether the intervention perhaps promotes incremental focus on client-
specific fraud-related risks. We further examine whether the intervention is associated with the
percentages of fraud risk responses for a given risk factor that relate to the nature, timing, or
extent, respectively, of planned procedures. We perform these analyses to provide insight into
whether the intervention is associated with what is being done to respond to fraud risk, how
17
much is being done, and/or when these responses occur. We make no predictions related to the
association between these qualitative variables and treatment effects.
III. METHOD
Participants and Research Design
We conducted the study in conjunction with, and immediately following, the fraud brainstorming
sessions of a sample of 77 audit engagements from July 2013 through January 2014.7 Three audit
firms (two Big 4 firms and one international firm) participated in the study, with relative
contribution levels of 77 percent, 16 percent, and seven percent, respectively.8 Each engagement
team in the sample was randomly assigned to either the treatment condition (N = 37 audit
partners) or the control condition (N = 40 audit partners).9 Each partner in the treatment
condition, TREATMENT = 1, received a memo informing him/her that his/her audit engagement
will be involved in a fraud brainstorming research study; this memo also included the
experimental intervention. Each partner in the control condition, TREATMENT = 0, received a
memo informing him/her that his/her engagement would be involved in a fraud brainstorming
research study; necessarily, this memo did not contain the experimental intervention.10
Figure 1
7 As previously noted, all engagements in the sample relate to continuing clients; none of the engagements in the
sample are new to the respective audit firm. 8 The distribution of Treatment (Control) condition observations across audit firms is as follows: 55 (59), 12 (12),
and 4 (6) observations relate to Firm A, Firm B, and Firm C, respectively. The authors obtained institutional review
(i.e., human-subjects) approval for the study, and all participants were given the option to not complete the study. 9 We visited each participating audit firm office to meet with the senior audit partner that served as the “local office
champion” for this research study (e.g., business unit managing partner or lead area technical partner) and the
individual that would serve as the local office contact person. During the meeting, we delivered the experimental
materials and trained both individuals on the process of administering the experiment. We emphasized the
importance that the respective “champion” randomly select and randomly assign the engagements to the
experimental conditions. The individuals assured us that they would adhere to the administration, random selection,
and random assignment processes diligently. We have no reason to believe that there were any systematic biases in
the selection or assignment of engagements. However, due to client confidentiality constraints, we were unable to
oversee the actual selection or assignment processes. We articulate this issue further in the conclusion. 10
We printed all memos on audit firm letterhead. Memos were addressed from a prominent senior partner in the
participant’s specific business unit or region (e.g., business unit managing partner or lead area technical partner).
18
Panel A displays the wording of the memo provided to partners in both the treatment and control
conditions, and Figure 1 Panel B depicts the research design and project logistics.
INSERT FIGURE 1 HERE
After the fraud brainstorming session, one audit manager and one audit senior on each
engagement were notified about participating in the study. Therefore, the managers and seniors
were not aware that partners received experimental instructions prior to the brainstorming
session, nor were they aware during brainstorming that the session would be a part of a research
study.11
These individuals then completed a survey requesting information about the client, the
brainstorming process, and various associated outcomes. A total of 75 managers and 73 seniors
completed the surveys.12
The authors coded the quantitative data from the surveys. Two second-
year PhD students performed the coding of the qualitative data from the surveys.13
Manipulation Check
Table 2 contains comparisons, between experimental conditions, of managers’ and
seniors’ perceptions of indicators of the audit partner fraud brainstorming behaviors. We use
these comparisons as a manipulation check. TRAINING_OPP measures the extent to which the
audit partner emphasized fraud brainstorming as a training/professional development opportunity
on a scale from 1 (low emphasis) to 10 (high emphasis). PTR_EXPERIENCES is a dichotomous
variable equal to one if the audit partner discussed his/her prior experiences with fraud during
brainstorming. EFFECTIVE_EFFICIENT measures the extent to which the audit partner
discussed the importance of effective and efficient brainstorming on a scale from 1 (no
11
From an institutional review perspective, the experimental instructions provided to the partners are conceptually
identical to other firm-sponsored partner-only training programs of which managers and seniors are also unaware. 12
The cover letter on the survey instruments states “You may refer to workpapers and your notes when completing
the survey.” We did not measure the extent to which participants consulted these materials; however, several
respondents included notes from the session and/or sections of audit programs with their survey responses. 13
Each of these individuals previously held a Senior Manager position at a Big 4 Firm, with eight years and 12
years, respectively, of public accounting experience. Inter-rater reliability among the coders was 89%.
19
discussion) to 10 (significant discussion). CALIBRATED_RESPONSE is a dichotomous variable
equal to one if the audit partner addressed the issues of effectiveness and efficiency to promote
an appropriately calibrated response to fraud risk; equals zero otherwise. PS_SPECIFIC is a
dichotomous variable equal to one if the audit partner discussed the importance of professional
skepticism with respect to specific accounts on the engagement with a higher level of fraud risk;
equals zero otherwise. PS_GENERAL is a dichotomous variable equal to one if the audit partner
discussed the importance of professional skepticism in general throughout the audit; equals zero
otherwise. See Appendix A for variable definitions.
INSERT TABLE 2 HERE
The results show that managers’ and seniors’ perceptions do, indeed, differ by
experimental condition. Partners in the treatment condition (as compared to the control
condition) more often discussed prior experiences with fraud during brainstorming
(PTR_EXPERIENCES, p = 0.01), which is a specific indicator of treating the session as a
training opportunity. Partners in the treatment condition more often addressed issues of
effectiveness and efficiency to promote an appropriately calibrated response to fraud risk
(CALIBRATED_RESPONSE, p = 0.01). Additionally, partners in the treatment condition more
often discussed the importance of professional skepticism with respect to specific accounts on
the engagement with a higher level of fraud risk (PS_SPECIFIC, p = 0.02). The pattern of these
univariate comparisons illustrates that the intervention given to partners in the treatment
condition positively shifted managers’ and seniors perceptions of topics discussed and issues
emphasized by the audit partner during brainstorming, thereby providing evidence that the
intervention successfully manipulated these partner behaviors.
20
In contrast, there are no significant univariate differences between the other three
indicators of partner behavior, TRAINING_OPP, EFFECTIVE_EFFICIENT, or PS_GENERAL.
The treatment (control) condition means are as follows: TRAINING_OPP, means = 6.57 (6.44),
EFFECTIVE_EFFICIENT means = 7.15 (6.94), and PS_GENERAL, means = 100 percent (96
percent). The relatively high levels (above the midpoints and nearly 100 percent) of the
TRAINING_OPP, EFFECTIVE_EFFICIENT, and PS_GENERAL measures, together with the
lack of differences in these measures between conditions, implies that audit partners address
these relatively more general discussion topics and issues of emphasis routinely in practice.
There are likely upper-bound limits on the emphasis that partners can reasonably place on these
considerations simultaneously in practice; therefore, it is not entirely surprising that we find no
differences in these measures across conditions. Moreover, our experimental intervention
successfully manipulated the behavior of audit partners on more specific dimensions where
univariate tests indicate larger improvement opportunities in practice.
These results help to understand mixed findings in prior research regarding the efficacy
of interventions that remind auditors about professional skepticism in general (i.e., Carpenter and
Reimers 2013; Gissel 2013). These studies use an intervention that is consistent with the wording
in AU 316, that is, to maintain “the proper state of mind throughout the audit” (paragraph 14),
which emphasizes professional skepticism in general. Our results indicate that an intervention
emphasizing professional skepticism with respect to specific accounts on the engagement with a
higher level of fraud risk will be more effective than an intervention requesting partners to
emphasize professional skepticism in general, perhaps reflecting the fact that auditors already
receive significant general communications about professional skepticism in practice.
21
Measures for Process and Outcome Effects
Dependent Variables
We test the association between the intervention and both process and outcome measures
of fraud brainstorming. Process variables include PS_CHG_GENERAL, PS_CHG_SPECIFIC,
DISCUSSION_MGT, and DISCUSSION_RESP, and SESSION_LENGTH. PS_CHG_GENERAL
and PS_CHG_SPECIFIC measure changes in the self-assessed levels of professional skepticism
on the engagement during fraud brainstorming.14
We calculate these variables as the difference
between self-assessed levels of professional skepticism after brainstorming and self-assessed
levels of professional skepticism before brainstorming, where participants self-assess their own
levels of professional skepticism on the sample engagement relative to an “average” or “normal”
client on a scale from 1 (much lower than normal) to 10 (much higher than normal).15
PS_CHG_GENERAL measures the change in the individual’s self-assessed level of professional
skepticism on the engagement in general. PS_CHG_SPECIFIC measures the change in the
individual’s self-assessed level of professional skepticism with respect to specific accounts on
the engagement with a higher level of fraud risk. DISCUSSION_MGT is the extent of discussion
during brainstorming about how management might perpetrate fraud, on a scale from 1 (very
low) to 10 (very high). DISCUSSION_RESP is the extent of discussion during brainstorming
about audit responses to fraud risk, on a scale from 1 (very low) to 10 (very high).
SESSION_LENGTH is the number of minutes spent brainstorming. H1 predicts these process
variables will be positively associated with TREATMENT.
14
We do not measure trait skepticism (e.g., Hurtt 2010) because we do not expect the intervention to influence traits
of managers and seniors. 15
Participants self-assess each of these levels of professional skepticism using the survey instrument they receive
after the fraud brainstorming session is complete.
22
Fraud risk factor identification outcome variables include RISKS_NUMBER,
RISKS_NEW, %REV_REC, and %OVERRIDE. RISKS_NUMBER is the number of fraud risks
identified during fraud brainstorming. RISKS_NEW is the number of fraud risks identified during
brainstorming that are new for the current year audit. We measure both RISKS_NUMBER and
RISKS_NEW using quantitative counts of risk factor data. H2 predicts that RISKS_NUMBER and
RISKS_NEW will be positively associated with TREATMENT.
The other two fraud risk factor identification outcome variables, %REV_REC and
%OVERRIDE, measure qualitative characteristics of risk factors. %REV_REC is the percentage
of fraud risk factors identified that relate to revenue recognition. %OVERRIDE is the percentage
of fraud risk factors identified that relate to management override of controls. AU 316 states
“auditors should ordinarily presume” that improper revenue recognition is a fraud risk (AICPA
2002b, 41). Additionally, AU 316 directs auditors to “consider the possibility that management
override of controls could occur” in the context of fraud risk (AICPA 2002b, 42). We analyze
%REV_REC and %OVERRIDE to determine whether TREATMENT is associated with the types
of fraud risk factors identified, and we make no directional predictions related to these
associations. Table 3 provides examples of fraud risk factors coded as relating to revenue
recognition and management override of controls.
INSERT TABLE 3 HERE
Fraud risk response outcome variables include PROC_NUMBER, PROC_NEW,
PROC_UNPRED, TAILOR, %RELATE_NATURE, %RELATE_EXTENT, and
%RELATE_TIMING. PROC_NUMBER is the number of procedures planned to respond to fraud
risks in the current year audit. PROC_NEW is the number of planned procedures that are new for
the current year audit. PROC_UNPRED is the number of planned procedures intended to
23
incorporate an element of unpredictability in the audit. TAILOR is a dichotomous variable equal
to one if the engagement team eliminated fraud risk responses that had been used in prior year
audits (i.e., tailoring the audit plan to the current year audit); equals zero otherwise. H2 predicts
that PROC_NUMBER, PROC_NEW, PROC_UNPRED, and TAILOR will be positively
associated with TREATMENT.
The other three fraud risk response outcome variables, %RELATE_NATURE,
%RELATE_EXTENT, and %RELATE_TIMING, measure qualitative characteristics of fraud risk
responses. Each of these variables is calculated as the percentage of responses to a fraud risk
factor that relate to the nature, extent, and timing, respectively, of planned procedures. We
analyze these variables to determine whether TREATMENT is associated with the types of
planned responses to fraud risk, and we make no directional predictions related to these
associations. Table 3 provides examples of these fraud risk response variables.
Control Variables
We include in hypothesis-testing models control variables used in prior research (Brazel
et al. 2010). CLIENT_SIZE is based on revenue and is coded as follows: 1 = < $100 million, 2 =
$100 million - $500 million, 3 = > $500 million - $1 billion, 4 = > $1 billion - $5 billion, 5 = >
$5 billion. PUBLIC is a dichotomous variable equal to one if the client is publicly traded; equals
zero otherwise. INDUSTRY_FS is a dichotomous variable equal to one if the client is in the
financial services industry; equals zero otherwise. INDUSTRY_GV/NP is a dichotomous variable
equal to one if the client is in the government or not-for-profit industry; equals zero otherwise.
INDUSTRY_MFG is a dichotomous variable equal to one if the client is in the manufacturing
24
industry; equals zero otherwise.16
INHERENT_RISK is the level of inherent risk associated with
the overall engagement on a scale from 1 (low) to 10 (high). FRAUD_RISK is the level of fraud
risk associated with the overall engagement on a scale from 1 (low) to 10 (high). As in Brazel et
al. (2010), we control for COMPLEXITY, which is measured as the number of auditors assigned
to the engagement (coded as follows: 1 = 0-5 auditors, 2 = 6-10 auditors, 3 = 11-15 auditors, 4 =
16-20 auditors, and 5 = > 20 auditors) divided by CLIENT_SIZE. TEAM_EXPERTISE is the
engagement team’s level of expertise on the client on a scale from 1 (very low) to 10 (very high).
PTR_CLIENT_EXPC is the level of the experience of the engagement partner on the respective
engagement, coded as 1 = first year on engagement, 2 = relatively new to engagement, and 3 =
moderate or significant amount of experience on engagement.17
MGR_CLIENT_EXPC is the
number of months the lead engagement manager has served on the engagement.
SR_CLIENT_EXPC is the number of months the lead engagement senior has served on the
engagement. EXPERIENCE_FRR is the number of engagements the respondent served on in
which fraudulent financial reporting was identified, coded as 1 = 0, 2 = 1-2, 3 = >2. Experience
and expertise are elements of fraud knowledge, which Hammersley (2011) notes is a critical
determinant of fraud hypothesis generation. MANAGER1 is a dichotomous variable equal to one
if the survey respondent is a manager; zero if the survey respondent is a senior.
FORMAL_FORMAT is a dichotomous variable equal to one if the nature of the format of the
discussion for fraud brainstorming is either round robin or nominal group; equals zero
16
INDUSTRY_MISC is a dichotomous variable equal to one if the client's industry is Retail, Energy, High
Tech/Communications, Healthcare/Pharmaceuticals, or Other; zero otherwise. We exclude this variable from the
analyses to avoid singularities in the models. 17
This variable implicitly controls for partner turnover. We calculate it as a categorical variable because the
manager and senior participants likely do not know the exact number of months that the audit partner served on the
engagement.
25
otherwise.18
We make non-directional predictions for all control variables, except for the
following variables in the models with process-related dependent variables. In those models, we
expect that CLIENT_SIZE, PUBLIC, INHERENT_RISK, FRAUD_RISK, COMPLEXITY, and
EXPERIENCE_FFR will be positively associated with the dependent variables.
IV. RESULTS
Descriptive Results
Table 4, Panel A presents descriptive statistics on the dependent variables. With respect to
process-related dependent variables, changes in professional skepticism in general and with
respect to specific accounts, PS_CHG_GENERAL and PS_CHG_SPECIFIC, are higher (p = 0.02
and p = 0.02, respectively) in the treatment condition (mean = 0.73 and mean = 0.87,
respectively) than in the control condition (mean = 0.45 and mean = 0.51, respectively). The
mean of DISCUSSION_MGT is 7.12 on a scale from 1 (very low) to 10 (very high), with no
significant univariate difference between the two conditions. The mean DISCUSSION_RESP is
7.03 on a scale from 1 (very low) to 10 (very high), with a marginally lower extent of discussion
about responses to fraud risk in the treatment condition (p = 0.09). The mean
SESSION_LENGTH is about 35 minutes, and is longer (by about 10 minutes) in the treatment
condition (p = 0.04).
INSERT TABLE 4, PANEL A HERE
With respect to fraud risk factor identification outcome dependent variables,
RISKS_NUMBER is 3.43 with no differences between the conditions. The number of new fraud
risks identified during brainstorming, RISKS_NEW, is relatively low with an overall sample
mean of 0.52. However, RISKS_NEW is higher (p = 0.01) in the treatment condition (mean =
18
We collected data on other control variables used in Brazel et al. (2010), but do not include control variables in
our hypothesis-testing models that are consistently insignificant.
26
0.72) than in the control condition (mean = 0.33). The mean percentage of risk factors related to
revenue recognition, %REV_REC, is 30 percent, with no significant differences between the
conditions. The mean percentage of risk factors related to management override of controls,
%OVERRIDE, is 19 percent, with no significant differences between the conditions.
With respect to fraud risk response outcome dependent variables, PROC_NUMBER, is
5.55, with no significant differences between the conditions. The number of new planned
procedures, PROC_NEW, is low with a mean of 0.42. However, PROC_NEW is higher (p =
0.05) in the treatment condition (mean = 0.58) compared to the control condition (mean = 0.29).
The mean number of procedures planned to incorporate an element of unpredictability in the
audit, PROC_UNPRED, is 1.72, with no significant differences between the conditions. Finally,
engagement teams seem reluctant to eliminate prior-year procedures, TAILOR, with a mean of 20
percent. However, engagement teams are marginally more likely to eliminate procedures in the
control condition than in the treatment condition (25 percent and 14 percent, respectively; p =
0.08). The mean percentage of fraud risk responses relating to the nature of planned procedures,
%RELATE_NATURE, is 86 percent and the mean percentage of fraud risk responses relating to
the extent of planned procedures, %RELATE_EXTENT, is 13 percent; there are no significant
differences between the conditions for either variable. The mean percentage of fraud risk
responses relating to the timing of planned procedures, %RELATE_TIMING, is two percent and
is lower in the treatment condition (p = 0.02) than in the control condition.
Table 4, Panel B presents descriptive statistics on the control variables. CLIENT_SIZE is
in the range of $500 million to $1 billion, and clients in the control condition are marginally
larger (p = 0.05). Sixty-two percent of the sample is PUBLIC, and this percentage is significantly
higher in the control condition (p = 0.02). In terms of industry membership, the largest
27
representations are in financial services (29 percent) and manufacturing (25 percent). We also
control for membership in the government/not-for-profit industry (seven percent). Means of
INHERENT_RISK and FRAUD_RISK are 4.71 and 4.13, respectively on scales from 1 (low) to
10 (high), and do not differ by experimental condition. Mean COMPLEXITY is 0.85, which is
similar to Brazel et al. (2010), and also does not differ by experimental condition.
INSERT TABLE 4, PANEL B HERE
The mean TEAM_EXPERTISE is 7.75, which indicates a perception that sample
engagement teams possess a relatively high level of expertise. The mean PTR_CLIENT_EXPC is
2.63, which indicates roughly moderate levels of experience on the client.19
Neither of these two
variables differs by experimental condition. The means of MGR_CLIENT_EXPC and
SR_CLIENT_EXPC are approximately 37 months and 23 months, respectively.
SR_CLIENT_EXPC is lower in the treatment condition (p = 0.04). The number of engagements
on which the respondent experienced fraudulent financial reporting in the past is quite low, with
a mean of 1.24 (indicating zero to two engagements), with no differences between experimental
conditions. The mean of MANAGER1 is 0.51, indicating that 51% and 49% of the surveys were
completed by managers and seniors, respectively. In terms of audit firm representation, 77
percent of the sample is from Firm A, 16 percent is from Firm B, and 7 percent is from Firm C
(untabulated). With respect to brainstorming session format, open discussion is the predominant
method (88 percent of the time), with round robin (17 percent) and nominal group (5 percent)
being used much less frequently.20, 21
Format type does not differ by experimental condition.
19
Ten respondents indicated the partner was new to the engagement for the current year audit. Inferences are
unchanged when we exclude these observations from the analyses. 20 Format type does not differ significantly by audit firm. 21
At 2.31 or less, the VIFs from the hypothesis-testing model estimated using linear regression are all well below
the 10.00 threshold recommended by Belsley et al. (1980). Thus, collinearity does not appear to be a concern in our
hypothesis-testing analyses.
28
Hypothesis Tests
Process Variables
Table 5 focuses on dependent variables relating to brainstorming processes. 22
There is a
positive association between TREATMENT and both PS_CHG_GENERAL (t = 1.95, p = 0.03)
and PS_CHG_SPECIFIC (t = 1.74, p = 0.04), consistent with expectations in H1. There is also a
positive and marginally significant association between TREATMENT and DISCUSSION_MGT
(t = 1.31, p = 0.10), but not DISCUSSION_RESP (t = -1.23, p = 0.22). The results further show a
positive association between TREATMENT and SESSION_LENGTH (t = 1.84, p = 0.04). This
implies that the amount of audit firm resources applied to implement the field intervention is not
excessive (with a mean increment of about 10 minutes) and provides context for potential budget
management concerns in the field. When considered with the positive outcome effects (described
subsequently), the results for SESSION_LENGTH suggest that fraud brainstorming quality
outcome effects achieved via the intervention come at a relatively low cost. Taken together, these
results imply that the intervention yields larger increases in professional skepticism, both in
general and with respect to specific accounts with a higher risk of fraud, more discussion about
how management might perpetrate fraud, and longer fraud brainstorming sessions, providing
support for H1.
INSERT TABLE 5 HERE
CLIENT_SIZE is positively associated with PS_CHG_SPECIFIC (t = 1.74, p = 0.04),
DISCUSSION_MGT (t = 2.41, p < 0.01), and SESSION_LENGTH (t = 1.97, p = 0.03). PUBLIC
is positively associated with DISCUSSION_MGT (t = 1.68, p = 0.05), but is unexpectedly
22
We cluster standard errors by engagement in all models to control for within-engagement correlation. The field
experiment was conducted in the same manner at all three of the firms participating in the study and we do not
expect within-firm correlation. However, we clustered standard errors by audit firm in all models and find that doing
so does not change the results.
29
negatively associated with PS_CHG_GENERAL or PS_CHG_SPECIFIC (t = -1.77, p = 0.06; t =
-2.52, p = 0.01, respectively), which may indicate that professional skepticism is already quite
high for these engagements. FRAUD_RISK is positively associated with PS_CHG_SPECIFIC (t
= 1.31, p = 0.09), DISCUSSION_MGT (t = 1.93, p = 0.03), and DISCUSSION_RESP (t = 2.95, p
< 0.01). TEAM_EXPERTISE is positively associated with both DISCUSSION_MGT (t = 2.31, p
= 0.02) and DISCUSSION_RESP (t = 2.10, p = 0.04).
We also find interesting associations between brainstorming processes and partner,
manager, and senior experience on the client. PTR_CLIENT_EXPC is negatively and marginally
significantly associated with PS_CHG_GENERAL (t = -1.77, p = 0.08) and DISCUSSION_MGT
(t = -1.96, p = 0.05). These results imply an inertia effect whereby longer audit partner tenure on
an engagement yields “stickiness” in processes associated with brainstorming sessions.
Interestingly, the results imply the opposite with respect to brainstorming processes and the
number of months the lead engagement manager has served on the engagement. Specifically,
MGR_CLIENT_EXPC is positively and marginally significantly associated with
DISCUSSION_MGT (t = 1.84, p = 0.07) and DISCUSSION_RESP (t = 1.93, p = 0.06),
suggesting that continuity at the manager level enhances brainstorming processes.
SR_CLIENT_EXPC is marginally negatively associated with PS_CHG_SPECIFIC (t = -1.67, p =
07) and is negatively associated with DISCUSSION_MGT (t = -2.53, p = 0.03). Finally,
MANAGER1 is negatively associated with both PS_CHG_GENERAL and PS_CHG_SPECIFIC
(t = -3.94, p < 0.01; t = -2.40, p = -0.02, respectively), and is positively associated with
DISCUSSION_MGT (t = 2.89, p < 0.01), suggesting that seniors and managers experience
brainstorming sessions somewhat differently. Finally, a FORMAL_FORMAT (nominal group or
round robin) is positively associated with SESSION_LENGTH (t = 2.21, p = 0.03).
30
Fraud Risk Factor Outcome Variables
Table 6 focuses on fraud risk factor identification outcome dependent variables
associated with brainstorming. With respect to quantitative measures, there is a positive
association between TREATMENT and RISKS_NUMBER (t = 1.75, p = 0.04), and a positive
association between TREATMENT and RISKS_NEW (t = 2.10, p = 0.02), both consistent with
H2. There are no significant associations between TREATMENT and measures of qualitative
characteristics of risk factors (%REV_REC and %OVERRIDE). This implies that the fraud risk
profile of clients in the field does not differ by experimental condition. Moreover, this lack of
associations suggests that the intervention did not promote incremental consideration of
commonly identified fraud risk factors that specifically require attention under AU 316.
INSERT TABLE 6 HERE
Regarding control variables, there is a positive association between INHERENT_RISK
and both RISKS_NUMBER (t = 2.33, p = 0.02) and RISKS_NEW (t = 2.21, p = 0.03).
CLIENT_SIZE is negatively associated with RISKS_NEW (t = -2.15, p = 0.04); since all
engagements in the sample were continuing engagements with the respective audit firms, this
result is consistent with audit firms deploying more resources in prior years to identify fraud
risks at larger clients. Similarly, PTR_CLIENT_EXPC is negatively associated with RISKS_NEW
(t = -2.05, p = 0.04), suggesting engagement teams accumulate relevant fraud risk factors over
time. PTR_CLIENT_EXPC is positively associated with %OVERRIDE (t = 2.93, p < 0.01),
potentially alleviating concerns that audit partners might grow increasingly comfortable with
management as partner tenure increases. Lastly, MANAGER1 is positively and marginally
significantly associated with both %REV_REC (t = 1.81, p = 0.08) and %OVERRIDE (t = 1.72, p
31
= 0.09), suggesting that managers were more cognizant of specific provisions in AU 316 when
completing the surveys than seniors.
Fraud Risk Response Outcome Variables
Table 7 focuses on fraud risk response outcome dependent variables associated with
brainstorming. We find no significant associations between TREATMENT and any of the
measures related to audit procedures: number of procedures (PROC_NUMBER), new procedures
(PROC_NEW), or procedures intended to incorporate an element of unpredictability
(PROC_UNPRED).23
The lack of treatment effects for H2 related to all three procedures
variables is unexpected, but consistent with prior literature that shows auditors often have
difficulty linking evaluations of fraud risk with appropriate fraud risk responses (e.g., Mock and
Turner 2005; Hammersley 2011; Hammersley et al. 2011), or with substantive test modifications
in other audit tasks (Mauldin and Wolfe 2014). Moreover, in contrast with expectations in H2,
the results show a negative association between TREATMENT and TAILOR (z = -3.15, p < 0.01),
which indicates that the intervention is associated with reduced tailoring such that there is a
lower likelihood of eliminating fraud risk responses that had been used in the prior year.24
INSERT TABLE 7 HERE
Conversely, our analyses of qualitative fraud risk response outcome dependent variables
provide interesting insights related to H2. There is a positive and marginally significant
association between TREATMENT and %RELATE_NATURE (t = 1.86, p = 0.07), and negative
associations between TREATMENT and both %RELATE_EXTENT (t = -2.03, p = 0.05) and
23
It is interesting to note that univariate tests show PROC_NEW is greater (t = 1.65, p = 0.05) for engagements in
the treatment condition (mean = 0.58) than for engagements in the control condition (mean = 0.29). 24
We perform a median split on the FRAUD_RISK variable and re-estimate Model 13 (TAILOR) using subsamples
of observations with high fraud risk (greater than or equal to median) and low fraud risk (less than or equal to
median). The association between TREATMENT and TAILOR is negative for both the high fraud risk subsample (t =
-2.50, p = 0.01) and the low fraud risk subsample (t = -3.20, p < 0.01). Therefore, the reluctance to eliminate fraud
risk responses from the prior year audit in the treatment condition appears to occur regardless of the level of fraud
risk.
32
%RELATE_TIMING (t = -2.04, p = 0.05). These results reveal that, compared to engagement
teams in the control condition, engagement teams in the treatment condition respond to fraud risk
through relatively more modifications to the nature of planned procedures (e.g., strengthening
procedures performed, developing and performing new procedures) and relatively fewer
modifications to the extent (e.g., increased sample size, use of lower scope) and timing (e.g.,
testing at final vs. interim, testing at interim in addition to final) of planned procedures. This is
consistent with fraud risk responses of engagement teams in the treatment condition focusing
more on what is being done to address fraud risks and focusing less on how much is being done
and when it is being done. If modifications to the nature of planned procedures address fraud risk
factors more effectively and/or efficiently than modifications to the extent or timing of planned
procedures, then the intervention is associated with favorable fraud brainstorming outcomes.
Interesting findings regarding control variables include a negative association between
FRAUD_RISK and %RELATE_TIMING (t = -2.27, p = 0.03), indicating that auditors are less
likely to respond to fraud risk by modifying the timing of planned procedures when they evaluate
heightened fraud risk. COMPLEXITY is negatively associated with both PROC_NEW (t = -1.69,
p = 0.09) and PROC_UNPRED (t = -2.40, p = 0.02), and is positively associated with both
TAILOR (t = 1.67, p = 0.09) and %RELATE_EXTENT (t = 2.15, p = 0.04). These results
highlight the difficulty of responding to fraud risks on complex audit engagements in that
auditors struggle to identify new procedures to respond to fraud risk, struggle to incorporate
unpredictability into these audits, but that auditors respond to this complexity by carefully
tailoring and planning larger sample sizes. TEAM_EXPERTISE is positively and marginally
significantly associated with PROC_UNPRED (t = 1.94, p = 0.06), suggesting client-specific
team expertise helps auditors to incorporate elements of unpredictability into the audit plan.
33
PTR_CLIENT_EXPC, is negatively and marginally significantly associated with TAILOR (t = -
1.87, p = 0.06), suggesting further potentially negative consequences of audit partner tenure with
respect to fraud brainstorming outcomes (i.e., “stickiness”) and complementing the findings
related to PRT_CLIENT_EXPC in the models analyzing brainstorming processes (discussed
previously). MANAGER1 is positively and marginally significantly associated with
%RELATE_NATURE (t = 1.71, p = 0.09) and negatively and marginally significantly associated
with %RELATE_EXTENT (t = -1.67, p < 0.10), suggesting that, compared to seniors, managers
focused more on what was being done to respond to fraud risk and less on how much was being
done to respond to fraud risk when completing the surveys. These results complement those in
Hammersley et al. (2011), which reveal that audit seniors often respond to fraud risk through
indiscriminate sample size increases rather than through effective audit program modifications.
Finally, the results show a negative and marginally significant association between
FORMAL_FORMAT and TAILOR (t = -1.67, p < 0.10), implying that such a format yields less
willingness to tailor via eliminating previously used procedures.
Supplemental Analysis
We analyze the association between TAILOR and TREATMENT separately for public and
private clients to examine the possibility that the unwillingness to tailor relates to concerns over
justifying the elimination of fraud risk responses during PCAOB inspections. TREATMENT is
negatively associated with TAILOR in Model 13 when we include only public clients in the
model (t = -2.18, two-tailed p = 0.03). Additionally, a two-sample t-test indicates a significant
univariate difference between the mean values of TAILOR for public clients in the treatment and
control conditions (t = -2.11, p = 0.04). Due to a lack of variation between TAILOR and several
independent variables in the sample of private clients, we lose a number of observations and are
34
unable to fit Model 13 to the sample of private clients. However, a two-sample t-test indicates no
significant univariate difference between the mean values of TAILOR for private clients in the
treatment versus control conditions (n = 71, t = .39, one-tailed p = .35), suggesting that public
client engagements drive the unexpected negative association between TREATMENT and
TAILOR. This is consistent with engagement teams being concerned about PCAOB inspections,
which encourages them to avoid eliminating procedures performed in prior years.
V. CONCLUSION
We conduct a field experiment to test whether an intervention can influence the approach
audit partners take in leading fraud brainstorming sessions and whether this is associated with
brainstorming processes and outcomes in natural hierarchical audit teams. The results suggest
partners tend to address relatively more general discussion topics and issues of emphasis relevant
to brainstorming routinely in practice (emphasizing the session as a training opportunity in
general, discussing effective/efficient brainstorming in general, and discussing professional
skepticism in general.) The intervention successfully improved the approach of partners with
respect to more specific discussion topics and issues of emphasis relevant to brainstorming
where there appears to be room for improvement in practice (discussion of prior experiences
with fraud during brainstorming, discussion of issues of effectiveness/efficiency to promote an
appropriately calibrated response to fraud risk, and discussion of the importance of professional
skepticism with respect to specific accounts on the engagement with a higher level of fraud risk).
Our results extend prior research in interesting ways. For example, while Lynch et al.
(2009) find that a content facilitation intervention improves the identification of relevant fraud
risk factors using student subjects and Carpenter and Reimers (2013) find a positive association
between the degree of partner emphasis on professional skepticism and fraud risk assessments
35
using managers, Gissel (2013) finds no association between manager emphasis on professional
skepticism and the sharing of private information using staff and seniors. Our results reconcile
and extend these prior findings by showing that the effectiveness of an intervention in fraud
brainstorming depends on whether the intervention is specific (as opposed to general) and is
related to audit partner leadership behavior where there is room for improvement in practice.
Consistent with our expectations related to fraud brainstorming processes, the
intervention is associated with increases in professional skepticism during brainstorming, both in
general and with respect to specific accounts with a higher level of fraud risk, more discussion
about how management might commit fraud, and longer brainstorming sessions. Our process-
related results extend Carpenter’s (2004) findings related to professional skepticism and are
consistent with her finding that when a partner expresses a preference for effectiveness versus
efficiency the audit team spends more time in brainstorming.
Consistent with our expectations related to fraud brainstorming outcomes, the
intervention is associated with the identification of a higher number of fraud risks and a higher
number of new fraud risks. However, we find no association between the intervention and the
percentages of fraud risk factors identified that relate to revenue recognition and relate to
management override of controls, suggesting that engagement teams in both experimental
conditions encounter similar conditions in the field. In contrast with our expectations, we find no
evidence of associations between the intervention and the generation of audit responses to fraud
risks and a negative association between the intervention and the willingness of engagement
teams to tailor the audit plan to the current year audit by eliminating fraud risk factors identified
in prior year audits. This latter result is attributable to the public clients in the sample. We do
find associations between the intervention and qualitative characteristics of responses to fraud
36
risk that are consistent with engagement teams in the treatment condition focusing more on what
to do to respond to fraud risk, and less on how much to do or when to do it.
Taken together, our results extend the fraud brainstorming literature that examines the
efficacy of explicit instructions. Therefore, our study also has important implications for
practitioners and regulators. We designed the intervention with the help of senior leadership at
the audit firms that participated in this study; these individuals were interested in improving
auditor judgments in brainstorming that require professional skepticism. These instruction
guidelines are simple, actionable items that audit firms can incorporate into methodology and
training programs to enhance brainstorming effectiveness and address recent PCAOB concerns
about “mechanical implementation” of AU 316 (PCAOB 2007, 4). To this end, our study
illustrates that an intervention can be an effective means to re-focus audit teams on the
importance of fraud detection and deterrence and promote diligent compliance with AU 316.
Limitations and Future Research
One limitation of this study is that it seems audit partners already address relatively more
general discussion topics and issues of emphasis relevant to fraud brainstorming routinely in
practice (i.e., emphasizing the session as a training opportunity in general, discussing
effective/efficient brainstorming in general, and discussing professional skepticism in general).
This implies high quality professional behavior for which no intervention is necessary to
improve brainstorming processes and outcomes. Future research might explore other potential
actionable interventions in this task setting, as well as investigating interventions that may be
applicable to engagement team members other than the audit partner. Future research can also
examine the effect of multi-period interventions. Our study tests the effectiveness of an
37
intervention in a single brainstorming session and it is possible that the effectiveness of the
intervention will change in subsequent brainstorming sessions.
A second limitation relates to the fact that due to client confidentiality constraints, we
were unable to oversee the actual selection and assignment of engagements in the study. As
previously noted, we took steps to emphasize the importance of random selection and random
assignment during our meetings with the senior audit partners that served as the “local office
champions” of the research study. We have no reason to believe that any systematic bias exists in
the selection of engagements into the sample or the assignment of engagements to experimental
conditions. It is possible that the senior audit partners might have chosen to include only “good”
clients in the sample; however, the descriptive statistics reveal variation in measures of inherent
risk (min/max of 1-10) and fraud risk (min/max of 1-9) in the overall sample, which suggests
that this is not the case. It is difficult to predict whether and how the senior audit partners might
have been motivated to systematically bias assignment of engagements to experimental
conditions. However, we find the same levels of variation in measures of inherent risk (min/max
of 1-10) and fraud risk (min/max of 1-9) in both the treatment and control conditions, which
alleviates concerns about potential biases in the assignment “good” clients to a particular
experimental condition.
Finally, because of institutional review requirements, the partners all knew they were
participating in research relating to their fraud brainstorming session. This knowledge might
somehow have affected their behavior, e.g., by being more diligent than normal. Random
assignment to experimental conditions should alleviate this concern. Further, it is important to
note that the managers and seniors did not know about the experiment while they participated in
the brainstorming session, and it is their perceptions upon which we base our measures.
38
REFERENCES
American Institute of Certified Public Accountants (AICPA). 2002a. Consideration of Fraud in
a Financial Statement Audit. Statements on Auditing Standards No. 99. New York, NY:
AICPA.
American Institute of Certified Public Accountants (AICPA). 2002b. Consideration of Fraud in
a Financial Statement Audit. AU Section 316. New York, NY: AICPA.
Asare, S. K., and A. M. Wright. 2004. The effectiveness of alternative risk assessment and audit
program planning tools in a fraud setting. Contemporary Accounting Research 21 (2):
325-352.
Bedard, J.C., and K. M. Johnstone. 2010. Audit partner tenure and audit planning and pricing.
Auditing: A Journal of Practice & Theory 29 (2): 45-70.
Bellovary, J. L., and K. M. Johnstone. 2007. Descriptive evidence from audit practice on SAS
No. 99 brainstorming activities. Current Issues in Auditing 1 (1): 1–11.
Belsley, D. A., E. Kuh, and R. E. Welsch. 1980. Regression Diagnostics Identifying Influential
Data and Sources of Collinearity. John Wiley & Sons: New York.
Brazel, J. F., T. D. Carpenter, and J. G. Jenkins. 2010. Auditors’ use of brainstorming in
consideration of fraud: Reports from the field. The Accounting Review 85 (4): 1273–
1301.
Carmichael, D. R. 2004. The PCAOB and the social responsibility of the independent auditor.
Accounting Horizons 18 (2): 127–133.
Carpenter, T. D. 2004. Partner Influence, Team Brainstorming, and Fraud Risk Assessment:
Some Implications of SAS No. 99. Unpublished dissertation, Florida State University.
Carpenter, T. D. 2007. Audit team brainstorming, fraud risk identification, and fraud risk
assessment: Implications of SAS No. 99. The Accounting Review 82 (5): 1119–1140.
Carpenter, T. D., J. L. Reimers, and P. Z. Fretzwell. 2011. Internal Auditors’ Fraud Judgments:
The Benefits of Brainstorming in Croups. Auditing: A Journal of Practice & Theory 30
(3): 211–224.
Carpenter, T. D., and J. L. Reimers. 2013. Professional skepticism: The effects of a partner’s
influence and the level of fraud indicators on auditors’ fraud judgments and actions.
Behavioral Research in Accounting 25 (2): 45–69.
Chen, X. C., K. T. Trotman, and F. H. Zhou. 2013. Electronic fraud brainstorming in
hierarchical audit teams: Does interaction help or hurt? Working paper, University of
Illinois at Urbana-Champaign and University of New South Wales.
Committee of Sponsoring Organizations of the Treadway Commission (COSO). 2011. Internal
Control – Integrated Framework. Available at http://www.coso.org/ic-
integratedframework-summary.htm.
Gissel, J. L. 2011. Private Information Sharing during SAS 99 Brainstorming: Effects of
Psychological Safety and Professional Skepticism. Working Paper, Marquette University.
Hammersley, J. S. 2011. A review and model of auditor judgments in fraud-related planning
tasks. Auditing: A Journal of Practice & Theory 30 (4): 101–128.
Hammersley, J. S., K. M. Johnstone, and K. Kadous. 2011. How do audit seniors respond to
heightened fraud risk? Auditing: A Journal of Practice & Theory 30 (3): 81–101.
Hoffman, V. B., and M. F. Zimbelman. 2009. Do strategic reasoning and brainstorming help
auditors change their standard audit procedures in response to fraud risk? The Accounting
Review 84 (3): 811–837.
Hurtt, R. K. 2010. Development of a Scale to Measure Professional Skepticism. Auditing: A
Journal of Practice & Theory 29 (1): 149–171.
39
Kerr, N. L., and R. S. Tindale. 2004. Group performance and decision making. Annual Review of
Psychology 55 (1): 623–655.
Klein, G. 1999. Sources of Power: How People Make Decisions. Cambridge, MA: MIT Press.
Landis, M., S. I. Jerris, and M. Braswell. 2008. Better brainstorming. Journal of Accountancy
206 (4): 70–73.
Lynch, A. L., U. S. Murthy, and T. J. Engle. 2009. Fraud brainstorming using computer-
mediated communication: The effects of brainstorming technique and facilitation. The
Accounting Review 84 (4): 1209–1232.
Mauldin, E. G., and C. J. Wolfe. 2014. How do auditors address control deficiencies that bias
accounting estimates? Contemporary Accounting Research (forthcoming).
Mock, T. J., and J. L. Turner. 2005. Auditor identification of fraud risk factors and their impact
on audit programs. International Journal of Auditing 9 (1): 59–77.
Osborn, A. F. 1957. Applied Imagination: Principles and Procedures of Creative Thinking. New
York: Charles Scribner’s Sons.
Public Company Accounting Oversight Board (PCAOB). 2007. Observations on Auditors’
Implementation of PCAOB Standards Relating to Auditors’ Responsibilities with Respect
to Fraud. Release No. 2001-001, January 22. Washington, DC: PCAOB. Available at
http://pcaobus.org/Inspections/Documents/2007_01-22_Release_2007-001.pdf.
Swieringa, R. J., and K. E. Weick. 1982. An assessment of laboratory experiments in accounting.
Journal of Accounting Research 20 (Supplement): 56-101.
Trotman, K. T., R. Simnett, and A. Khalifa. 2009. Impact of the type of audit team discussions
on auditors’ generation of material frauds. Contemporary Accounting Research 26 (4):
1115–1142.
Wood, D. A., and J. Pickerd. 2011. Problems to avoid when brainstorming fraud risks. The CPA
Journal (April): 64–65.
40
Figure 1
Wording of Experimental Intervention and Research Design/Project Logistics
Panel A. Wording of Partner Memo
[TREATMENT AND CONTROL CONDITIONS] Our firm is collaborating with Professor X at the
University of X on a research study in which you are being asked to participate. The purpose of the
study is to assist the firm in developing its audit methodology and training programs related to fraud
brainstorming. Your participation relates to your role as the engagement partner on the audit client
noted above. Your involvement will be limited to simply completing the annual fraud brainstorming
activities and notifying the contact person in your office when those activities are complete. Your
participation in this study is completely voluntary, and you may opt out of this study by informing the
contact person in your local office that you do not wish to participate. Your participation will be
confidential.
[TREATMENT AND CONTROL CONDITIONS] There is no requirement that the brainstorming be
conducted as a stand-alone meeting; it can be conducted during part of another meeting (e.g., the annual
planning meeting). For research validity purposes, please do not discuss this research with your
colleagues.
[TREATMENT CONDITION ONLY] We ask that you attend to the following five instructions during
fraud brainstorming for this engagement. We provide examples to illustrate possible ways for you to
implement these instructions; however, these examples are not an exhaustive list of implementation
ideas.
1. Emphasize fraud brainstorming as a training/professional development opportunity for the audit
team members present.
Discuss any relevant personal experience on engagements involving fraud.
Discuss some fraud/forensic topics that are relevant to this engagement (e.g., the fraud
triangle with respect to specific accounts with a higher level of fraud risk).
Actively mentor both the audit manager and in-charge auditor in terms of how to most
effectively identify and appropriately respond to fraud risks.
Be cognizant of the fact that your leadership during the session sets the tone for the
engagement team members as they work to appropriately assess and respond to fraud
risk during planning and conduct of the engagement.
2. Discuss the importance of effective and efficient fraud brainstorming.
Discuss the downside risks to the firm of a failure to identify fraud risks early in the
audit.
Discuss features of SAS 99 relevant to this particular engagement, and emphasize the
resources the firm has developed to facilitate fraud brainstorming. Discuss the
importance of documenting compliance with SAS 99 for litigation and review purposes.
Discuss how devoting resources to inappropriate fraud risk procedures can hinder audit
efficiency.
41
3. Discuss the importance of professional skepticism targeted at specific accounts with a
potentially higher level of fraud risk. Emphasize both effectiveness and efficiency to promote an
appropriately calibrated response to fraud risk.
Discuss specific accounts that the engagement team has identified as having a
potentially higher level of fraud risk.
Direct the engagement team to consider whether accounts that have historically been
identified as having a higher level of fraud risk are still appropriately identified as such.
Specifically consider whether a higher assessed level of fraud risk on such accounts is
warranted for the current year audit.
Challenge the engagement team to consider new procedures and/or improvements to
existing procedures.
Discuss relevant and appropriate procedures that can incorporate an element of
unpredictability into the audit.
Compare the client’s financial condition and performance to that of other companies in
its industry using pertinent industry averages (e.g. ratios, trends). Discuss how the
client’s results are either inconsistent or consistent with these averages.
4. Discuss the importance of professional skepticism in general throughout the audit.
Discuss the fact that while certain accounts have a potentially higher level of fraud risk,
there could be fraud in an area not traditionally associated with higher fraud risk.
Encourage the audit manager and in-charge auditor to be alert for anything unusual or
unexpected during the conduct of the audit.
Discuss the importance of considering the risk of management override of internal
controls.
Consider asking the following types of questions to encourage professional skepticism:
(1) What potential frauds may have occurred? (2) How could management conceal the
potential frauds? (3) How could the audit plan be modified to detect concealed fraud?
5. Do not communicate to the engagement team, or any of your other colleagues, that you have
been given these instructions. Doing so will compromise the validity of the research.
[TREATMENT AND CONTROL CONDITION] Federal law requires that we notify you that there are
no risks associated with your participation in this research study, nor are there specific benefits to you
personally. Your participation will help the firm develop its audit methodology and training programs
related to fraud brainstorming. Please notify the contact person in your office when you have completed
the fraud brainstorming activities. At that point, the audit manager and in-charge auditor will complete
a survey about the fraud brainstorming activities/discussion. Your notification of the contact person in
your local office that the fraud brainstorming activities are complete indicates your consent to
participate in this study.
42
Panel B. Research Design/Project Logistics
Local office
contacts
Audit Partners
in Treatment
Group
Audit Partners
in Control
Group
Audit
Manager and
Audit Senior
participants
Completed
Survey
Instruments
Review of surveys
at Audit Firm Level
to Ensure No
Mention of Client-
Identifying
Information
Local office
contacts
Completed
and Audit
Firm-
Reviewed
Survey
Instruments
Memo without
Experimental
Intervention
Memo with
Experimental
Intervention
Experimental
MaterialsResearchers
Notification
that fraud
brainstorming
session is
complete
Uniform
Survey
Instrument Local
Office
Contacts
Completed
Survey
Instruments
43
Appendix A
Variable Definitions
Variable Name Description
Dependent Variables:
PS_CHG_GENERAL
The change in reported professional skepticism on the engagement in general during fraud
brainstorming, calculated using before and after measures on a scale from 1 = much lower
than normal to 10 = much higher than normal.
PS_CHG_SPECIFIC
The change in reported professional skepticism with respect to specific accounts with a
higher level of fraud risk during fraud brainstorming, calculated using before and after
measures on a scale from 1 = much lower than normal to 10 = much higher than normal.
DISCUSSION_MGT The extent of discussion during fraud brainstorming about how management might
perpetrate fraud on a scale from 1 = very low to 10 = very high.
DISCUSSION_RESP The extent of discussion during fraud brainstorming about audit responses to fraud risk on a
scale from 1 = very low to 10 = very high.
SESSION_LENGTH The number of minutes spent in fraud brainstorming.
RISKS_NUMBER Number of fraud risks identified by the engagement team in the current year audit.
RISKS_NEW Number of fraud risks identified by the engagement team for the current year audit, but not
identified in prior year audits.
%REV_REC Percentage of fraud risk factors identified that relate to revenue recognition.
%OVERRIDE Percentage of fraud risk factors identified that relate to management override of controls.
PROC_NUMBER Number of procedures planned by the engagement team to respond to fraud risks in the
current year audit.
PROC_NEW Number of procedures planned by the engagement team to respond to fraud risks in the
current year audit, but not performed in prior year audits.
PROC_UNPRED Number of procedures planned by the engagement team intended incorporate an element of
unpredictability in the current year audit.
TAILOR
A dichotomous variable equal to one if the engagement team eliminated fraud risk
responses that had been used in prior year audits as a result of fraud brainstorming; zero
otherwise.
%RELATE_NATURE Percentage of responses to a fraud risk factor that relate to the nature of planned
procedures.
%RELATE_EXTENT Percentage of responses to a fraud risk factor that relate to the extent of planned
procedures.
%RELATE_TIMING Percentage of responses to a fraud risk factor that relate to the timing of planned
procedures.
Treatment Variable:
TREATMENT A dichotomous variable equal to one if the partner on the engagement received the memo
with the experimental intervention; zero otherwise.
44
Independent Variables:
CLIENT_SIZE
Size of the client based on total revenue, coded as follows: 1 = < $100 million, 2 = $100
million - $500 million, 3 = > $500 million - $1 billion, 4 = > $1 billion - $5 billion, 5 = >
$5 billion.
PUBLIC A dichotomous variable equal to one if the client is publicly traded; zero if the client is
privately owned.
INDUSTRY [FS, GV/NP,
MFG]
Dichotomous variables equal to one if the client's industry is [Financial Services,
Government/Not-for-Profit, or Manufacturing]; zero otherwise.
INDUSTRY_MISC A dichotomous variable equal to one if the client's industry is Retail, Energy, High
Tech/Communications, Healthcare/Pharmaceuticals, or Other; zero otherwise.
INHERENT_RISK Overall engagement-level inherent risk assessment on a scale from 1 = low risk to 10 =
high risk.
FRAUD_RISK Overall engagement-level fraud risk assessment on a scale from 1 = low risk to 10 = high
risk.
COMPLEXITY TEAM SIZE/CLIENT SIZE
TEAM_EXPERTISE The engagement team's level of expertise on the client on a scale from 1 = very low to 10 =
very high.
PTR_CLIENT_EXPC
The experience level of the engagement partner on the respective engagement, coded as 1 =
first year on engagement, 2 = relatively new to engagement, and 3 = moderate or
significant amount of experience on engagement.
MGR_CLIENT_EXPC Number of months the lead engagement manager has served on the engagement.
SR_CLIENT_EXPC Number of months the lead engagement senior has served on the engagement.
EXPERIENCE_FFR The number of engagements the respondent served on in which fraudulent financial
reporting was identified, coded as follows: 1 = 0, 2 = 1-2, 3 = > 2.
MANAGER1 A dichotomous variable equal to one if the survey respondent is a manager; zero if the
survey respondent is a senior.
FORMAL_FORMAT A dichotomous variable equal to one if the nature of the format of the discussion for fraud
brainstorming is round robin or nominal group; zero otherwise
Manipulation Check Variables
TRAINING_OPP
TRAINING_OPP measures the extent to which the audit partner emphasized fraud
brainstorming as a training/professional development opportunity on a scale from 1 (low
emphasis) to 10 (high emphasis).
PTR_EXPERIENCES PTR_EXPERIENCES is a dichotomous variable equal to one if the audit partner discussed
his/her prior experiences with fraud during brainstorming.
EFFECTIVE_
EFFICIENT
EFFECTIVE_EFFICIENT measures the extent to which the audit partner discussed the
importance of effective and efficient brainstorming on a scale from 1 (no discussion) to 10
(significant discussion).
CALIBRATED_RESP
CALIBRATED_RESPONSE is a dichotomous variable equal to one if the audit partner
addressed the issues of effectiveness and efficiency to promote an appropriately calibrated
response to fraud risk; equals zero otherwise.
PS_GENERAL
PS_SPECIFIC is a dichotomous variable equal to one if the audit partner discussed the
importance of professional skepticism with respect to specific accounts on the engagement
with a higher level of fraud risk; equals zero otherwise.
PS_SPECIFIC
PS_GENERAL is a dichotomous variable equal to one if the audit partner discussed the
importance of professional skepticism in general throughout the audit; equals zero
otherwise.
45
TABLE 1
Summary of Previous Brainstorming Research
Panel A: Experimental Designs and Incremental Contribution
Experimental Manipulations in a
Laboratory Setting
Experimental Manipulations in a
Field Setting
Guidance
No mention of the
term
“brainstorming”
Trotman et al. (2009)
Hoffman and Zimbelman (2009)
Brainstorming
without explicit
guidelines
Carpenter (2007)
Hoffman and Zimbelman (2009)
Lynch et al. (2009)
Carpenter et al. (2011)
Chen et al. (2013)
Present Study
Auditing-Specific Laboratory Interventions Intended to Facilitate Fraud Brainstorming:
Brainstorming with
explicit guidelines
Trotman et al. (2009)
Present Study
Content facilitation
Lynch et al. (2009)
Psychological
safety
Gissel (2013)
Partner/manager
emphasis
Effectiveness/
efficiency
Professional
skepticism
Carpenter (2004)
Carpenter and Reimers (2013)
Gissel (2013)
Present study
Present Study
Strategic reasoning Hoffman and Zimbelman (2009)
46
TABLE 1
Summary of Previous Brainstorming Research
Panel B: Dependent Measures
Research Study Process-related
Dependent
Variables
Outcome-related Dependent Variables
Carpenter (2004) Time spent
by audit
teams
Accuracy of fraud risk assessments
Quantity of ideas generated
Professional skepticism
Carpenter (2007) Quantity of ideas generated
Quality of ideas generated
Fraud risk assessments
Carpenter et al.
(2011)
Fraud risk assessments
Quantity and quality of fraud risks
Hoffman and
Zimbelman
(2009)
Modification of nature and extent of planned audit procedures
Budgeted hours
Trotman et al.
(2009)
Quantity of potential misstatements due to fraud
Quality of potential misstatements due to fraud (expert-identified
frauds and expert-identified misstatements)
Proportion of rare potential frauds identified to total potential
frauds identified
Lynch et al.
(2009)
Fraud risk assessments
Quantity of relevant fraud risks
Gissel (2013)
Fraud risk assessments
Willingness to share private information about the fraud
Chen et al.
(2013)
Fraud risk assessments
Quantity of fraud risk factors
Quantity and quality of fraud hypotheses
Auditors’ mental representations for hypothesis testing
Carpenter and
Reimers (2013)
Fraud risk assessments
Quantity of relevant fraud risk factors identified
Quantity of relevant audit procedures
47
TABLE 2
Comparison of Audit Partner Behavior by Experimental Condition
Overall
[n=148]
Treatment
[n=71]
Control
[n=77]
Statb
p-valueb Item
a
n Mean
Std.
Dev. Med
n Mean
Std.
Dev. Med
n Mean
Std.
Dev. Med
TRAINING_OPP
148 6.50 2.47 7.00
71 6.57 2.50 6.00
77 6.44 2.45 7.00
0.32
0.38
PTR_EXPERIENCES
147 0.50 0.50 0.00
70 0.60 0.49 1.00
77 0.40 0.49 0.00
4.95
0.01
EFFECTIVE_EFFICIENT 148 7.04 2.39 7.00
71 7.15 2.42 8.00
77 6.94 2.37 7.00
0.56
0.29
CALIBRATED_RESP
146 0.87 0.34 1.00
70 0.94 0.23 1.00
76 0.80 0.40 1.00
5.15
0.01
PS_GENERAL
147 0.98 0.14 1.00
70 1.00 0.00 1.00
77 0.96 0.19 1.00
1.18
0.14
PS_SPECIFIC
147 0.96 0.20 1.00
70 1.00 0.00 1.00
77 0.92 0.27 1.00
3.87
0.02
a See definitions in Appendix A.
b Continuous variable statistics are t-statistics from means tests. Indicator variable statistics are Chi-Squared statistics with Yates' continuity corrections. All p-values
are from one-tailed tests.
48
TABLE 3
Examples of Qualitative Data Coding
Fraud Risk Factors Fraud Risk Responses
Related to Revenue
Recognition
Related to Management
Override of Controls
Related to Nature of
Planned Procedures
Related to Extent of
Planned Procedures
Related to Timing of
Planned Procedures
“Revenue recognition -
cutoff, deferred revenue &
deferred costs” (Manager,
Treatment group)
“Revenue Recognition - in
relation to cost estimation”
(Manager, Control group)
“Misappropriation of
America Recovery
Reinvestment Act funds”
(Senior, Control group)
“Revenue - complex
accounting, including
multiple-element sales and
percentage of completion
valuation and timing of rev.
recognition (cut-off)”
(Senior, Control group)
“Changes in specific
customer contracts may cause
errors in financial reporting
and/or give management of
significant entities an
opportunity to manipulate
revenue recognition policy to
meet financial forecasts”
(Senior, Control group)
“Dominance at executive
level by one particular
person could lead to
manipulation in order to
meet set targets.” (Manager,
Treatment group)
“Unsubstantiated JE's
related to revenue and/or
expense” (Manager, Control
group)
“Small operations,
Controller sees a lot,
management override of
controls” (Senior, Treatment
group)
“Management override of
controls via post-close or
top side journal entries”
(Senior, Treatment group)
“Manipulation of earnings
via inappropriate manual,
top-side entries” (Senior,
Control group)
Management override of
controls through manual
journal entries (multiple
mentions of several
variations)
“Detailed contract review on
significant [revenue]
contracts” (Manager,
Treatment group)
“Analysis of [inventory mark-
down reserve account]
balance throughout the year
and prior year” (Manager,
Treatment group)
“Discuss w/ management
level of bulk purchases and
inventory monitoring”
(Manager, Treatment group)
“A/R confirmations”
(Manager, Control group)
“[Test the following control:]
contracts not activated until
dually signed and no material
written changes” (Senior,
Treatment group)
“Review for fraud incentives
arising from covenant regs,
reg environment, earnings
estimates, etc.” (Senior,
Treatment group)
“Revenue cut-off testwork -
detail testing” (Senior,
Control group)
“Scoped in sales and A/R
for a new location”
(Manager, Treatment
group)
“Review of smaller-mid
size bank accounts for
evidence of improper
review/cut-offs” (Manager,
Control group)
“[firm] is testing all sales
transactions above $200k.
Additionally a sample is
tested for the remaining
population.” (Senior,
Treatment group)
“Increase extent of testing
of derivative financial
instruments & contract
review (i.e., classification
of contract as derivative or
accrual acct.)” (Senior,
Control group)
“Vouch 100% of revenue
received to cash payments”
(Senior, Control group)
“In AP, will select items
specifically from new
facility to test cutoff”
(Senior, Control group)
“Test [long-term agreements]
quarterly” (Manager,
Treatment group)
“Test closer to or at YE for
certain procedures”
(Manager, control group)
“Review significant
agreements quarterly.”
(Senior, Treatment group)
“Changing procedure timing
or scoped components from
prior year” (Senior, Control
group) (also relates to extent)
“Perform receivable
confirmations at an interim
date.” (Senior, Control
group)
“We will also perform a
detailed cut-off test at interim
(5 before and 5 after).
Typically, we would [only]
test 5 before and 5 after year
end for cut-off.” (Senior,
Control group)
49
TABLE 4
Descriptive Statistics
Dependent Variables and Control Variables
Panel A: Dependent Variables
Overall
[n=148]
Treatment
[n=71]
Control
[n=77]
Process Variablesa n Mean
Std.
Dev. Med Min Max
n Mean
Std.
Dev. Med
n Mean
Std.
Dev. Med
Statb
p-valueb
PS_CHG_GENERAL 146 0.58 0.87 0.00 -2.00 3.00
70 0.73 0.91 0.00
76 0.45 0.82 0.00
1.99
0.02
PS_CHG_SPECIFIC 148 0.68 1.00 0.00 -2.00 4.00
71 0.87 1.11 1.00
77 0.51 0.87 0.00
2.16
0.02
DISCUSSION_MGT 148 7.12 1.64 7.00 2.00 10.00
71 7.17 1.71 7.00
77 7.07 1.58 7.00
0.36
0.36
DISCUSSION_RESP 148 7.03 2.02 7.00 1.00 10.00
71 6.73 2.15 7.00
77 7.30 1.86 8.00
-1.08
0.09
SESSION_LENGTH 148 34.83 29.75 30.00 5.00 200.00
71 39.44 36.01 30.00
77 30.58 21.90 25.00
1.79
0.04
Fraud Risk Factor Outcome Variablesa
RISKS_NUMBER 145 3.43 1.98 3.00 1.00 11.00
69 3.61 1.82 3.00
76 3.28 2.11 3.00
1.02
0.16
RISKS_NEW 145 0.52 0.94 0.00 0.00 4.00
69 0.72 1.11 0.00
76 0.33 0.70 0.00
2.54
0.01
%REV_REC 147 0.30 0.29 0.33 0.00 1.00 70 0.32 0.34 0.27 77 0.29 0.23 0.33 0.79 0.43
%OVERRIDE 147 0.19 0.22 0.14 0.00 1.00 70 0.18 0.22 0.14 77 0.20 0.22 0.09 -0.62 0.54
Fraud Risk Response Outcome Variablesa
PROC_NUMBER 139 5.55 4.07 4.00 1.00 18.00
66 5.50 3.94 4.00
73 5.59 4.21 4.00
-0.13
0.45
PROC_NEW 139 0.42 1.01 0.00 0.00 7.00
66 0.58 1.25 0.00
73 0.29 0.70 0.00
1.65
0.05
PROC_UNPRED 139 1.72 2.71 1.00 0.00 25.00
66 1.47 3.29 1.00
73 1.95 2.05 2.00
-1.01
0.16
TAILOR 148 0.20 0.40 0.00 0.00 1.00 71 0.14 0.35 0.00 77 0.25 0.43 0.00 2.00 0.08
%RELATE_NATURE 404 0.86 0.32 1.00 0.00 1.00 207 0.88 0.30 1.00 197 0.85 0.34 1.00 1.15 0.25
%RELATE_EXTENT 404 0.13 0.31 0.00 0.00 1.00 207 0.11 0.28 0.00 197 0.14 0.33 0.00 -1.15 0.25
%RELATE_TIMING 404 0.02 0.11 0.00 0.00 1.00 207 0.00 0.03 0.00 197 0.03 0.16 0.00 -2.37 0.02 a See variable definitions in Appendix A.
b Continuous variable statistics are t-statistics from means tests. Indicator variable statistics are Chi-Squared statistics with Yates' continuity corrections. All p-values are two-tailed, except
those with directional predictions.
50
TABLE 4
Descriptive Statistics
Dependent Variables and Control Variables
Panel B: Control Variables
Overall
[n=148]
Treatment
[n=71]
Control
[n=77]
Control Variablesa
n Mean
Std.
Dev. Med Min Max
n Mean
Std.
Dev. Med
n Mean
Std.
Dev. Med
Statb
p-valueb
CLIENT_SIZE 146 3.02 1.30 3.00 1.00 5.00
69 2.80 1.33 3.00
77 3.22 1.25 4.00
-1.97
0.05
PUBLIC 148 0.62 0.49 1.00 0.00 1.00
71 0.52 0.50 1.00
77 0.71 0.45 1.00
5.07
0.02
INDUSTRY_FS 148 0.29 0.46 0.00 0.00 1.00
71 0.21 0.41 0.00
77 0.36 0.48 0.00
3.45
0.06
INDUSTRY_GV/NP 148 0.07 0.25 0.00 0.00 1.00
71 0.00 0.00 0.00
77 0.13 0.34 0.00
7.94
0.00
INDUSTRY_MFG 148 0.25 0.43 0.00 0.00 1.00
71 0.30 0.46 0.00
77 0.21 0.41 0.00
1.09
0.30
INDUSTRY_MISC 148 0.39 0.49 0.00 0.00 1.00 71 0.47 0.50 0.00 77 0.31 0.47 0.00 -1.92 0.06
INHERENT_RISK 148 4.71 2.00 5.00 1.00 10.00
71 4.55 2.07 4.00
77 4.86 1.94 5.00
-0.94
0.35
FRAUD_RISK 148 4.13 1.74 4.00 1.00 9.00
71 4.17 1.93 4.00
77 4.09 1.56 4.00
0.27
0.79
COMPLEXITY 146 0.85 0.38 0.75 0.33 2.00
69 0.86 0.41 1.00
77 0.84 0.34 0.75
0.37
0.71
TEAM_EXPERTISE 148 7.75 1.61 8.00 0.00 10.00
71 7.67 1.81 8.00
77 7.83 1.42 8.00
-0.60
0.55
PTR_CLIENT_EXPC 148 2.63 0.61 3.00 1.00 3.00
71 2.61 0.64 3.00
77 2.65 0.58 3.00
-0.43
0.67
MGR_CLIENT_EXPC 148 36.95 28.04 32.00 0.00 109.00 71 33.23 31.68 24.00 77 40.38 23.90 38.00 1.56 0.12
SR_CLIENT_EXPC 148 23.12 13.94 23.00 0.00 51.00 71 20.61 14.19 18.00 77 25.44 13.39 24.00 2.13 0.04
EXPERIENCE_FFR 147 1.24 0.47 1.00 1.00 3.00
71 1.24 0.49 1.00
76 1.24 0.46 1.00
0.03
0.97
MANAGER1 148 0.51 0.50 1.00 0.00 1.00 71 0.49 0.50 0.00 77 0.52 0.50 1.00 0.32 0.75
FORMAL_FORMAT 148 0.22 0.42 0.00 0.00 1.00
71 0.20 0.40 0.00
77 0.25 0.43 0.00
0.28
0.60 a See variable definitions in Appendix A.
b Continuous variable statistics are t-statistics from means tests. Indicator variable statistics are Chi-Squared statistics with Yates' continuity corrections. All p-values are two-tailed, except
those with directional predictions.
51
TABLE 5
Regression Results: Process Variables
Dependent Variables: PS_CHG_GENERAL, PS_CHG_SPECIFIC, DISCUSSION_MGT, DISCUSSION_RESP,
SESSION_LENGTH b
(1)
PS_CHG
_GENERAL
(2)
PS_CHG
_SPECIFIC
(3)
DISCUSSION
_MGT
(4)
DISCUSSION
_RESP
(5)
SESSION_
LENGTH
Variablea Pred. Coefficient Coefficient Coefficient Coefficient Coefficient
Sign (t-statistic)c (t-statistic)
c (t-statistic)
c (t-statistic)
c (t-statistic)
c
TREATMENT + 0.33** 0.31** 0.36* -0.43 10.46**
(1.95) (1.74) (1.31) (-1.23) (1.84)
Control Variables
CLIENT_SIZE
+ 0.05 0.14** 0.29*** 0.03 7.11**
(0.68) (1.74) (2.41) (0.17) (1.97)
PUBLIC + -0.33* -0.52** 0.50** -0.23 -0.08
(-1.77) (-2.52) (1.68) (-0.60) (-0.02)
INDUSTRY_FS +/- 0.29 0.25 0.07 0.14 5.42
(1.44) (1.14) (0.17) (0.28) (0.85)
INDUSTRY_GV/NP +/- 0.47 0.20 1.01* 1.42** 2.59
(1.05) (0.47) (1.90) (2.39) (0.29)
INDUSTRY_MFG +/- 0.12 0.12 0.44* 0.21 18.32**
(0.63) (0.68) (1.29) (0.52) (2.05)
INHERENT_RISK + -0.02 -0.07 -0.22*** -0.10 1.35
(-0.45) (-1.28) (-2.74) (-0.85) (0.81)
FRAUD_RISK + 0.03 0.07* 0.17** 0.36*** 1.02
(0.52) (1.31) (1.93) (2.95) (0.45)
COMPLEXITY + 0.05 0.03 0.19 -1.52** 6.26
(0.21) (0.17) (0.43) (-2.52) (0.69)
TEAM_EXPERTISE +/- 0.02 0.09 0.17** 0.18** 1.53
(0.42) (1.63) (2.31) (2.10) (0.83)
PTR_CLIENT_EXPC +/- -0.21* -0.13 -0.34* -0.06 3.08
(-1.77) (-0.93) (-1.96) (-0.22) (0.74)
MGR_CLIENT_EXPC +/- 0.00 0.00 0.01* 0.01* -0.12
(0.35) (-0.64) (1.84) (1.93) (-1.13)
SR_CLIENT_EXPC +/- 0.00 -0.01* -0.03** -0.01 -0.29
(-0.47) (-1.67) (-2.53) (-1.07) (-0.88)
EXPERIENCE_FFR + -0.14 -0.14 0.26 0.49* 4.49
(-1.08) (-0.91) (1.06) (1.64) (0.97)
MANAGER1 +/- -0.50*** -0.41** 0.75*** 0.41 -0.630
(-3.94) (-2.40) (2.89) (1.25) (-0.18)
FORMAL_FORMAT +/- 0.17 -0.12 0.15 0.58 16.21**
(1.06) (-0.68) (0.43) (1.62) (2.21)
CONSTANT +/- 1.13 0.66 4.93*** 5.05*** -31.24
(1.57) (0.90) (5.10) (3.41) (-0.89)
Observations 143 145 145 145 146
Adjusted R2 .099*** .118*** .179*** .159*** 0.186
a See variable definitions in Appendix A.
b All dependent variables are continuous and are analyzed using OLS.
c Numbers in parentheses are t-statistics based on Rogers standard errors, which are clustered at the engagement level and
control for serial correlation and heteroskedasticity (Petersen 2009).
*, **, and *** represent significance at 10%, 5%, and 1%, respectively. All p-values are two-tailed except those with
directional predictions.
52
TABLE 6
Regression Results: Fraud Risk Factor Outcome Variables
Dependent Variables: RISKS_NUMBER, RISKS_NEW, %REV_REC, %OVERRIDEb
(6)
RISKS_
NUMBER
(7)
RISKS_
NEW
(8)
%REV
_REC
(9)
%OVER
RIDE
Variablea
Pred. Coefficient Coefficient Coefficient Coefficient
Signd (t-statistic)
c (t-statistic)
c (t-statistic)
c (t-statistic)
c
TREATMENT +
d 0.59** 0.38** -0.01 -0.040
(1.75) (2.10) (-0.24) (-0.82)
Control Variables
CLIENT_SIZE
+/- -0.07 -0.20** -0.02 -0.01
(-0.28) (-2.15) (-0.91) (-0.35)
PUBLIC +/- 0.10 0.07 0.07 0.02
(0.24) (0.39) (0.98) (0.52)
INDUSTRY_FS +/- 0.32 0.22 -0.25*** -0.10*
(0.53) (0.80) (-4.17) (-1.95)
INDUSTRY_GV/NP +/- 0.45 0.44 -0.07 -0.01
(0.79) (1.51) (-0.77) (-0.13)
INDUSTRY_MFG +/- 0.64 0.68*** 0.01 0.00
(1.48) (3.19) (0.18) (0.06)
INHERENT_RISK +/- 0.33** 0.13** -0.02 0.00
(2.33) (2.21) (-1.38) (-0.33)
FRAUD_RISK +/- -0.19 0.00 0.01 0.01
(-1.13) (-0.02) (0.37) (0.99)
COMPLEXITY +/- 0.32 -0.42 -0.12* -0.02
(0.57) (-1.52) (-1.72) (-0.30)
TEAM_EXPERTISE +/- -0.10 0.02 -0.02 0.01
(-0.69) (0.37) (-0.91) (1.28)
PTR_CLIENT_EXPC +/- 0.09 -0.30** -0.02 0.10***
(0.29) (-2.05) (-0.66) (2.93)
MGR_CLIENT_EXPC +/- 0.01 0.00 0.00 0.00
(1.34) (0.29) (-0.44) (-0.29)
SR_CLIENT_EXPC +/- 0.00 -0.01 0.00 0.00
(-0.07) (-1.33) (0.01) (1.32)
EXPERIENCE_FFR +/- -0.31 -0.07 -0.04 -0.01
(-0.80) (-0.44) (-0.93) (-0.40)
MANAGER1 +/- -0.16 -0.10 0.07* 0.05*
(-0.52) (-0.78) (1.81) (1.72)
FORMAL_FORMAT +/- 0.37 -0.09 0.04 0.06
(0.85) (-0.58) (0.86) (1.43)
CONSTANT +/- 2.59 1.28* 0.84*** -0.21
(1.52) (1.73) (3.44) (-1.29)
Observations 142 142 144 144
Adjusted R2 .019** .189** .216*** .068**
a See variable definitions in Appendix A.
b All dependent variables are continuous and are analyzed using OLS.
c Numbers in parentheses are t-statistics based on Rogers standard errors, which are clustered at the
engagement level and control for serial correlation and heteroskedasticity (Petersen 2009). d We predict a positive coefficient on the TREATMENT variable in the models with RISKS_NUMBER
and RISKS_NEW as dependent variables. We do not make directional predictions for the
TREATMENT variable in the models with %REV_REC and %OVERRIDE as dependent variables.
*, **, and *** represent significance at 10%, 5%, and 1%, respectively. All p-values are two-tailed
except those with directional predictions.
53
TABLE 7 Regression Results: Fraud Risk Response Outcome Variables
Dependent Variables: TAILOR, PROC_NUMBER, PROC_NEW, PROC_UNPRED, %RELATE_NATURE, %RELATE_EXTENT,
%RELATE_TIMINGb
(10)
PROC_
NUMBER
(11)
PROC_
NEW
(12)
PROC_
UNPRED
(13)
TAILORa
(14)
%RELATE
_NATURE
(15)
%RELATE
_EXTENT
(16)
%RELATE
_TIMING
Variablea
Pred. Coefficient Coefficient Coefficient Odds Ratiob Coefficient Coefficient Coefficient
Sign (t-statistic)c (t-statistic)
c (t-statistic)
c (z-statistic)
c (t-statistic)
c (t-statistic)
c (t-statistic)
c
TREATMENT +/- -0.13 0.13 -0.24 0.11*** 0.09* -0.09** -0.03**
(-0.16) (0.57) (-0.40) (-3.15) (1.86) (-2.03) (-2.04)
Control Variables
CLIENT_SIZE +/- -0.10 -0.18 0.06 1.08 0.00 0.01 0.01
(-0.23) (-1.51) (0.35) -0.19 (-0.11) (0.46) (1.02)
PUBLIC +/- 0.59 -0.25 0.69 0.79 0.08 -0.08* 0.00
(0.65) (-1.17) (1.63) (-0.31) (1.61) (-1.70) (-0.13)
INDUSTRY_FS +/- -0.76 0.07 -0.06 0.06** -0.03 0.02 0.00
(-0.69) (0.36) (-0.09) (-2.30) (-0.45) (0.44) (-0.08)
INDUSTRY_GV/NP +/- 0.31 -0.01 0.23 NA
d 0.15** -0.16*** -0.04
(0.18) (-0.04) (0.30) NAd (2.41) (-2.64) (-1.48)
INDUSTRY_MFG +/- 0.98 0.61 0.27 1.51 -0.04 0.05 0.00
(0.87) (1.62) (0.39) -0.53 (-0.80) (1.02) (0.33)
INHERENT_RISK +/- 0.37 0.02 0.03 1.13 -0.02 0.02 0.00
(1.47) (0.27) (0.23) -0.57 (-1.37) (1.46) (0.17)
FRAUD_RISK +/- 0.14 0.06 -0.06 0.76 0.02 -0.02 -0.01**
(0.39) (1.12) (-0.33) (-1.38) (1.63) (-1.23) (-2.27)
COMPLEXITY +/- -0.58 -0.41* -1.26** 6.09* -0.06 0.10** 0.00
(-0.52) (-1.69) (-2.40) 1.67 (-1.18) (2.15) (-0.05)
TEAM_EXPERTISE +/- 0.00 0.03 0.30* 1.17 -0.02 0.01 0.00
(-0.01) (0.65) (1.94) -0.64 (-1.24) (1.12) (1.17)
PTR_CLIENT_EXPC +/- -0.48 0.10 0.01 -0.44* 0.04 -0.03 -0.01
(-0.73) (0.66) (0.02) (-1.87) (1.08) (-0.77) (-0.65)
MGR_CLIENT_EXPC +/- 0.02 0.00 0.01 1.02 0.00 0.00 0.00
(1.15) (0.47) (0.81) -1.42 (-1.00) (0.96) (-0.37)
SR_CLIENT_EXPC +/- -0.02 0.00 -0.01 0.99 0.00 0.00 0.00
(-0.84) (-0.61) (-0.67) (-0.32) (0.98) (-1.21) (-1.11)
EXPERIENCE_FFR +/- 0.21 -0.21 0.18 1.13 -0.02 0.02 0.00
(0.28) (-1.37) (0.41) -0.21 (-0.43) (0.58) (0.16)
MANAGER1 +/- -0.20 0.03 -0.24 1.25 0.07* -0.06* 0.00
(-0.31) (0.26) (-0.51) -0.76 (1.71) (-1.67) (0.35)
FORMAL_FORMAT +/- -0.18 -0.11 0.11 0.34* -0.01 0.01 0.01
(-0.19) (-0.62) (0.19) (-1.67) (-0.20) (0.26) (0.72)
CONSTANT +/- 4.86 0.74 -0.35 0.46 0.86*** 0.03 0.05
(1.28) (1.10) (-0.26) (-0.19) (3.80) (0.12) (1.48)
Observations 136 136 136 135 400 400 400 R
2 measure
e .024** 0.075 .005** .300** .030* .033** 0.018
a See variable definitions in Appendix A.
b PROC_NUMBER, PROC_NEW, PROC_UNPRED, %RELATE_NATURE, %RELATE_EXTENT, and %RELATE_TIMING are continuous
dependent variables and are analyzed using OLS. TAILOR is a dichotomous variable (see Appendix A) and model (11) is estimated using
logistic regression; we report Odds Ratios and z-statistics for this model. c Numbers in parentheses are t-statistics or z-statistics, as noted, based on Rogers standard errors, which are clustered at the engagement level
and control for serial correlation and heteroskedasticity (Petersen 2009). d None of the 10 observations from the Government/Not-for-profit industry eliminated fraud risk procedures used in the prior year. This
indicator variable is therefore excluded from model (10) due to lack of variation on the dependent variable. e Adjusted R
2 reported for OLS models; Pseudo R
2 reported for logit model.
*, **, and *** represent significance at 10%, 5%, and 1%, respectively. All p-values are two-tailed.