automatic aggregation in auditing: with an application to systemic risk anticipation philip elsas...

Automatic Aggregation in Auditing: with an Application to

Systemic Risk Anticipation

Philip ElsasComputationalAuditing.com

Newark, New Jersey November 6-7, 2009

19th World Continuous Auditing and Reporting Symposium

ComputationalAuditing.com

Introduction• Since 2003: Company - Canada, Netherlands

• 1988-2003: Deloitte. with intermezzo at Bakkenist Management Consultants, sold to Deloitte.

• 1990-1996: PhD Computational Auditing

- Principal, chief architect & inventor of Smart Audit Support - Smart Audit Support: since 1994 key in Deloitte’s worldwide audit practice. Currently integrated in ‘The Deloitte Audit’- System blueprint in chapter 5 of …

- PhD in Mathematics & Computing Science, on Financial Auditing - In parallel to Smart Audit project, 30% part-time, Vrije Universiteit- Directly after appearance awarded with the biennial Alfred Coini Prize for the best publication in Auditing

Offering software and consultancy services to innovateaudit practices and audit software firms

1

The Dutch Tax Office used Computational Auditing in 2001-2003 as frame of reference to compare Big 4 planning and decision-support models & systems to investigate how to improve audit productivity (57 page report); considers Smart Audit Support ‘leader of the pack’


Agenda

Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation

• Web platform for audit support: “What is the content?”

• Aggregation mechanisms: quantitative, qualitative & confidence

• Web platform for audit support: “How to use that content?”

2

• Managing the use of aggregation & classification

• Royal NIVRA: ‘Golden opportunity for the audit profession’, Identify a way to contribute to systemic risk anticipation


Web platform for audit support: What’s the content?

by auditors, for auditors• ACL AuditExchange (AX 2), Business Assurance Platform

Interactive Audit

Documentation* Audit pack: a bundle of interrelated forms, specific for an industry, or sector

3

• Deloitte’s ‘Builder Player Platform’-architecture

• CaseWare Open Engagement & CaseWare IDEA

• Audit support architecture of a big audit firm, or of a shared back-office of a group of smaller audit firms

Audit repository: data, scripts for analytics (CM), findings

Working paper templates & scripts, DMS & KMS, partially organized per type of industry (website building system)

Platform of audit packs* with check lists & audit planning templates,organized per type of industry

All mentioned + capturing context to offer guidance indetermining & configuring scripts for data analysis,addressing the key questions:- “When to do which test?”- “What to do with the test results?”


p.334 4

p.337

Specified Audit Methods

drive integralPlanning,

Execution & Documentation

Proven Architecture‘Correctness by Construction’

Deloitte’s Smart Audit Support: Interactive Audit Documentation published in Word and browsers,World’s Strongest Audit Support*

* Dutch Tax Office

InstantaneousAdequate

Flexible Questionnaire integrated in Web Forms: By making explicit what is needed to answer “When to do which audit test?” & “What to do with the test results?” you articulate a body of multiple-choice questions, tables, etc., connected by choice-labeled relevancy links, embodying an approach, a method, or even, if possible, a workflow process, to guide how to achieve assurance

Effective: don’t miss relevant issue

Efficient: no access to less relevant issues

Drives & Captures the ‘Story of the Audit’

Optimal mitigation of litigation risk

Conditional Relevancy


Smart Audit Support’sdocument index related toDeloitte’s International Audit Approach(1990’s)

p.336

5

PERFORM PRE-ENGAGEMENTACTIVITIES

Assess Engagement Risk

Establish Terms of Engagement

Perform Preliminary Analytical Procedures

Understand the Client's Business

Understand the Accounting Process

Determine Planning Materiality

Develop Client-Service Objectives

Understand the Control Environment

Assess Risk at the Account and Potential-Error Level

Rely on Controls ? Control Reliance Strategy ?

Identify ControlsIdentify Controls and,if Efficient, Establisha Rotation Plan

Test Controls

Perform FocusedSubstantive Tests

Perform Basic Levelof Substantive Tests

Perform IntermediateLevel of

Substantive Tests

Evaluate Results of Tests

Perform Financial Statement Review

Perform Subsequent Events Review

Obtain Management Representations

Report on Financial Statementsand Render Management Letter

PERFORMPRELIMINARYPLANNING

ASSESSRISK

DEVELOPAUDITPLAN

PERFORMAUDITPLAN

CONCLUDEANDREPORT

That Mitigate Risk

Specific Identified Risk No Specific Identified Risk

NO YES YES NO

p.62

All planning docs are smart forms





All planning docs are smart forms with built-in

Conditional Relevancy

Example audit pack

In addition to $200M yearly cost reduction ROI is:- Relevant Doc & Planning, no more no less- Comfortable & stringent way to get it

Yearly ROI guess: 20K man-yrs/yr x $10K cost reduction/man-yr = $200M

Deloitte’s approach


Interactive Audit Documentation:Dedicated Functionalities for the Audit Team

Filling out a web-based questionnaire with multiple-choice questions:

“The Auditor’s New Clothes”, 2008, Tom Koning & the ‘Audit Navigator’,translation into English is pending

Capturing the ‘Story of the Audit’,ISA 315.122

6

Functionalities for audit workflow operators

• Activates dedicated support to indicate how to:– Specify a norm for an entity-level control– Specify a fraud risk, including a description of who is able & how to do it– Specify a norm for initial numerical analysis; when within norm, no extra

tasks– Specify or configure a script for a data analysis tool– Decide to involve an external specialist in your audit team (e.g. forensic, EDP)

• Activates relevant, more detailed questions & de-activates irrelevant

• Aggregates audit risk/audit evidence, according to a prescribed processing scheme, as captured in risk summarization tables

• Plans and configures audit tasks to constitute an audit plan, for example, based on accumulated risk:

– To be able to rely on a specific assertion level control– To further investigate the risk by planning substantive procedures

• Shows when to stop investigating an account, a process or an assertion

• Sets a risk classification to ‘significant inherent risk’

• Documents and guides: – “What has been done?” & “What has to be done?”– “What information has been found?” & “What’s the impact on the audit?”


Agenda





7




Web platform for audit support: How to use that content? ‘business wise’

by auditors, for auditors

Interactive audit documentation & business positioning:

8

Successfully positioned

within Deloitte

“Audit Software: From Bench Warmer to Star Player”, Royal NIVRA, “de Accountant”, March 2009, pp. 12-18,

Annotated translation into English by Dutch-American Translations & ComputationalAuditing.com

• Professional bodies of CPAs and standard setters upload high-level guidance packs à la ISA & strict forms à la Tax.Basis to be refined upon, but not overruled

• Building & uploading by fee-earning expert auditors

• Downloading & use by fee-paying engagement teams

• Broker-fee for the hosting platform provider

• Trade in audit packs between member firms

• External auditors develop tailored packs & on-line services for client’s internal audit department.Why? Marketing strategy of ‘vendor lock-in’


Web platform for audit support: How to use that content? ‘society wise’

by auditors, for auditors

Interactive audit documentation & ‘open pack’-platform:

9

“Audit Software: From Bench Warmer to Star Player”, Royal NIVRA, “de Accountant”, March 2009, pp. 12-18,

Annotated translation into English by Dutch-American Translations & ComputationalAuditing.com

Invitation to CaseWare & ACL: do you want to contribute to proposing a tailored version to AICPA & CICA?

• Uploading by content providing expert auditors, using a dedicated content builder

• Downloading by engagement teams,using a generic player to apply content

• Content is certified, published & hosted by A. an audit firm’s global and national office (layers)B. a professional body of auditors C. a standard setter or regulator

each granting access rights to their members, ideally with ‘content overlaying’ (A on top of B, B on top of C)


10

Recap ‘Builder Player Platform’-architecture

“How to get the data?” is not the challenge anymore. Today, audit analytics fully focuses on:

“How to use the data?” & “How to manage that use?”

Aggregation & classification are key methods of using data, so let’s have a look into how to manage aggregation & classification

“What keeps audit leaders up at night?”, ACL, 2008

Support in capturing

audit methods

Support in applying audit methods

Support in classifying audit methods

Goal of the PlatformGoal of the Builder Goal of the Player

Builder Player

“Audit Automation as the Foundation of

Continuous Auditing”, Michael Alles,

Alexander Kogan, Miklos Vasarhelyi &

Donald Warren, 16th WCAS, 2008


Agenda





11




12Aggregation scheme for risk assertions (cf 20)

Yahoo! SiteBuilder + own plug-ins to specify, visualize & interact with aggregation links (W3C SVG)

What do the arrows mean?

E.g. Table A1.2.1

accumulates risks regarding the assertion

‘Systems that retain …’

based upon underlying

feeding questions such

as E1.6 & classifies &

propagates the accumulated risk to Table A1.2 & A1 to contribute to driving the

configuring, via table S2, of

audit tasks constituting the

audit plan

Expressible, in a similar way, in Deloitte’s Smart Audit Support, see: ‘Computational Auditing’, p.328

Experiments with Adobe Flex, MXML & Google Open Docs, considering CaseWare’s Open Engagement Website Building System

Risk summarization tables capturing assertion-based aggregation schemes

Managing the use of aggregation & classification

The arrow is an Audit Workflow operator


Aggregation, Process Mining & Workflow 13

Input: event log with journals, e.g. SAP

Output: smart flowchart

Analyzing 3232 cases, classi-fying casualties (red arrows):A. Invoice receipt without prior approval (2537x)B. Approval acquired after purchase completion (261x)C. Purchase order established for rejected request (9x)D. Handled order status skip- ping receipt (875x), etc.


Based on: “Towards a Computer-Assisted Audit Analysis of Business Processes: Process Mining as Tool for IT Auditors”, Maria Bezverhaya, Emiel Caron & Piet Goeyenbier, ‘de EDP-Auditor’, NOREA, 2009

D

A

C

B

Design-time workflowvs. run-time workflow

Push signal from Technical University of Eindhoven, ProM, Fluxicon & Anne Rozinat

Pull signal from audit practitioners & IT audit educators

Computational Auditing: - focus on discovery of supercycle - framing ‘stand alone’ workflows- connecting to 80 years of audit theory


M: Majority Owner-ManagerS: Sales departmentB: Buy/Purchase departmentF: Financial departmentT: IT departmentW: Warehouse managerL: Labor/salary accountsP: Planning departmentC: Creditor accountsD: Debtor accountsA: Application

Agent Legend

C b f t

F m d

D s t

A tL f t

P t

P t

W t

A t

A t

S

A

AL F

L F

L F

MM D F

D

C

B F

B F

W

P

P

P

P

W

A

A

A

A

C mD f t

S t

A t

F t

B f t B f t

P t

W t

L f

225

25 200

225

500

25

25

1,000400

400100

20

20

20

20

500

400

Agent’s access is associated to:1. Transactions2. States3. Flows

Capital letter: authorized, legitimate accessSmall letter: illegitimate access

14Ernst & Young’s Smart Flowchart Pilot Study

Case by Hans Verkruijsse & EY team, 2005-2006

More on integrated audit analytics: “Enterprise-level Process Documentation incorporating Automatic Audit Analytics”, 2008 Deloitte/KU Symposium & follow-up with Raj Srivastava & EY CARAT

Approach: Powerful and easy system to support practice, founded in theory

World’s strongest ‘business process’-oriented auditing theory: classical Dutch auditing theory (80+ years)

& its best-fitting rigorous process theory: Petri nets tailored to the auditing domain

Dynamic: Transaction Profit & Loss Item

T

Static: State Balance ItemS

Top-level is Supercycle, or Top-cycle. Connects traditional cycles

Case in Efrim Boritz’ CAATTs class, 2007-2008

Fit recognized by Jagdish Gangolly, 2007-2008

EY’s evaluation report:- Clarifying. Refreshing.- Systematic framework guides input preparation process (2009: new style)- Quantitatively motivated process decomposition


New in 2009:Process mining; pilots by a Big 4, UvA.nl & CWI.nl Focus on top-cycle discovery

Output: 1. ‘As Is’ diagram (‘Ist’)2. Identify ‘To Be’ (‘Soll’)3. Built-in audit analytics

Input: event log


Typology of Top-cycles 15

Scientific foundation: rationally rigorous. With mathematical & computational formalization.Superbly suited for the digital age. Recognized as such in accelerating pace. Easy by new tech

Limperg, Starreveld, Frielink, Blokdijk & Veenstra


Top-cycle: normative backbone of the ‘business process’-oriented audit approach

previous slide:

example supercycle

Top-cycle concept & typology: Central result of integral evolution. Of ‘business process’-oriented Auditing Theory, Auditing Practice & Auditing Education. Over 60-80 years

Typology/classification of top-cycles: ordered by the strength of the backbone

Unfortunately hardly translated into English


16

Starreveld et al.

Typology of Top-cycles

Frielink et al.

Supercycle-backboned

Audit Approach

Volumes 1, 2a, 2b, etc.


‘Industry classification’-based auditing concepts,

norms & methods

Decisive advantage of these concepts, norms & methods: no need to prove again in practice, since practice was

part of the evolution process


Agenda





17




Mechanism for quantitative aggregation 18

2 Receivables

3 Inventories+ =

See: “On Positioning XBRL Assurance Business Rules in a Computational Infrastructure for Modern Auditing”, 2009, University of Kansas, Annual International Conference on XBRL

Aggregation in XBRL: - Calculation linkbase- XBRL Formula

Plug-in ‘type polymorphism’ mechanism (transferable) from programming language into XBRL Assurance Builder & Player

Domain-Specific Language (DSL) for auditing: Pacioli, developed by Dutch software partner in cooperation with national research center for mathematics and computer science in the Netherlands (CWI) & University of Amsterdam

5 Assets

5 Current Assets

At least one noncurrent inventory

All three inventories are current

{XBRL US GAAP Taxonomy

or

Articulate XBRL Assurance functionality using a dedicated website builder (plug-ins) instead of handcrafting XBRL Formula’s

Type Polymorphism: Least Upper Bound in the Taxonomy


For reasons of efficiency: establish a full aggregation as early as possible in the audit process (observation by William Kinney)

Mechanism for qualitative aggregation: 19

Irreplaceable in the sense that there is no way for an external auditor to compen-sate its lacking or failing, while it is indispensable for a rationally justifiable approval

“X-Raying Segregation of Duties: Support to Illuminate an Enterprise’s Immunity to Solo-Fraud”with discussions & response, IJAIS, June 2008

Solo-fraud free? Design, Implementation & Operation

Continuous auditing web service (hosted via external auditor?)intercepts every Authorization Change Request to signal:

refuse

human intervention required

OK

Efrim’s proposal (2008): Large-scale introductory study for this science-based method. As for new medicine. New method on top of Dutch auditing theory as incarnated in computational process theory. Collaboration with Canada. Identification of budget doubling when large audit firm steps in. Current status: pilots by Big 4 Dutch member firm

Method locatingwho has too manyauthorizations inone hand creating a dangerous opportunity for traceless embezzlement,jeopardizing the integrity of financial statements

How to aggregate weak spots in the Internal Control that are both irreplaceable and indispensable, e.g. weak spots in Segregation of Duties?

“Get it right at entry level”

Focal point in modern auditing? Launched at Accountant.nl by Jules Muis, Oct. 2009. Directly endorsed by Hans Blokdijk, Marc van Hilvoorde and others. Berry Wammes, CEO Royal NIVRA, directly stated the intent to position “Get it right at entry level” as the theme for the NIVRA spring 2010 debate series

Top-of-iceberg solo-frauds:1. Madoff2. Stanford 3. Kerviel, etc.

Clarifies why & how weak spots in the SoD require a hot-line direct-top-level aggregation mechanism


20Mechanism for confidence-level aggregation (cf 12)

Based on:

Sun,Srivastava& Mock,2006

“An Informa-tion SystemsSecurity RiskAssessment Model”,pp. 43-48

This can be realized in Deloitte’s Smart Audit Support with a plug-in for Dempster-Shafer-Srivastava confidence-level computations


Agenda





21



Early Warning System as Killer App for XBRL Assurance & Continuous Auditing: speeding up getting their ‘Place & Future’ into ‘Here & Now’

“The PCAOB and the Social Responsibility of the Independent Auditor” Douglas Carmichael, Founding Chief Auditor of the PCAOB

‘Golden Opportunity’

Jan Helderman,President

Royal NIVRA,Accountant.nl,

Sept. 2009


Proposed Solution1. An off-the-shelf system for tracking-and-tracing bar-coded products,

configured for, and populated by ‘XBRL tagged’ financial products

2. A regulator-mandated auditor attests internal controls for the XBRL reporting channel to the new governmental systemic risk agency. Allowing for a continuous data stream—further subjected to audit tests, sampling & monitoring—with on-the-fly automatic aggregation into systemic risk indicators (release 1.0: ‘Bookstaber’ indicators)

How far away? XBRL Assurance is closer than ever

22‘Golden Opportunity’ Royal NIVRA: “Preparing for an Audit Mandate to Contribute to Systemic Risk Anticipation”, magazine, web & adopted in ‘Sharing Knowledge’-project

1. Instead of expecting more from XML, start expecting more from the builder-based approach to XBRL & continuous auditing

2. Release 1.0: matter of weeks or months, not years

Jumpstart by cooperation of top-specialists Rick Bookstaber, Miklos Vasarhelyi, Raj Srivastava & Charlie Hoffman, and preferably in cooperation with a Big 4 audit firm

Small step for XBRL & Continuous Auditing,

quantum leap for the financial world

Bailing out inflates moral hazard, early warning deflates

More rigor on macro, more rigor on micro: use Dutch auditing

Limperg’s Theory of Rationalized Confidence

Dutch Auditing Day, hosted by Royal NIVRA, November 25, 2009, agenda’s keynote & key discussion: “risk systems & systemic risk”

automatic aggregation in auditing: with an application to systemic risk anticipation philip elsas...

Documents

audit practices

audit productivity

audit profession

smart audit project

smart audit support

audit software firms

big audit firm

worlds strongest audit