2015 isaca nacacs - audit as controls factory
TRANSCRIPT
Audit As A Controls Factory
Nate Anderson, Internal Audit, SearsCliff Nuxoll, Internal Audit, Sears
PRESENTATION OBJECTIVES
• Overview of data analytics concepts– Summarize audit analytics concepts & tools– Reinforce concepts through examples & lessons– Analytics team best practices– Present practical tools & approaches to
analytics
• Challenge traditional view of Audit Analytics– Consider services Audit can provide while
remaining independent and objective
OUTLINE
• Audit analytics – Overview
• Key ingredients to audit analytics– Methodology & Approach– Building an analytics team– Overview of commonly used tools
• Analytics in action– Monitoring controls– Audit aids– Ad-hoc analysis
• Lessons learned• Maintaining Independence & Objectivity
AUDIT ANALYTICS OVERVIEW
• Definition
• Industry Insights
• Key Trends
• Key Ingredients
AD-HOC ANALYSIS
Auditor obtains useful data
Data is loaded for analysis
Results of analysis
Summary insights
Goals: Test general hypothesis (e.g., determine root cause for sample of negative margin sales)
AUDIT AUTOMATION
Auditor aid engaged
Automated routine
Results for auditor
Analytics Routine/Program
Goals: Improve efficiency, accuracy, or effectiveness of audit processes
CONTINUOUS AUDITING / MONITORING
Analytics Routine/Program
Data feed to audit
Automated routine
Output for action/decision
Goal: Enable risk monitoring, support risk decision, and/or facilitate control activity
STATISTICAL ANALYSIS / MODELINGData feed
to auditStats/modeling
routineOutput for
action/decision
Goal: Descriptive statistics procedure or modeling to test hypothesis, increase understanding, or make
prediction
INDUSTRY INSIGHTS
• PwC 2014 State of the IA Profession Survey
• Protiviti 2015 IA Capabilities & Needs Survey
PWC 2014 STATE OF PROFESSION SURVEYHow is Internal Audit doing?• 49% (senior mgmt) & 60% (board) believe IA is
delivering on expectations• 45% (senior mgmt) & 70% (board) believe IA adds
significant value• 29% (senior mgmt) & 51% (board) believe IA is
leveraging technology effectively in execution of audit services
Where are the opportunities for IA to improve?• #1 area respondents want greater IA involvement
in: – Increased reliance on big data & analytics (80%)
• “[IA] functions should always be looking to add value by expanding their capabilities in [data analytics].”
PROTIVITI 2015 IA SURVEY
• 5 of 7 areas (out of 36 total) where audit improvement is most urgently needed relate to analytics.
• Data analytics skills were the top area of desired growth in 2013 (4 of top 5) and 2014 (6 of top 9)
“Need to Improve” Rank1 Auditing IT Security
1 (tie) Computer-assisted audit tools (CAATs)
3 Data analysis tools – data manipulation
4 Marketing internal audit internally
5 Fraud – monitoring
6 Data analysis tools – statistical analysis
7 Continuous auditing
PROTIVITI 2015 IA SURVEY
• “There continues to be significant dialogue among internal audit functions about the need to leverage technology-enabled auditing tools, but they are not achieving progress.”
• “CAEs and internal audit leaders should consider whether this is becoming a never-ending journey”
• “Will [audit analytics] continue to be discussed but not implemented?”
KEY TRENDS
• Democratization of data
• Visualization growth
• On-demand computing power
KEY TRENDS: DEMOCRATIZATION OF DATA
Major growth in data
Unstructured Structured
80% 20%
Majority is unstructured & raises new opportunities & concerns
New methods to store, access & analyze unstructured data
KEY TRENDS: DATA VISUALIZATION GROWTH
Significant advances in visualization tools
KEY TRENDS: ON-DEMAND COMPUTING POWER
Leverage cloud for power & storage
KEY INGREDIENTS TO AUDIT ANALYTICS
Approach
Tools
Team
Methodology
AUDIT ANALYTICS METHODOLOGY
Problem to analyze
Get/Process
data
Analyze results
Measure insights
Apply learnings
ELEMENTS OF AGILE PHILOSOPHY
Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.
Just do it.
AGILE MANIFESTO
“We are uncovering ways of developing software by doing it and helping others do it. Through this work we have come to value:
That is, while there is value in the items on the right, we value the items on the left more.”
Individuals & interactions Over Processes & toolsWorking software Over Comprehensive documentationCustomer collaboration Over Contract negotiationResponding to change Over Following a plan
AGILE ELEMENTS WITHIN OUR APPROACH• Agile
– Obsess over problem to be solved– No “analysis paralysis”– Delivery early, often, and modestly (small
releases)– Improve incrementally– Learn from reality quickly and with little money
• Traditional– Dangerous set up: Design everything, code
everything, promise to deliver big later.– Rigid scope and plan– Over-reliant on consultants
ATTRIBUTES OF AGILE TEAMS
• Culture of transparency without penalties
• Reward early experimentation (and failure)
• Self-organizing and self-managing teams
• Cross-functional teams“I had never failed. I’ve just found 10,000 ways which do not work.”
- Thomas Edison
CHANGING WITH TECHNOLOGY
Leverage data warehouses
Leverage big data
Leverage open source
1970 2015Time
Com
plex
ity
AUDIT ANALYTICS TEAM
Insights
Coder
Analyst
Business
Expert
SKILLSET: BUSINESS EXPERT
• Leverages personal insights and relationships• Focus on solving real world problems• Business unit experience• Prioritize risks
Problem to analyze
Get/Process
data
Analyze results
Measure insights
Apply learnings
SKILLSET: CODER
• Knows where and how to gather data• Able to code in multiple languages• Works well with key IT practitioners• Developer experience
Problem to analyze
Get/Process
data
Analyze results
Measure insights
Apply learnings
SKILLSET: ANALYST
• Evaluate key risks based on data• Drive solutions based on analysis• Excellent problem solver• Can visualize results
Problem to analyze
Get/Process
data
Analyze results
Measure insights
Apply learnings
ANALYTICS LEADERSHIP TEAM
CAE
AnalystsBusiness Experts
Coders
• Sponsor key to success• Must be open to any approach that gets results
• Strong practitioner• Great business knowledge
• Strong practitioner• Understands how to
manage IT resources and projects
Analysts
IT Audit Lead
Corporate Audit Lead
TYPICAL ANALYTICS PROCESS FLOW
Requirements
Business Expert
Coder Analyst
LESSONS LEARNED: RESOURCING
1. Diversity is critical.
2. Be ready to replace key personnel.
Auditors Coders
Coders Business Experts
AUDIT ANALYTICS TOOLS
Visualize
Analyze
OrganizeAcquire
MICROSOFT OFFICE SUITE
Acquire / ETL Organize Analyze Visualize Price Difficult
y
TOP AUDIT ANALYTICS SOFTWARE
Acquire / ETL Organize Analyze Visualize Price Difficult
y
GARTNER MAGIC QUADRANT – BI TOOLS
Top tier Open source
Completeness of Vision
Abi
lity
to E
xecu
te
TOP VISUALIZATION SOFTWARE
Acquire / ETL Organize Analyze Visualize Price Difficult
y
MICROSOFT BI TOOLSET
Acquire / ETL Organize Analyze Visualize Price Difficult
y
TOP BI OPEN SOURCE (FREE)
Acquire / ETL Organize Analyze Visualize Price Difficult
y
TECHNOLOGIST TOOLS
Acquire / ETL Organize Analyze Visualize Price
ANALYTICS SOLUTION EXAMPLES
• Monitoring Controls– Patriot Act Compliance – Pharmacy Compliance – Gift Card Compliance
• Audit Enhancement– Access Benchmark
• Ad-Hoc Risk Analytics– Gift card analytics– Employee Store Risks– Telecom spend
MONITORING CONTROLS
• Hosted web applications– Patriot act compliance– Pharmacy compliance– Gift Card compliance
• Collaboration between business & audit
• Aid business in mitigating significant risks
PATRIOT ACT COMPLIANCE
• Replaced pre-existing weekly Excel reports with continuous online tracking system – accuracy improvement of 500%
• Findings are generated nightly and appended to the current report
• Related transaction details are populated under each finding
PHARMACY POLICY COMPLIANCE
• Requested by Legal to protect against costly fines• LDAP-authenticated system requires Pharmacists
and Pharmacy Managers to agree/ disagree to policy on a weekly basis
• Users sign in and enter pharmacy location number
PHARMACY POLICY COMPLIANCE
• Once signed into the system with a user id and location number, users come to the policy page
• Upon agreement, user information and pharmacy location are logged
• In the case of a disagreement, Managers & Directors are notified via email to take appropriate action
GIFT CARD COMPLIANCE
Periodic review and action (sign-off) on potential risk events:
• Required sign-off
• Business unit management oversight of sign-off, participation, risk events
AUDIT ENHANCEMENT
• Hosted web application– Access benchmark
• Improves audit activities
• Typically enhances: – Efficiency– Effectiveness– Uniformity of approach
ACCESS BENCHMARK
Concept: - Access list repository for audit & IT compliance- Regular snapshots of access for critical IT assets- Enables self-service access reviews by control owners
ACCESS BENCHMARK – COVERAGE
Sarbanes-Oxley IT Components
Count
Environments (LDAP, AD, etc.) 10+
Applications 50+
Databases 150+
Systems 200+
Datasets 50+
Production Directories 50+
Utilities 5+
• Implemented across LDAP, Active Directory, mainframe hosts, Sun, AIX, Linux, HP-UX, Windows, AS/400, MySQL, SQL Server, DB2, Oracle, Teradata, Informix, PeopleSoft, etc.
ACCESS BENCHMARK – WALK-THROUGH• Primary functions:
– Admin – Add IT assets, map reviewers, manage access
– Reviewer – Down/upload of mapped access reviews– Auditor – Download of completed reviews
ACCESS BENCHMARK – REVIEWER VIEW
# of accounts requiring review
All IT assets related to
user
Download current list
Relevant technology layer
ACCESS BENCHMARK – REVIEWER VIEW
Enabled drag and drop of completed access reviews
Upload occurs; data validation performed
ACCESS BENCHMARK – REVIEWER VIEW
ACCESS BENCHMARK – AUDITOR VIEW
Download List
Select technology layer
Select review “as of” date
ACCESS BENCHMARK – BENEFITS
• Effective access reviews and re-certifications
• Uniformity in approach & quality
• Enables 100% coverage (all IT assets & accounts)
• Solution is scalable (can leverage for SOX, PCI, etc.)
• Accurate “critical information asset” inventory
• Value of weekly access snapshots
AUDIT ENHANCEMENT “MUST HAVES”• Ready access to:
– employee & contractor data– Key transactional data access (e.g., point-of-sale)
• Statistical aides (assist with sample selection, etc.)
• Focus on repetitive activities in areas such as compliance
AD-HOC RISK ANALYTICS
• Conducted with desktop software– Gift card analytics (tableau)– Store employee risks (power bi)– Telecom spend (tableau)
• Enhances risk assessments, audits
• Requires savvy & assertive auditors
GIFT CARD ACTIVITY OVER TIME
Day Dt
2014
Q3 Q4
2015
Q1
Aug 11 Aug 26 Sep 10 Sep 25Day of Day Dt [2014]
Oct 1 Nov 1 Dec 1 Jan 1Day of Day Dt [2014]
Jan 14 Jan 29 Feb 13Day of Day Dt [2015]
0
10
20
30
40
50
60
Gift Cards Issued
Gift Card Trend by Date
Continuous control implemented Flawed program
launched; quickly addressed
145
126
114
75
15
4515
59
49
78
48
88
63
33
27
2716
36
76
60
24
64
34
1211
91
41
5
5
9
3
7
6
4
4
2
2
1
1
Gift Card by State
SUSPICIOUS ACTIVITY BY STATE
States with significant activity
States where no activity is allowed
Dist Mgt Name
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150Gift Cards Issued
Abraham
Bill Joe
Billy Bob
Billy Jack
Billy Jean
Bobbie Sue
Carl
Carol Sue
Cliff
Dan
Jason
Jasper
John Boy
Johnny
Joya
Kelly
Krista
Krueger
Larry
Lea
Leroy
Lucy
Mack
Nate
Patty
Richard
Ricky Bobby
Tim Bo
Virgil
Wilber
Wyatt
Zeb
Gift Card by District Manager
SUSPICIOUS ACTIVITY BY DISTRICTDistricts with significant suspicious activity
STORE EMPLOYEE RISKS
Shifts < 3 hours
Qty of edits
Qty of self-corrects
Qty of self-corrects
STORE EMPLOYEE RISKSHigh qty of self-corrections to hours
High qty of manual hours edits
High qty of both concerns
TELECOM SPEND
• Where is biggest cost recovery opportunity?– Over allocation / overcharge– Obscure service charges– International call/text usage– Unneeded feature removal– Closed sites / lines not in use– Call/text/data plan optimization– General use overage
TELECOM SPEND: VENDOR 1Quickly highlight key cost recovery opportunities
~$350k savings proposed
TELECOM SPEND: VENDOR 2Quick overview of amount of recovery by reason
~$2.2m savings proposed
Top recovery reason: Unused lines/circuits
TELECOM SPEND: CLOSED SITE/ UNUSED LINES
SHMC-38445 and SHMC-99999 may be false positives; need more data
Abnormally large sites: - Store- Corporate
Significant number relate to corporate
TELECOM SPEND: BY SITE
Identify greatest opportunities for preventive controls
TELECOM SPEND: DRILL-DOWN ON CORPORATE
Visualization Summary: • Quick, big-picture view• Convey conclusions & approach to key stakeholders
LESSONS LEARNED
• Most valuable technical skill
• Toolbox approach
• Affordably sourcing team
MOST VALUABLE TECHNICAL SKILLS1. SQL. And then really advanced SQL.Learn it.Love it.Live it.Essential for finding, browsing, evaluating, analyzing, and filtering data
2. Excel – Lots can be done before limitations emerge
3. Tableau – Includes all essential ingredients
4. Depends on the need, familiarity, etc.
TOOLBOX APPROACH: BEST TOOL WINS• What step are you on in your data analytics
journey?
• How to move forward without:– Looking too far ahead– Spending unnecessary $$$
• Successful tools for Sears Holdings:– Everyone: Excel, Access– Front-end team: ACL, Tableau– Back-end team
• Linux servers (free, powerful server)• MySQL (free, powerful database)• Cassandra (free, powerful NoSQL database)
AFFORDABLY SOURCING TEAM
1. Coders as interns– Freedom and creativity of role should appeal to
them– Do not ask them to be auditors
2. Data analysts as interns – Subject matter is attractive (fraud, security, etc.)
3. Auditors with coding background– Increases likelihood of obtaining versatile data
analytics practitioners
ENTERPRISE RISK MANAGEMENT FAN
* Internal Audit acts as facilitator and host only
INDEPENDENCE & OBJECTIVITY
“Independence is the freedom from conditions that threaten the ability of the internal audit activity to carry out internal audit responsibilities in an unbiased manner.”
“Objectivity is an unbiased mental attitude that allows internal auditors to perform engagements in such a manner that they believe in their work product and that no quality compromises are made. Objectivity requires that internal auditors do not subordinate their judgment on audit matters to others.”
– Section 1100 – Independence and ObjectivityInternational Standards for the Professional
Practice of Internal Auditing
INDEPENDENCE IMPAIRMENT THOUGHTS• Are we “implementing risk responses on management’s
behalf”? • Are we “taking accountability for risk management”?• Are we remaining able to audit these controls without
bias?
1. We are remaining independent of the performance of the control, we are unbiased, while we are increasing our control oversight.
2. We do not make risk response decisions; we do not manage risk for management.
Most Importantly: If we never have to answer these questions, how much value are we adding?
