implementing a risk-based title iv audit program presented by: jarod paulson, compliance manager...
TRANSCRIPT
Implementing a risk-based Title IV audit program
Presented by:Jarod Paulson, Compliance Manager
Erin Hage, Audit and Compliance Specialist
3
Agenda
• Compliance framework• Identifying and assessing risk• Measuring compliance results• Using audit results to continuously
improve• Capella’s risk based TIV audit
program• Questions
4
Compliance framework
• Maintaining TIV compliance is essential– Non-compliance threatens your institution’s
ability to participate in TIV, can result in fines, and damage to brand.
• The external regulatory environment is fluid– Negotiated rulemaking, Reauthorization
• An audit program should Identify compliance gaps, Measure effectiveness, and provide visibility to needed improvements– Changing behavior vs. providing information
5
Identifying and assessing risk
• Risk Based Approach – Risk Assessment
• Top 10 audit and program review findings• Prior Audit Findings or Program review findings• NASFAA or ED assessment tools• FSA handbook, Bluebook & CFR
– Risk Rating• Determine which rules are most impactful for your
institution and design controls in response to residual risk.
– Sampling Methodology• Sample sizes should be risk based and used to
identify; not control risk. • e.g. 20, 40, 60 risk based, SOX methodology
6
Measuring compliance results
• Individual quality monitoring • Process monitoring• Mock audits• Monitoring schedule for non-
operational requirements
8
Using audit results to continuously improve
• Coaching– Individual performance and accountability
• Training– Job aid resources– Policy and Procedures
• Reporting– Provide visibility to process owners and
key stakeholders
• Remediation tracking– Lean methodology
9
Using audit results to continuously improve
• Process Improvement principles– Redesign– Eliminate– Standardize– Integrate– Automate– Off-load– Clarify
11
Capella’s risk based TIV audit program
• Individual Audits– Purpose
• Quality Assurance• TIV Compliance• Performance Improvement• Visibility to competencies on an individual level
12
What’s next?
1. Identify tasks2. Risk rate each task3. Divide tasks into groups4. Establish a sample size5. Create testing attributes6. Create goals7. Create reporting8. Create a remediation plan
13
Identify Tasks
• Determine what is considered a task– e.g. Verification, R2T4
• Ensure you have visibility to all potential “tasks” at your school
14
Risk rating
• The following factors can be used to determine the risk of each task:– Audit Risk– Complexity– Business Environment– Volume– Impact
15
Risk rating
• Use a scale of 1-3 for each factor– Example:
Complexity
3 – Complex (multiple steps, multiple people involved, multiple dependencies/inputs
2 – Mildly Complex (less than 3 core steps to the process/handoffs)
1 – Simple (low dependency on other processes)
16
Risk Rating
• Average the 5 risk ratings for each task to determine the final risk rate.
Audit Risk Complexity
Business Environment Volume Impact Average
R2T4 Calculations 3 3 1 1 3 2.2
Awarding TEACH Grant 2 3 3 1 2 2.2
17
Divide tasks into test groups• Find similarities among the tasks
and begin to group similar ones together– e.g. R2T4 and PWD calculations could
be combined into one test group
• Other example test groups may be:– Federal Grants– Application– Award Adjustments
18
Establish Sample Size
• Choose a relevant sample size– 20,40,60
• How many hours can be dedicated to this audit?
19
Create testing attributes
Test Group Attribute 1 Attribute 2 Attribute 3 Attribute 4
Federal Grants
Learner is eligible for the grant awarded
Learner was awarded the correct Grant amount
Experienced Teacher Letter/Test Score is sufficient (if applicable)
If funds were authorized, learner is eligible for the disbursement
• Identify what makes a task correctFor example:• Student meets eligibility requirements• Correct award amount
20
Create goals
• Set an Error Rate or Success Rate goal for individuals and the team
• Delivering a success rate instead of an error rate to individuals may create a more positive feeling surrounding audit results– e.g. 3% error rate = 97% success rate
21
Create Reporting
• What do you want to report?– Employee error rates– Team error rates
• How frequently will you report the results?–Monthly, Quarterly, Annually
22
Create Reporting
• Methods–Monthly result meetings– Charts/graphs– Summary by test group
23
Create Reporting – Monthly Error Rates
Test Group Name Federal Grant Test GroupMonth Name January
Processor Items Failed Items Tested Error RateEmployee 1 0 0 0%Employee 2 0 10 0%Employee 3 1 10 10%Employee 4 0 5 0%Employee 5 0 0 0%Employee 6 0 0 0%Employee 7 0 0 0%Employee 8 0 10 0%Employee 9 0 10 0%Employee 10 0 0 0%Team Totals 1 45 2%
24
Create Reporting – Monthly Error Rate
Test Group Name Federal Grant Test Group Federal Grant Test Group Federal Grant Test Group Federal Grant Test Group
Month Name January February March YTD - 2015
ProcessorTotal Failed
Total Tested
Error Rate
Total Failed
Total Tested
Error Rate
Total Failed
Total Tested
Error Rate
Total Failed
Total Tested
Error Rate
Employee 10 0 0% 0 0 0% 0 0 0% 0 0 0%
Employee 20 10 0% 0 10 0% 0 10 0% 0 30 0%
Employee 31 10 10% 1 10 10% 1 10 10% 3 30 10%
Employee 40 5 0% 0 1 0% 0 5 0% 0 11 0%
Employee 50 0 0% 0 0 0% 0 0 0% 0 0 0%
Employee 60 0 0% 0 0 0% 0 0 0% 0 0 0%
Employee 70 0 0% 0 0 0% 0 0 0% 0 0 0%
Employee 80 10 0% 1 10 10% 0 10 0% 1 30 3%
Employee 90 10 0% 0 7 0% 0 10 0% 0 27 0%
Employee 100 0 0% 0 0 0% 0 0 0% 0 0 0%
Team Totals1 45 2% 2 38 5% 1 45 2% 4 128 3%
25
Create Reporting – Isolating the issue
Federal Grant Test Group R2T4 Test Group Award Adj Test Group
YTD Error Rate - 2015 YTD Error Rate - 2015 YTD Error Rate - 2015Employee 1
0% 0% 0%Employee 2
0% 0% 5%Employee 3
10% 0% 5%Employee 4
0% 5% 4%Employee 5
0% 0% 2%Employee 6
0% 0% 4%Employee 7
0% 0% 5%Employee 8
3% 0% 2%Employee 9
0% 0% 5%Employee 10
0% 0% 5%Team Totals
3% 3% 4%
26
Create reporting – Individual employee
Award Ad-justment
R2T4 Federal Grant Overall error rate0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
5%0%
10%5%
Employee 3 YTD Error Rate
Employee 3 YTD Error Rate
27
Create a remediation plan
• Identify who is accountable for taking action on the data
• Communicate results to individual processors
• Determine if training or coaching sessions need to scheduled
• Track your remediation plan• Report the outcomes
28
Capella’s risk based TIV audit program• Monthly self-audit
– Process based audits– Test frequency and sample size determined
by risk rating– Red, Yellow, Green tracking
• Rolling results, quarterly trending and root cause.
– University wide partnership in remediation– Deficiencies are given high visibility
• Department and university leadership, cross functional compliance committee, BOD audit committee.
• Other Audits (based on annual audit plan)
29
Capella remediation structure
31
key learnings
• TIV compliance is an institutional responsibility (partnerships across the university)
• Risk assessment should be ongoing• Devoted resources • Zero findings mindset (you will never
be perfect)• Respectful of people, reward good
results, error rate vs. quality score, explain the why.
32
Results
• Continued and increased investment in the program
• Stronger partnerships with other departments
• Strong Compliance results and reputation
33
Questions?