kristen grenzebach, learning program manager, anthem inc. · kristen grenzebach, learning program...
TRANSCRIPT
2/12/2015
1
Jessica Vander Zanden, CHC Dirc. Of Medicare Compliance and
Star Quality, Network Health
Kristen Grenzebach, Learning Program Manager, Anthem Inc.
1
Kristen Grenzebach, Learning Program Manager, Anthem Inc.([email protected])
2
Sunday, February 15, 2015 – 12:30-2:00pm
2/12/2015
2
� About Anthem – A Health Benefits Leader
� Course History
� “It Takes a Village”
� Defining the Audience
� The Evaluation
� Analyzing the Results
4
`̀
BC or BCBS licensed plans (6)
Medicaid presence (11)
BC or BCBS licensed plans + Medicaid presence (8)
BC or BCBS plans in 14 states and
Medicaid presence in 19 states
MedicareMedicareMedicareMedicare
4%4%4%4%
BlueCardBlueCardBlueCardBlueCard
14%14%14%14%
MedicaidMedicaidMedicaidMedicaid
13%13%13%13%
Individual Individual Individual Individual
5%5%5%5%
National National National National AccountsAccountsAccountsAccounts
19%19%19%19%
LocalLocalLocalLocalGroupGroupGroupGroup
41%41%41%41%
FEPFEPFEPFEP
4%4%4%4%
Diverse customer base of
37 million medical members
2/12/2015
3
• Annual web-based, self-paced training since 2002
• Content: Ethics, Compliance, Privacy, Fraud & Abuse, Information security and HR
• Partner with Training Systems Design (TSD)
• Hosted on LMS powered by Saba
• Customized by audience selection
Collaborate with business areas
Set expectations for all users
Acknowledge feedback
Different
viewpoints,
priorities
and
expertise
Face
challenges
Tackle
problems
Find
solutions
Deliver
results
2/12/2015
4
KPI Definition
Quality/Satisfaction:
Courseware Effectiveness
& Delivery
Effectiveness of course design including: content, scope
(breadth/depth), flow, degree of interactivity.
Effectiveness of online delivery including: ease of use and
navigation, interaction opportunities, graphics and
illustrations that are meaningful and within context.
Learning EffectivenessInsight into the level of learning (new knowledge and skill
gains)
Job Impact
(Ability to Apply)
Insight into the level and likelihood of application and
impact to the job
Value
(Return on Expectations)
Indicator of overall value and perceived benefit of the
course vs. the opportunity cost of being away from the job
• Measure 4 Key Performance Indicators (KPI)• Analyze all comments into category themes• Offer opportunities for improvement
2/12/2015
5
Start upStart upStart upStart up
Statement of work
Goals/Instructional
objectives
Requirements and
constraints
Analysis & DesignAnalysis & DesignAnalysis & DesignAnalysis & Design
Completed
information
gathering
Content points
Instructional
strategy
DevelopmentDevelopmentDevelopmentDevelopment
Instructional flow
Scripts and
screens
ProductionProductionProductionProduction
User Acceptance
Pilot version
Final product
Implement & Implement & Implement & Implement &
EvaluateEvaluateEvaluateEvaluate
Master course
Other services as
contracted
2/12/2015
6
� Pilot Testing – Pre Course
� Focus Group – Post Course
� Manager Follow Up
2/12/2015
7
Going from one-size-fits-all to the right ‘break down’
•Defined “functional areas” of population
•Determine appropriate scenarios for each
•Worked with designers to code
2011
2/12/2015
8
15
2012
Compliance Training
I Am Anthem Training (75-90 minutes, required for all)
Medicare Compliance
Medicaid Compliance
Fraud & Abuse
Manager course
2/12/2015
9
17
2014
2/12/2015
10
•Location
•Business AreaParticipant DemographicsParticipant DemographicsParticipant DemographicsParticipant Demographics
•Material was appropriate to my needs
•Scenarios presented helped me understand the contentCoursewareCoursewareCoursewareCourseware
•Online course was user friendly and easy to navigateOnline DeliveryOnline DeliveryOnline DeliveryOnline Delivery
•I would rate the overall training communication (pre-
course messages, screensavers, website, etc) to be:CommunicationsCommunicationsCommunicationsCommunications
•The training was effective in helping me understand the
process (where and how) to report suspected violationsLearning EffectivenessLearning EffectivenessLearning EffectivenessLearning Effectiveness
•I would be able to apply the concepts learned in this
course to the workplace ImpactImpactImpactImpact
•This course met my expectations
•What about this course was useful to you?ValueValueValueValue
Specific Course Content (60% of Most Useful
comments)
� By far, respondents found the scenarios and examples presented to be most useful. 58% of all Course Content comments were example or scenario related.
� Next most useful was the overall information presented on Fraud & Abuse, followed by information related to Refresher, such as updates and reminders regarding policies and processes.
Courseware Findings (20% of Most Useful comments)
� The response to the course online format and delivery was solidly positive. Learners who submitted comments reported that the courseware was easy to navigate, interactive, and presented the material in a fun and engaging manner.
Specific Course
Content
%
Comments
Scenarios 33%
Examples 25%
Fraud 6%
Refresher 4%
Courseware
%
Comments
Easy/User
Friendly 27%
Videos 6%
Interactive 5%
Resources 4%
2/12/2015
11
• All Good / NA / Nothing (55.3% of Improvement comments) – Over half of the comments relating to improving training were tied to “All Good / NA / Nothing to Improve” indicating that respondents felt the training could not be further improved to be more relevant to their position.
• Content (26.2% of Improvement comments) – Requests for improvements received from respondents were largely centered on customization of scenarios and examples to allow for a variety of roles across the organization.
• Courseware ( 10.2% of Improvement comments) – The vast majority of courseware improvement remarks (61.8%) involved requests for customization by business unit or job role.
• Problem (5.3% of Improvement comments) – Some respondents reported experiencing system functionality issues. Most issues reported were related to issues with screen response (froze or slow to respond).
Content % Comments
Scenario 25%
Examples 14%
Answers 6%
Definitions 5%
Relevant 5%
All Good / Apply Learning on the Job / Apply Content – Nearly half of all respondents indicated that they are currently applying or plan to apply the content, both on the job and during daily interactions with members, associates, and vendors. Those who plan on applying the content reported greater awareness when accepting gifts, sending secure emails, and general awareness of policies and procedures.
All Good / Apply on Job / Apply
Learning % Comments
Apply/Currently Apply 31%
WAH/Offsite 8%
Claims 7%
HIPAA 5%
SIU 4%
Members 4%
Gifts 4%
2/12/2015
12
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Online Delivery &
Courseware
Learning Effectiveness Ability to Apply Value
2010
2012
2014
2008 (Baseline)
BIC - Benchmark
Customized Customized Customized Customized
Curricula/Exceptional Curricula/Exceptional Curricula/Exceptional Curricula/Exceptional
ResultsResultsResultsResults
Asking for Asking for Asking for Asking for
feedbackfeedbackfeedbackfeedback
Analyzing Analyzing Analyzing Analyzing
feedbackfeedbackfeedbackfeedback
“Village”“Village”“Village”“Village”
Culture/ Culture/ Culture/ Culture/
LeadershipLeadershipLeadershipLeadership
2/12/2015
13
Contact Me: Contact Me: Contact Me: Contact Me: [email protected]@[email protected]@anthem.com