disability pricing and dental fraud detection: supervised and unsupervised learning october 11, 2007...
TRANSCRIPT
![Page 1: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/1.jpg)
Disability PricingDisability Pricingandand
Dental Fraud Detection:Dental Fraud Detection:Supervised and Unsupervised Supervised and Unsupervised
LearningLearningOctober 11, 2007October 11, 2007
Jonathan Polon FSAClaim Analytics Inc.www.claimanalytics.com
![Page 2: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/2.jpg)
• Definitions
• Dental Fraud
• Disability Pricing
AgendaAgenda
![Page 3: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/3.jpg)
DefinitionsDefinitions
![Page 4: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/4.jpg)
Supervised LearningSupervised Learning
• Known outcome associated with each record in training dataset
• Eg, Sports betting: predict winners based on observations from past games
• Objective: build a model to accurately estimate outcomes from predictor variables for each record
• Commonly referred to as predictive modeling
![Page 5: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/5.jpg)
Unsupervised LearningUnsupervised Learning
• No known outcome associated with any record in the training dataset
• Eg, Grocery stores: define types of shoppers and their preferences
• Learning objective: self-organization or clustering; finding structure in the data
![Page 6: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/6.jpg)
Dental Fraud DetectionDental Fraud DetectionUsingUsing
Unsupervised LearningUnsupervised Learning
![Page 7: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/7.jpg)
Dental Fraud Project Dental Fraud Project OverviewOverview• Explore use of pattern detection tools in
detecting dental claim anomalies
• 2004 data
• 1,600 Ontario GPs (non-specialists)
• 200,000 claims
• Many examples of anomalous activity discovered
![Page 8: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/8.jpg)
Traditional ToolsTraditional Tools
Rule-based
Strong at identifying claims that match known types of fraudulent activity
Limited to identifying what is known
Typically analyze at the level of a single claim, in isolation
![Page 9: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/9.jpg)
Pattern Detection ToolsPattern Detection Tools
• Analyze millions of claims to reveal patterns– Technology learns what is normal and what is atypical
– not limited by what we already know
• Categorize each claim. Reveal dentists with high percentage of atypical claims– Immediately highlight new questionable behaviors
– Find new large-dollar schemes
– Also identify frequent repetition of small-dollar abuses
![Page 10: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/10.jpg)
Methodology – 3 Methodology – 3 PerspectivesPerspectives1. By Claim e.g. Joe Green’s semi-annual
check-up
2. By Tooth e.g. all work done in 2004 on Joe’s bicuspid by Dr. Brown
3. By General Work e.g. all general work (exams, fluoride, radiographs, scaling and polishing) done on Joe in 2004 by Dr. Brown
![Page 11: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/11.jpg)
Methodology – 2 Methodology – 2 TechniquesTechniques1. Principal components analysis
2. Clustering
![Page 12: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/12.jpg)
PCA Cluster
Claim by claim Tooth by tooth General work
Methodology – SummaryMethodology – Summary
![Page 13: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/13.jpg)
Principal Components Principal Components AnalysisAnalysis•Visualization technique
•Allows reduction of multi-dimensional data to lower dimensions while maximizing amount of information preserved
•Powerful approach for identifying outliers
![Page 14: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/14.jpg)
PCA: Simple ExamplePCA: Simple ExampleP C A: S am p le D ata
-6
-4
-2
0
2
4
6
-6 -4 -2 0 2 4 6
X -Ax is
Y-A
xis
Lots of variance between points around both axes
![Page 15: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/15.jpg)
PCA: Simple ExamplePCA: Simple ExampleP C A: Ro ta te Axes 45 o
-6
-4
-2
0
2
4
6
-6 -4 -2 0 2 4 6
X -Ax is
Y-A
xis
Create new axes, X’ and Y’: rotate original axes 45º
![Page 16: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/16.jpg)
PCA: Simple ExamplePCA: Simple Example
In the new axes, very little variance around Y’Y’ contains little “information”
P C A: Ro ta ted Axes
-6
-4
-2
0
2
4
6
-8 -6 -4 -2 0 2 4 6 8
X '-Ax is
Y'-
Ax
is
![Page 17: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/17.jpg)
PCA: Simple ExamplePCA: Simple Example
Can set all Y’ values to 0 – ie, ignore Y’ axisResult: reduce to 1 dimension with little loss of info
P C A: Red u ce to 1 D im en s io n
-6
-4
-2
0
2
4
6
-8 -6 -4 -2 0 2 4 6 8
X '-Ax is
Y'-
Ax
is
♦ Original
● Revised
![Page 18: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/18.jpg)
PCA: Identifying Atypical PCA: Identifying Atypical DentistsDentists
![Page 19: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/19.jpg)
PCA: Identifying Atypical PCA: Identifying Atypical DentistsDentists
1. Begin at the individual transaction level
2. Determine the average transaction for each dentist
3. Graph quickly isolates dentists that are outliers
![Page 20: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/20.jpg)
Dentist Average 90th percentile
PCA: Identifying Atypical PCA: Identifying Atypical DentistsDentists
![Page 21: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/21.jpg)
PCA: SummaryPCA: Summary
1. Visualization allows for easy and intuitive identification of dentists that are atypical
2. Manual investigation required to understand why a dentist or claim is atypical
![Page 22: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/22.jpg)
Clustering TechniquesClustering Techniques
•Categorization tools
•Organize claims into several groups of similar claims
•Applicable for profiling dentists by looking at the percentage of claims in each cluster
•We apply two different clustering techniques
– K-Means
– Expectation-Maximization
![Page 23: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/23.jpg)
Clustering ExampleClustering Example
Clustering ExampleClustering Example
![Page 24: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/24.jpg)
Example: Clusters found – by Example: Clusters found – by ClaimClaim 1. Major dental problems
2. Moderate dental problems
3. Age > 28, minor work performed
4. Work includes unbundled procedures
5. Minor work and expensive technologies
6. Age < 28, minor work performed
![Page 25: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/25.jpg)
Clusters: Identifying Atypical Clusters: Identifying Atypical DentistsDentists
1. Begin at the individual claim level
2. Calculate the proportion of claims in each cluster – in total, and for each dentist
3. Isolate dentists with large deviations from average
![Page 26: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/26.jpg)
Clusters: Identifying Atypical Clusters: Identifying Atypical DentistsDentists
Proportion of General Work by ClusterK-Means Clustering
0%
20%
40%
60%
80%
100%
% C
laim
s X
Average 19% 2% 6% 20% 6% 9% 21% 17%
Dent A 0% 0% 0% 0% 0% 49% 0% 51%
1 2 3 4 5 6 7 8
![Page 27: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/27.jpg)
Clustering Techniques: Clustering Techniques: SummarySummary
1. Clustering provides an easy to understand method to profile dentists
2. Unlike PCA, clustering tells why a given dentist is considered atypical
3. Effectively identifies atypical activity at the dentist level
![Page 28: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/28.jpg)
Pilot Project ResultsPilot Project Results
• PCA identified 68 dentists that are atypical
• Clusters identified 182 dentists that are atypical
• 36 dentists are identified as atypical by both PCA and Clusters
• In total, 214 of 1,644 dentists are identified as atypical (13%)
• Billings by atypical dentists were $2.5 MM out of $16.1 MM billed by all dentists (15%)
![Page 29: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/29.jpg)
ExamplesExamplesOfOf
Atypical DentistsAtypical Dentists
![Page 30: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/30.jpg)
Proportion of General Work by ClusterK-Means Clustering
0%
20%
40%
60%
80%
100%%
Cla
ims
X
Average 19% 2% 6% 20% 6% 9% 21% 17%
Dent A 0% 0% 0% 0% 0% 49% 0% 51%
1 2 3 4 5 6 7 8
Analysis: Dentist A is far beyond norms in clusters 6 and 8; both indicate high charges in ‘general dentistry’
What we discovered: Each of Dentist A’s patients is being billed for at least 30 minutes of polishing and 45 minutes of scaling
Dentist ADentist A
![Page 31: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/31.jpg)
Dentist BDentist B
Proportion of Teeth by ClusterK-Means Clustering
0%
20%
40%
60%
80%
100%%
Cla
ims
X
Average 22% 9% 5% 13% 19% 25% 8%
Dent B 5% 20% 21% 0% 0% 0% 54%
1 2 3 4 5 6 7
Analysis : Dentist B is far beyond norms in cluster 7
What we discovered: Frequent extractions and anesthesia Dentist B looks like an oral surgeon, yet is a general practitioner
![Page 32: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/32.jpg)
Dentist CDentist C
Proportion of General Work by ClusterEM Clustering
0%
20%
40%
60%
80%
100%%
Cla
ims
X
Average 23% 77%
Dent C 66% 34%
1 2
Analysis: Very high proportion in Cluster 1, suggesting many high-ticket visits
What we discovered: First, Dentist C performs an inordinate amount of scaling. Second, Dentist C has emergency examinations with atypically high frequency
![Page 33: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/33.jpg)
Dentist MDentist M
Analysis: Dentist M has a disproportionate amount of work beyond the 90th percentile of work by all dentists on all teeth
What we discovered: Dentist M appears to utilize lab work very heavily
![Page 34: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/34.jpg)
More Atypical DentistsMore Atypical Dentists
• Frequent use of panoramic x-rays
• Frequent use of nitrous oxide – including with procedures rarely associated with anesthesia
• Large number of extractions, often using multiple types of sedation
• Very high proportion of claims for crowns and endodontic work
• Individual instruction on oral hygiene provided with very atypical frequency
![Page 35: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/35.jpg)
Dental Fraud SummaryDental Fraud Summary
• Highly effective in identifying dentists with claim portfolios significantly different from the norm
• Enables experts to quickly identify and focus on those dentists with atypical claims activity
Pattern detection:
![Page 36: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/36.jpg)
Disability PricingDisability PricingWithWith
Predictive ModelingPredictive Modeling
![Page 37: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/37.jpg)
• Protects against loss of income due to illness or injury
• Annual incidence rates ≈ 3 per 1,000
• Claim duration ranges from a few days to a few decades
About Group LTD InsuranceAbout Group LTD Insurance
![Page 38: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/38.jpg)
• Base rates: tabular, depend on age, gender, EP and max benefit duration
• Several loading factors
• Demographic info (eg, occupation, area, salary)
• Group characteristics (eg, group size, industry)
• Plan features (eg, benefit %, e’e contributions)
Typical Rate StructureTypical Rate Structure
![Page 39: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/39.jpg)
• Uncover and quantify complex relationships between pricing factors and claim experience
• Maintain wholeness of data
• Improved accuracy vs traditional methods
Why use PM for pricing?Why use PM for pricing?
![Page 40: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/40.jpg)
The ChallengePredictive models can be black boxes, difficult to interpret
The ObjectiveA process to make rate structure explainable to:
Predictive Modeling + Predictive Modeling + PricingPricing
• Management • Sales Force
• Regulators • Customers
![Page 41: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/41.jpg)
The 5 Steps -The 5 Steps -A PM Pricing ProjectA PM Pricing Project
![Page 42: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/42.jpg)
The Five StepsThe Five Steps
1.Set objective
2.Data
3.Select technique and predictors
4.Train the model
5.Validate
![Page 43: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/43.jpg)
Develop a predictive model to:
• Predict claim incidence rates
• Predict claim severity, conditional upon claim being made
for each member of census data
1. Set objective1. Set objective
![Page 44: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/44.jpg)
Required:
• Accurate census data for many groups
• Plan features for these groups (EP, Ben%)
• Characteristics of these groups (region, industry)
• Claim data that often can be linked to census
2a. Pull Data2a. Pull Data
![Page 45: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/45.jpg)
Split Data Into Three Subsets
• Training
• Testing
• Validation
2b. Sample Data2b. Sample Data
![Page 46: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/46.jpg)
• Begin exploring with simpler tools
• Learn key predictors, relationships
• Migrate to more complex tools
• Incorporate exploration learning into model design
• Use test sample to evaluate models
3. Select technique3. Select technique
![Page 47: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/47.jpg)
4. Train the Model4. Train the Model
Pricing factors
Pricing factors
Challenge: Understanding the Model
• Need to extract pricing factors from “black box”
![Page 48: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/48.jpg)
Approach:• Build several models
• Each using same modeling technique
• Each using same data
• Each excluding different predictor
• Compare actual to predicted claim cost across excluded predictor
Understanding the ModelUnderstanding the ModelTraining the Training the
ModelModelTraining the Training the
ModelModel
![Page 49: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/49.jpg)
Looking into the Black BoxLooking into the Black Box
Training the Training the ModelModel
Training the Training the ModelModel
![Page 50: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/50.jpg)
Example: model includes all predictors, consider historic claims by industry
Demonstrates accuracy of model, but provides no information about claim drivers
Industry Actual Predicted
Act/Pred
Finance $1,600 $1,600 100%
Health $1,200 $1,200 100%
Training the Training the ModelModel
Training the Training the ModelModel
![Page 51: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/51.jpg)
Example: model includes all predictors except industry, consider historic claims by industry
Isolates impact of industry from all other factors
Industry Actual Predicted
Act/Pred
Finance $1,600 $1,800 89%
Health $1,200 $1,000 120%
Training the Training the ModelModel
Training the Training the ModelModel
![Page 52: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/52.jpg)
Example: exclude two predictors: region and occ class
Region White Collar
Blue Collar
North East 115% 95%
Mid West 75% 105%
South 105% 115%
Isolates impact by region and occupation class from all other factors
Training the Training the ModelModel
Training the Training the ModelModel
![Page 53: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/53.jpg)
Benefits of Extracting Factors
1. Ability to test importance of predictors, singly or in combinations
• Test predictors that are typically not priced for
• Quantify impact of each predictor or pair of predictors
2. Facilitates use of the traditional Base Rate Approach
Training the Training the ModelModel
Training the Training the ModelModel
![Page 54: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/54.jpg)
Base Rate:
• Function of age, gender, EP, max duration
Loadings:
• For other demographic info, group characteristics, plan features
• Determine by looking inside black box
• Company can adjust final factors
Base Rate ApproachBase Rate ApproachTraining the Training the
ModelModelTraining the Training the
ModelModel
![Page 55: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/55.jpg)
Base Rate:
Base rate = 1.20
Loadings:
Base Rate ExampleBase Rate Example
Ben% Region Industry
Salary Contrib Fact X
60% NE Health $60K Y 10.2
+5% -5% +15% -5% +10% -3%
Age Gender EP Max Dur
45 Male 180 days Age 65
Training the Training the ModelModel
Training the Training the ModelModel
![Page 56: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/56.jpg)
Base Rate = 1.20
Loadings = {+5%, -5%, +15%, -5%, +10%, -3%}
Rate = 1.20 *1.05 *0.95 *1.15 *0.95 *1.10 *0.97
Final Rate = 1.395
Base Rate ExampleBase Rate ExampleTraining the Training the
ModelModelTraining the Training the
ModelModel
![Page 57: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/57.jpg)
• Use excluded data for validation of model
• Compare actual vs predicted claim costs
• Ideally, compare predictive modeling approach to traditional approach
5. Pricing Validation5. Pricing Validation
![Page 58: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/58.jpg)
• Improved accuracy vs traditional methods
• Capture effects of interaction between several predictors
• Able to isolate impact of any predictor or pair of predictors
• Able to test new predictors for impact on claim cost
• Can maintain existing Base Rate structure
Advantages of PM: Pricing
![Page 59: Disability Pricing and Dental Fraud Detection: Supervised and Unsupervised Learning October 11, 2007 Jonathan Polon FSA Claim Analytics Inc](https://reader030.vdocuments.site/reader030/viewer/2022032722/56649cee5503460f949bcb98/html5/thumbnails/59.jpg)
Questions?Questions?