How to use Quantitative Analytics to Identify and Effectively Manage Risk

Why Use Data to Manage Risk• SEC page on Distribution in Guise• FINRA guidance aimed at Broker Dealers• Your Oversight Program should do more than just accept

FICCA’s / Questionnaires

Use the data available to you to rate your distributors efficiency in their controls.

Research Findings• An oversight program can take flight on existing relationships and smartly

align itself on newly on-boarded counterparties using quantitative weighting of demographic attributes to estimate a risk tier or profile for a firm that has not worked through a periodic evaluation

• The annual or semi-annual review should leverage quantitative analytics of measured KPIs tied to specific control objectives to observe and quantify capability of control in a normalized fashion

• Once established this approach will provide the platform from which predictive analytics will provide intelligence and efficiency in the execution of the oversight program

Analytical ApproachFindings show two applicable types of quantitative analysis are in use and can be leveraged to provide insight to your counterparty

• The first will help estimate the risk that a given counterparty will likely provide and set broadly the amount of time spent in the first year on that entity

• The second will help an oversight program measure the effectiveness of the counterparty’s control objectives and therefore further qualify their risk in their profile risk ranking

Quantitative Methods EmployedTwo Primary Quantitative Methods Employed

• Profile Risk Ranking– Various key metrics and demographic elements weighted to estimate a

distribution partner’s risk

• Individual Control Objective Analytics– Individual KPIs driven from actual operational interaction with the

intermediary

Discussion ApproachMost standard Oversight Programs consist of core requirements that can be found in five pillars:

1. Counterparty Deal Management2. Product Setup/Maintenance3. Transparency Data Management4. Compliance 5. Fee / Services Management

Counterparty Deal Management Management of counterparty relationships in one single portal

Join and summarize data from all of your operational silos with

demographic and contract level data provided by your

counterparties

Information captured here will all role up to the counterparty risk

ranking

Counterparty Deal Management

Relevant Control Objectives• Third Party Oversight• Document Retention & Recordkeeping• Shareholder Communications• Business Continuity / DR

Potential Quantitative Elements• AUM• # of Accounts• Age of Relationship• Age of Last Legal Review• % growth of relationship over period• % growth of counterparty over period

Product Setup / Maintenance Fund parameter comparison rules (Load, 12b1, Fee, etc…)

Compare TA, Sub-TA and MFPII Data

Identify, resolve or retain discrepancies with comments

Product Setup / Maintenance

Relevant Control Objectives• Security Master Setup & Maintenance• Transaction Processing

Potential Quantitative Elements• Parameter Comparison Annotations

Transparency Data Management Centralized and Secure infrastructure for storing Transparency Data

Ability to perform advanced analytics: Inflow and Outflow Analysis

Market Timing Monitoring

Prospectus Compliance Analysis

Blue Sky Analysis Sales By Social Code

Initial vs. Subsequent

Transparency Data Management

Relevant Control Objectives• Transaction Processing• Blue Sky Reporting• Cash & Share Reconciliation• Lost & Missing Shareholders

Potential Quantitative Elements• Blue Sky Sales by State• Reconciliation of Account Balances• Account Monitoring• Inflows / Outflows• Measure # of Trade Breaks• # of Trade Violations associated with

rule setup

Compliance Demographic and policy level metrics that show your counterparty

has the controls and processes in place to meet standards defined

by the ICI.

Fund Companies should review these controls and policies on a

scheduled basis by conducting reviews that satisfy the board’s

request.

Compliance

Relevant Control Objectives• Risk Government• Code of Ethics• Information Security Program• Anti-Money Laundering and the Prevention of Terrorist Financing

Potential Quantitative Elements• Flags / Indicator showing certain

policies exist:– AML– KYC

• Scores / ratings from previous annual reviews

Fee and Service Management Analyze the rates you are paying distributors vs. the actual

services they are offering

Calculation and validation of invoicing

Settlement of Fee Invoicing

Leveraging transparency data for invoice reconciliation

Use fees from your TA services to baseline what you should pay

the Sub-TA platforms

Fee and Services ManagementRelevant Control Objectives• Sub account Billing & Invoice

Processing• Fee Calculations

Potential Quantitative Elements• Invoiced vs. Calculated Fee Discrepancies

• Comparison of rates across distribution partners

• Validation of Fee Amounts– Position Based– Asset Based

• Payment Funding Allocations• Tolerance Checks

– Change in accounts month over month– % fee variance vs. change in AUM

Profile Risk Ranking

Quantitative metrics from all pillars +

Qualitative demographic attributes weighted by importance

Example ApplicationProfile Attribute Values Weighting Firm 1 Score Firm 2 Score

Previous Review Score 1-100 30% 50 15 82 23

Sifi Designation Y/N 3% N 3 Y 3

Sub Accounts Y/N 15% Y 15 Y 15

AUM Tier 0-15% $4B 15 $4B 15

Trading Volume Tier 0-5% 220/Day 5 230/Day 5

International Accts Y/N 5% N 0 Y 5

Provides FICCA Y/N 12% Y 0 N 12

Servicing Firm Y/N 8% N 0 Y 8

Contract Issues Y/N 10% Y 0 N 10

53 = Tier 2 96 = Tier 4

Individual Control Objective Analytics

• Association of specific control objectives in the oversight framework with KPIs from the operational tools that support the process

• Normalization using key metrics like AUM, number of accounts or number of RIAs is key for use in the overall program analytics

ExamplesFees Paid vs. Finances Independent Calculation• A competent Fee Management process should produce a KPI that

illustrates the difference between the amount invoiced and the amount the fund company believes should have been paid

• This variance is a KPI that can be applied to the invoicing control objective in the FICCA framework to provide an “as observed” quantitative assessment of the performance of the counterparty’s controls

22c2 Market Timing Events Observed• A key KPI that is a result of the 22c2 process is the number of accounts

where market timing was identified. This metric can be tied to the control objectives for trade monitoring and market timing

NormalizationThe raw KPI results from the underlying oversight processes should be normalized for using quantitative demographic counterparty metrics. This ensures quantitative findings are kept in perspective.• In our first example, the total amount of fees paid should be used to

normalize the variance of the paid/calculated percentage• Similarly, in example two, the KPI of number of accounts where market

timing was observed in a specific time period should be normalized by dividing by the total number of sub accounts registered for the time period on the sub account platform

Predictive Analytics – the Next Step• Once your mature quantitative program has taken root,

Predictive Analytics should be leveraged to illustrate where actual risk is found

• Observed measurement tied to the quantitative oversight process will provide the quality data needed for Predictive Intelligence

Contributing Research • Direct insight from our panelists’ experience with mature,

large scale oversight programs• Observations from several major oversight programs that are

not Delta Data clients or panelists’ funds