Is it more than just smoke and mirrors?

Presented byGrant Tucker &Mike O’Rourke Ministry of Social Development

(Centre for Social Research & Evaluation

• You’ve brought the hardware

• Licensed and install the software (hopefully SAS!)

• Loaded some data onto the server(s)

So how come you can’t answer the question?

It’s all about us!

• Administer around 300,000 benefits• Pay 465,000 pensions• Grant 57,000 student allowances• Approve 143,000 student loans• Receive 46,000 notifications of possible child

abuse• Issue 1.2 million CSC cards• 50,000 Benefit investigations

• 9,000 staff

• 200 locations

• 5 Associate Ministers

• 2 Ministers

• 1 CEO

• SWIFTT• UCV

• SOLO• SAL • CYF• KEA• ABT

• BRC

• CMS

• SHAC

• WAM

• TRCE

• WIO

• SWIFTT 197 tables 3,420 fields

• UCV 107 tables 999 fields

• SOLO 214 tables 2,030 fields

• SAL 229 tables 3,202 fields

• CYF 218 tables 3,082 fields

• KEA 97 tables 2,290 fields

• BRC 15 tables 188 fields

• CMS 126 tables 1,868 fields

• SHAC 202 tables 3,317 fields

• WAM 105 tables 993 fields

• TRCE 115 tables 1,830

• WIO 10 tables 54 fields

• ABT 2 tables 30 fields

• Overall 11 Billion records

• In over 2,200 tables

• With over 38,000 fields

So how do YOU make sense of all this?

• What do you do when that urgent question comes in ?

• “How many left handed Lithuanians with a lisp are there in Lyttleton?”

• PANIC!!

• Are you tired of running around in a panic?

• Want to avoid those tense meetings with the CEO to explain why the numbers have changed?

• Are you sick of it being a new “adventure” to reproduce earlier numbers?

• We were! So …..going back…way back..

Back to 1995

• Food poisoned All Blacks lose to SA in Rugby World Cup final

• DSW decide to invest in a data warehouse

• DSW starts business rule process

IAP Data Warehouse Implementation

• Replication of operational systems tables

• Tracks and date-timestamps every change

• Raw feed – no cleansing – warts & all

• Ability to recreate data at any point in time since replication started

• SWIFTT since July 1996

• Currently 18 systems are replicated

Business Rules

The building blocks for turning data into information

What is a business rule?

What is a business rule?

• An agreed definition to describe and quantify a business count or measure

• The “thing” being counted should be pertinent and countable

Why have Business Rules?

Why have Business Rules?

• Consistency

• Transparency

• Productivity

• Business Continuity

Consistency

• One Version of the truth

• All analysts get the same result

• Auditors and forecasters love consistency

Transparency

• Reflects what the operational system(s) show

• Ability to drill down to the unit records

• Auditable

Productivity

• Training new staff• Training existing staff moving into new areas• Frees up experts from explaining and QA• Build programs from the code rule blocks

Continuity

• Documenting ‘Intellectual Property’ often held in personal programs or people’s heads

• Covers for staff turnover – they move on but the rules remain

Business Rule Group

• Initially reps from Income Support & SPA

• Dedicated co-ordinator

Business Rules Process

• Meet regularly every fortnight

• Prioritise rules to be developed

• Delegate work item

• Prepare draft rules

• Review draft rules

• Arrange signoff of finalised rules

What’s produced

• Plain English description of what the rule produces

• Any “Gotchas” or health warnings• Technical description of how the rule is

built• SAS code version of the rule ready to roll• Available from a web based Word

document

For Example

CHILDREN INCLUDED IN A BENEFIT - ReplicaModule NamePRNTCHDOBrief DescriptionDetermine if children included in benefit from Replica tables.Business RulesInclude all current recipients (except NZ Super) based on status of benefit. Due to the fact that NZ Superannuitants do not get paid more if they have dependant children, children details are not usually recorded unless a supplementary like Disability Allowance is paid.Select relationship records from RELAT. Find the latest relationship record for each child and check indicators and dates to see if the child is included in the benefit at the time of the extract.The Replica date to be used must be as at the day after the end of the period being reported on. Notes: 1. Replica tables post 8 March 1998 are created as at the end of SWIFTT processing and prior to the evening’s batch processing (before expiries) each day. Tables created prior to 9 March 1998 were not extracted at a particular point in the processing cycle.

Code RulesRead SBEN to extract details of all current records for selected service codes.

Use SERV to select appropriate service codes.Use SRVST = ‘3’ (Current) and SRVST = ‘4’ with RSNCT = ‘071’ (Paid by Director)

to select current records.Sort extracted sben records by SWN, SERV.Read RELAT to extract latest relationship records for a child and check to ensure child included at time of table.

Use RELTYP = ‘1’ (Relationship between caregiver and child).Use FROMDT to ensure relationship details apply on or before date of table.

Sort extracted relat records by SWN, SERV.Merge SBEN and RELAT records by SWN and SERVSort merged sben and relat records by SWN, SERV,CHSWN,INPDT, FROMDT.Identify current children.

Find latest record for a child and ensure:MAINTS not = ‘I’ (Inactive, child excluded) andFROMDT = or less than table date andTODT = . or TODT greater than or equal to the table date.

Define the date for the Replica extract.

SAS Code*************************************INPUTS*****************************************;%let dd = 15 ; *Change these for day ;%let mmm = JUN ; * month ;%let yy = 01 ; * year of Replica extract ;* Updated: 25 May 1999 - Check FROMDT not = . included in RELAT selection ;* Updated: 7 June 2001 - DPB check in RELAT to ensure CHIND = 'Y' removed ;************************************************************************************;*********************prntchdo-Children included ************************************;data sben (drop = repdate filedate maints); set SWF&ORG..sben (keep = swn serv aport srtet srvst repdate filedate maints where = (%extora(&dd&mmm&yy)

and (srvst = '3' or (srvst = '4' and rsncd = '071')) and (serv in ('020','320'))));run;************************* sort for merge with Relat ********************************; proc sort data = sben nodupkey; by swn serv;run;***************************** Reconstitute Relat ***********************************;data relat (drop = repdate filedate ); set SWF&ORG..relat (keep = reltyp fromdt todt swn serv chswn inpdt chind repdate filedate maints where = (%extora(&dd&mmm&yy) and (serv ne '060') and reltyp = '1' and (fromdt not = . and fromdt <= "&dd&mmm&yy"d) and maints ne 'I' and (todt =. or todt >= "&dd&mmm&yy"d)));run;.....

Issues faced

• Poor system documentation

• Lack of inhouse systems experts due to out sourced system development & turnover

• Other work load limited people’s ability to progress Business Rules work

• Difficulty reconciling to operational system reports

Addresses the issues

• Access to case dictionary information• Get Corporate DBAs to focus on

completeness & accuracy of documentation • Build relationships with Business Analysts• Build relationships with expert users• Dedicated Business Rules team• Access to operational systems for cross

checking

Addressing the issues

• Leverage on transparency of IAP replica

• If you want valuable information captured, make it valuable to those who capture it

• KPIs

• Feedback the information to staff to use in their day to day work

Lessons learnt

• Requires organisational commitment• It takes time and effort• Need right level of representation in the

Business Rule group• Chip away at the individual building blocks

that make up a measure• Acknowledge that they’re only as good as

the data in the operational systems

But once you’ve got the Building Blocks

Build!!

And Build

And build

And Build

And Build

The End

Any Questions?