Using the Data Masking Pack with Oracle 12c Cloud Control

In this example we will see how to mask sensitive data in a table using the Data Masking Pack which is included (as a separate licensed option) in Oracle 12c Cloud Control.

We create an Application Data Model first where we define which columns are considered sensitive and are candidates for data masking and then we create data masking policies or rules which instructs Oracle how to mask or scrub the data .

We can also use masking formats which are already supplied and ready to use out-of-the-box or we can create our own masking formats which can be then stored in a masking format library for future use.

Let us take the EMP table as an example.

We have cloned the table from the production database and in our test or development database we want to mask or hide any data which we consider to be confidential or sensitive from the development team or the user testing team for example.

Our data masking requirements are this:

1) Shuffle data in the EMP table and group it on the JOB column. So when someone selects a record for a particular employee belonging to the job category say SALESMAN, the data is masked and rows belonging to some other random employee but belonging to the same job category SALESMAN is returned instead

2) Hide the day and month the employee joined the company but retain the year value as the application requires the original year value and not some fictitious value

3) The salary for the job category PRESIDENT should not be revealed

Note that data masking will replace data unlike the Data Redaction feature in the 12c database where the data which is displayed or returned by a query is changed on the fly.

So we create for this exercise a table called EMP_MASK which is a copy of the EMP table owned by SCOTT.

This is the data in the table before the data masking:

SQL> select * from emp_mask;

EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO---------- ---------- --------- ---------- --------- ---------- ---------- ---------- 7369 SMITH CLERK 7902 17-DEC-80 800 20 7499 ALLEN SALESMAN 7698 20-FEB-81 1600 300 30 7521 WARD SALESMAN 7698 22-FEB-81 1250 500 30 7566 JONES MANAGER 7839 02-APR-81 2975 20 7654 MARTIN SALESMAN 7698 28-SEP-81 1250 1400 30 7698 BLAKE MANAGER 7839 01-MAY-81 2850 30 7782 CLARK MANAGER 7839 09-JUN-81 2450 10 7788 SCOTT ANALYST 7566 19-APR-87 3000 20 7839 KING PRESIDENT 17-NOV-81 5000 10 7844 TURNER SALESMAN 7698 08-SEP-81 1500 0 30 7876 ADAMS CLERK 7788 23-MAY-87 1100 20 7900 JAMES CLERK 7698 03-DEC-81 950 30 7902 FORD ANALYST 7566 03-DEC-81 3000 20 7934 MILLER CLERK 7782 23-JAN-82 1300 10

14 rows selected.

After the data masking job has been run, we can see that the table data has changed according to the data masking policies which we had defined.

SQL> select * from emp_mask;

EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO---------- ---------- --------- ---------- --------- ---------- ---------- ---------- 7844 WARD SALESMAN 7698 02-AUG-81 1250 500 30 7369 MILLER CLERK 7782 29-MAY-82 1300 10 7934 JAMES CLERK 7698 27-JAN-81 950 30 7788 FORD ANALYST 7566 18-DEC-81 3000 20 7521 ALLEN SALESMAN 7698 01-APR-81 1600 300 30 7654 TURNER SALESMAN 7698 25-NOV-81 1500 0 30 7839 KING PRESIDENT 10-MAY-81 10 7698 BLAKE MANAGER 7839 02-AUG-81 2850 30 7499 MARTIN SALESMAN 7698 29-MAY-81 1250 1400 30 7902 SCOTT ANALYST 7566 27-JAN-87 3000 20 7876 SMITH CLERK 7902 01-AUG-80 800 20 7566 JONES MANAGER 7839 29-MAY-81 2975 20 7782 CLARK MANAGER 7839 27-JAN-81 2450 10 7900 ADAMS CLERK 7788 02-AUG-87 1100 20

14 rows selected.

The SAL column for KING who is the PRESIDENT has a null value.

The day and mont h for the HIREDATE column has been changed to a random value while retaining the year.

In the pre-masked table, EMPNO 7844 had these values

EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO---------- ---------- --------- ---------- --------- ---------- ---------- ---------- 7844 TURNER SALESMAN 7698 08-SEP-81 1500 0 30

In the post-masked table we see that the data for the row with 7844 EMPNO has been shuffled with the original row which had the EMPNO 7521 as both these rows belonged to the job category SALESMAN

EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO---------- ---------- --------- ---------- --------- ---------- ---------- ---------- 7844 WARD SALESMAN 7698 02-AUG-81 1250 500 30

Note:

The following permissions are required for Data Masking.

EM_ALL_OPERATOR for Enterprise Manager Cloud Control users SELECT_CATALOG_ROLE for database users SELECT ANY DICTIONARY privilege for database users EXECUTE privileges for the DBMS_CRYPTO package

Let us take a look at the steps involved.

Create the test tables

SQL> create table emp_mask as select * from emp;

Table created.

SQL> create table dept_mask as select * from dept;

Table created.

SQL> alter table emp_mask add constraint pk_emp_mask primary key (empno);

Table altered.

SQL> alter table dept_mask add constraint pk_dept_mask primary key (deptno);

Table altered.

SQL> alter table emp_mask add constraint fk_emp_mask foreign key (deptno) 2 references dept_mask (deptno);

Table altered.

Connected.

Create the Application Data Model

An Application Data Model (ADM) is a knowledge base within Enterprise Manager that captures and stores application metadata, such as referential relationships and sensitive columns of an application.

From Enterprise menu > Quality Management > Data Discovery and Modelling

Click on Create

Select the schema SCOTT to be part of this application data model

Click on View Job Details

Highlight the TEST_DATA_MASK_ADM application data model and click on Edit

We can observe how the primary key /foreign key relationships between the DEPT_MASK and EMP_MASK tables has been captured and is now stored in the Application Data Model

Click on the Sensitive Columns tab and then click on Add

Enter the application name and table name and click on the Search button

It will display all the columns in the EMP_MASK table.

We select the HIREDATE, MGR,EMPNO and SAL columns of the table as the Sensitive columns

Click on Save and Return

We have now created the Application Data Model and identified which are the sensitive columns.

We can now create the Data Masking definitions as the next step.

Create the Data Masking Definitions

We provide a name for the Data Masking definition and select the appropriate Application Data Model.

The Reference Database field is automatically populated.

Click on Add

Enter SCOTT for Schema and EMP_MASK for Table Name and click on the Search button

Note: The columns that are displayed here are only the columns which we have identified as sensitive in the application data model we created earlier.

Select EMPNO and click on Define Format And Add

Select Shuffle from the Format Entry drop-down list and click on the Add button

In the Grouping Columns select Job

Click OK

Click on the Add button

Enter the same Schema Name and Table Name and click on the Search button

Select the HIREDATE column and click on the Define Format And Add button

In the Format Entry drop-down list select SQL Expression and click on the Add button

In the SQL Expression box enter :

trunc(%HIREDATE%,'YEAR') + trunc(dbms_random.value(1,364)

Click on OK button

Click on Add button

Select the SAL Column Name and click on Define Format And Add

Click on Add Condition button

In the Conditions text box enter :

JOB='PRESIDENT'

For Format Entry select Null Value

Select Default Condition and from the Format Entry drop-down select Preserve Original Data and click on Add

Click OK

If we expand the Advanced Options, we can see that we can also perform a number of pre as well as post masking tasks like refresh the optimizer statistics after the masking as data as well as data distribution would have changed because of the data masking job and we can also run some custom SQL scripts as well to perform other action like for example delete or truncate some other tables after data masking completes.

We now have to generate the data masking script.

Click on the Generate Script button.

Once the data masking script generation job has completed, we next can either schedule a job which executes the data masking script at a later time or run it immediately.

In this case we see that the data masking SQL script has been created in the $ORACLE_HOME/dbs directory location – this can be changed if required

SQL> select * from emp_mask;

EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO---------- ---------- --------- ---------- --------- ---------- ---------- ---------- 7369 SMITH CLERK 7902 17-DEC-80 800 20 7499 ALLEN SALESMAN 7698 20-FEB-81 1600 300 30 7521 WARD SALESMAN 7698 22-FEB-81 1250 500 30 7566 JONES MANAGER 7839 02-APR-81 2975 20 7654 MARTIN SALESMAN 7698 28-SEP-81 1250 1400 30 7698 BLAKE MANAGER 7839 01-MAY-81 2850 30 7782 CLARK MANAGER 7839 09-JUN-81 2450 10 7788 SCOTT ANALYST 7566 19-APR-87 3000 20

7839 KING PRESIDENT 17-NOV-81 5000 10 7844 TURNER SALESMAN 7698 08-SEP-81 1500 0 30 7876 ADAMS CLERK 7788 23-MAY-87 1100 20 7900 JAMES CLERK 7698 03-DEC-81 950 30 7902 FORD ANALYST 7566 03-DEC-81 3000 20 7934 MILLER CLERK 7782 23-JAN-82 1300 10

14 rows selected.

SQL> select * from emp_mask;

EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO---------- ---------- --------- ---------- --------- ---------- ---------- ---------- 7844 WARD SALESMAN 7698 02-AUG-81 1250 500 30 7369 MILLER CLERK 7782 29-MAY-82 1300 10 7934 JAMES CLERK 7698 27-JAN-81 950 30 7788 FORD ANALYST 7566 18-DEC-81 3000 20 7521 ALLEN SALESMAN 7698 01-APR-81 1600 300 30 7654 TURNER SALESMAN 7698 25-NOV-81 1500 0 30 7839 KING PRESIDENT 10-MAY-81 10 7698 BLAKE MANAGER 7839 02-AUG-81 2850 30 7499 MARTIN SALESMAN 7698 29-MAY-81 1250 1400 30 7902 SCOTT ANALYST 7566 27-JAN-87 3000 20 7876 SMITH CLERK 7902 01-AUG-80 800 20 7566 JONES MANAGER 7839 29-MAY-81 2975 20 7782 CLARK MANAGER 7839 27-JAN-81 2450 10 7900 ADAMS CLERK 7788 02-AUG-87 1100 20

14 rows selected.

SQL>