mdm smart search configuration - informatica search enablement.pdf · smart search provides the...

of 29

MDM Smart Search Configuration

DOCUMENT AUTHOR: Pawan Kumar DOCUMENT OWNER: Pawan Kumar

DATE CREATED: May 19, 2016 LAST UPDATED: June 6 , 2016

PROJECT: Producer/Broker Domain COMPANY: American Family Life Assurance Company

of 29

Document Reviewer(s) Name and Title Signature Date Adam Bartolomeo, Informatica Solution Architect

Document Approver(s) Approver and Title Signature Date

of 29

1. REFERENCE DOCUMENTS .......................................................................................................... 5

2. SMART SEARCH OVERVIEW ...................................................................................................... 5

3. WEBSPHERE PRE-CONFIGURATION ....................................................................................... 5

The following sections list WebSphere settings that must be set prior to beginning configuring Smart Search in MDM. .................................................................................................................................................................... 5

3.1 Registering Apache Solr MBeans ...................................................................................................................... 5

3.2 JVM generic arguments .................................................................................................................................... 6

4. CONFIGURING SMART SEARCH ................................................................................................ 6

4.1 Updating cmxcleanse.properties file ................................................................................................................ 7

4.2 Updating cmxserver.properties file .................................................................................................................. 8

4.3 Creating Zookeeper directory........................................................................................................................... 9

4.4 Configuring Process Server ............................................................................................................................. 10

4.5 Restart WebSphere..................................................................................................................................... 11

4.6 Checking solr admin console ....................................................................................................................... 11

5. RESETTING SMART SEARCH ................................................................................................... 12

5.1 Cleaning C_REPOS_CO_CS_CONFIG Metadata table ...................................................................................... 13

5.2 Re-generating Business Entity Schema ........................................................................................................... 14

5.3 cmxcleanse.properties ................................................................................................................................... 17

5.4 cmxserver.properties ..................................................................................................................................... 18

5.5 Stopping WAS service .................................................................................................................................... 19

5.6 Cleaning solr/zkp folders in MDM .............................................................................................................. 20

5.7 Stating WAS ................................................................................................................................................... 22

5.8 Checking solr admin console. ......................................................................................................................... 22

5.9 Running Index Job in MDM Hub ..................................................................................................................... 23

6. TURNING OFF SMART SEARCH ............................................................................................... 26

of 29

The following sections list Steps to turn off Smart Search if needed. ................................................................... 26

of 29

1. REFERENCE DOCUMENTS

2. SMART SEARCH OVERVIEW Smart Search provides the ability to perform searching within a specific searchable business entity type or within all the searchable business entity types based on a string value. The Informatica Data Director application displays the matching searchable fields. The results are ranked based on how closely the results match the string value. Filters can be applied to identify more relevant results. Smart Search uses solr as its search platform, solr is based on Apache Lucene. In order to use smart search, configurations on the process server and running jobs for indexing the data must be done. The following sections outline these steps.

3. WEBSPHERE PRE-CONFIGURATION

The following sections list WebSphere settings that must be set prior to beginning configuring Smart Search in MDM.

3.1 Registering Apache Solr MBeans To support smart search in Informatica Data Director, register Apache Solr MBeans. In the WebSphere Console, check if following property is set as mentioned below in the custom properties of the Java Virtual Machine in WebSphere. javax.management.builder.initial=javax.management.MBeanServerBuilder

Document Description MDM_1010_ProvisioningToolGuide_en.pdf MDM Smart Search Guide

of 29

3.2 JVM generic arguments The solr target directory needs to be added to the JVM generic arguments -Dsolr.solr.home=<INFAMDM root>\hub\cleanse\solr

For example: D:\infamdm\hub\cleanse\solr

The solr folder gets automatically created in the appropriate target when smart search is enabled.

4. CONFIGURING SMART SEARCH This section describes the steps to be followed for Smart Search Index configuration.

of 29

4.1 Updating cmxcleanse.properties file This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 1, #8.

1. On the MDM Application server, navigate to the MDM Installation directory, D:\infamdm\hub\cleanse\resources .

2. Using Notepad, open cmxcleanse.properties

3. Update the following existing Property values: • cmx.ss.enabled

Update the value from false to true • zookeeper.tickTime

Update the value from 2000 to 20000 • facet.infacount

Update the value from 1000 to 5000

4. Add the following new properties: • solr.allowAdminConsole=true • MAX_INITIAL_RESULT_SIZE_TO_CONSIDER =130

of 29

5. Save the changes and close the file

4.2 Updating cmxserver.properties file This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 3, #2

1. On the MDM Application server, navigate to the MDM Installation directory, D:\infamdm\hub\cleanse\resources

2. Using Notepad, open cmxserver.properties.

3. Update the following existing Property values: • cmx.ss.enabled

of 29

Update the value from false to true

4. Add the following new properties: • cmx.server.batch.smartsearch.initial.block_size =2000

4.3 Creating Zookeeper directory Create a folder for Zookeeper in Cleanse folder to store the in-memory snapshots of the data and the transactional log of updates to the database.

1. On the MDM application server, navigate to the cleanse folder: <MDM root>\hub\cleanse For example: D:\infamdm\hub\cleanse

2. Create new folder, zkp.

of 29

Note: Please make sure that the name of the directory is not too long (greater than 7 letters) and doesn’t contain underscore/hyphens in its name. 4.4 Configuring Process Server This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 1, #4

1. In MDM Hub console acquire write lock. 2. Navigate to the Process Server workbench 3. Click on Edit for the Process Server 4. The following properties in Process Server must be enabled:

Enable Search Processing Enable ZooKeeper

5. When ZooKeeper is enabled, continue configuring the Process Server: ZooKeeper ID: 1 ZooKeeper Client Port: 2181 ZooKeeper Leader Port: 0

of 29

ZooKeeper Follower Port : 0 ZooKeeper Directory : D:\infamdm\hub\cleanse\zkp ( created in step 4.3)

Note: Make sure that port assigned for ZooKeeper client port (2181) is open on server. This can be checked by netstat or telnet command by server admin.

4.5 Restart WebSphere Restart WebSphere to affect all changes for Smart Search.

4.6 Checking solr admin console To log into the solr admin console, go to URL : http://<MDMHOST:PORT>/MDESolr/#/~cloud http://dcamdm02:9080/MDESolr/#/~cloud

Navigate to cloud graph and you will see all subject area shard folder and its status. This takes some times to get created after WebSphere restart.

http://dcamdm02:9080/MDESolr/#/~cloud

of 29

Once the all shard folder are created, it can be validate at below location on server: D:\infamdm\hub\cleanse\solr

5. RESETTING SMART SEARCH It is best practice to disable smart search while running large data volumes in MDM, such as IDL.Also enabling Smart Search is inefficient prior to the completion of the IDL, as thousands of records will merge away and not need to be indexed for Smart Search

of 29

While configuring Smart Search at Aflac, several issues were resolved by the IPS team and some required further assistance with Informatica Global Customer Services (GCS). A support ticket was opened, Informatica Support Case number 01915350 for Smart Search.

5.1 Cleaning C_REPOS_CO_CS_CONFIG Metadata table

1. Log into the CMX_ORS database 2. Check C_REPOS_CO_CS_CONFIG metadata table

3. If this table contains data, truncate it.

4. Confirm the table is empty.

of 29

5.2 Re-generating Business Entity Schema In order to re-populate c_repos_co_cs_config table, we need to re-generate business entity schema from IDD config screen.

1. Login to IDD config screen , the URL will be something like http://dcamdm02:9080/config/config_login.jsf

2. Select the IDD application, Alfac IDD App,

of 29

3. Open the Application State menu and select “Not Deployed”

4. Once selected, the application will get un-deployed and deployment status will say as “Not Deployed”.

5. Click “Generate Business entity Schema “

of 29

6. After generateingBES. Select Close in the Informatica MDM Data Director pop-up window.

7. In the CMX_ORS database, confirm C_REPOS_CO_CS_CONFIG table has been repopulated

8. In the IDD Configuration URL, select Application State Full Deployment

of 29

9. Once the Aflac IDD App is successfully deployed, the status will be “Full Deployment” and status will be valid (green check mark)

5.3 cmxcleanse.properties

1. Log into MDM application server 2. Go to <MDM INSTALL>/hub/cleanse/resources folder ((D:\infamdm\hub\cleanse\resources) 3. Using Notepad, open the cmxcleanse.properties file

of 29

4. Check if smart search is enabled

5.4 cmxserver.properties

1. Log into the MDM application server 2. Go to <MDM INSTALL>/hub/server/resources folder (D:\infamdm\hub\server\resources) 3. Using Notepad, open cmxserver.properties

of 29

4. Make sure that Smart Search properties are enabled (true):

5.5 Stopping WAS service

1. Log into the MDM application server 2. Open Control Panel > Services 3. Stop WAS service. DO NOT click Start, need to remove files from Smart Search folders

of 29

5.6 Cleaning solr/zkp folders in MDM

1. On the MDM application server, go to D:/infamdm/hub/cleanse folder and take backup of the solr & zkp folders

2. Once the files are moved, , clean both of these folders (if nothing present in these folder , skip this process) Before cleaning the solrfolder:

of 29

After cleaning all files are deleted from the solr folder:

Before cleaning the zkp folder:

of 29

After cleaning zkp folder, all files are deleted:

5.7 Stating WAS 1. On the MDM application server, navigate to Control Panels > Services 2. Start WAS service

This would take some time to re-start WAS and MDM hub. 5.8 Checking solr admin console.

1. Log into the solr admin console, go to URL : <MDMHOST:PORT>/MDESolr/#/~cloud 2. Navigate to cloud graph and you will see all subject area folder status:

This shows that solr directory is connected to zookeeper and will get populated with shards. Once the all shard folder are created, it can be validate at below location on server: D:\infamdm\hub\cleanse\solr

of 29

5.9 Running Index Job in MDM Hub

1. Log into MDM Hub Console 2. Go to Batch Viewer workbench

3. Connect to cmx_ors database

of 29

4. Go to Batch Viewer Party Initially Index Smart Search data 5. Select Initial Index data for Search data Party Writing

6. To execute this job, select on “Initial Index data for Search data Party” and click on “Execute Batch”

of 29

7. When executing, the status the current status will displays as “Processing”. Keep refreshing for current status by clicking on “Refresh Status”

8. When successfully finished, it will show below details:

9. Similarly run Index job for Party Writing number:

10. Go to Batch Viewer Party Writing Number Initially Index Smart Search data 11. Select Initial Index data for Search data Party Writing Number

12. To execute this job click on “Execute Batch”

of 29

13. When executing, the status the current status will displays as “Processing”. Keep refreshing for current status by clicking on “Refresh Status”

14. When successfully finished, it will show below details:

15. This completes the Smart Search configuration. After enabling smart search, login to UI and test smart search.

6. TURNING OFF SMART SEARCH The following sections list Steps to turn off Smart Search if needed.

1. On the MDM Application server, navigate to the MDM Installation directory, D:\infamdm\hub\cleanse\resources

of 29

2. Open this cmxcleanse.properties file and make sure that the below properties is set as:

cmx.ss.enabled=false

3. On the MDM Application server, navigate to the MDM Installation directory, D:\infamdm\hub\server\resources

of 29

4. Open this cmxserver.properties file and make sure that the below properties is set as: cmx.ss.enabled=false

5. Disable ZooKeeper in MDM Hub :

of 29

6. Restart WebSphere which will disable Smart Search.

mdm smart search configuration - informatica search enablement.pdf · smart search provides the...

Documents