migrating attachments 2014

1 © Copyright 2011 EMC Corporation. All rights reserved.

RSA Archer Free Friday Tech Huddle

Craig Lee Manager, RSA Technical Support November 2014

Contact Information Email: [email protected]

mailto:[email protected]


Migrating Attachments From One Environment

or Application to Another Leveraging Data

Feed Manager


Training Agenda

• When Would I Need To Leverage This Functionality? • What Do I Need To Perform This Process? • Two Different Approaches With DFM • Archer to Archer Approach • File Transporter Approach • Common “Gotchas” • Demo • Questions?


When Would I Need To Leverage This Functionality?

If you:

• Have a need to replicate the record content for a particular application, including attachments, in a separate environment where utilizing a backup/restore is not possible

• Have a need to split an application out into multiple apps where you will be separating attachment fields

• Inadvertently deleted an attachment field and need to restore the missing attachments


What Do I Need To Perform This Process?

To perform a migration/restore of attachment files, you will need the following items: -Access to perform data feeds in the environment -If filenames have not been converted from file_id format, you will need access to the instance database in SQL to generate a batch copy/rename script -Source file listing a unique identifier for your record content along with the name of the attachment to be linked -Service account that has access to the directory that you have copied the attachment files to


Two Different Approaches With DFM

1. Archer to Archer data feed 2. File Transporter data feed


Archer to Archer Approach

This approach allows for the use of the native A2A functionality and the default transform. Using this approach you would simply need to create a report (or search XML) against the source application where the attachments are currently housed. You will want to have some sort of key identified if you will be inserting attachments into existing content in the target environment/application. Using an Archer to Archer feed will alleviate the need to interact directly with the database as the functionality exists to perform the file retrieval and rename back to the original text value simultaneously through the feed process. This is the optimal approach assuming connection between the two applications/environments is not an issue.


Archer to Archer Approach Items needed for migrating using an Archer to Archer feed: - A report against the target application that contains a minimum of the attachment

field(s), preferably also a key field if the intention is to update existing content - The report ID or GUID for referencing within the feed - A user account that has permissions to the report and records with attachments to

be migrated ***a user with app owner access will ensure permissions across the board***


Archer to Archer Approach Steps to implement the process: 1. Create a user that will have access to the application/report/records you wish to

migrate 2. Create a report against the target application that contains the fields you wish to

migrate over

- Copy the report GUID


Archer to Archer Approach Steps to implement the process: 3. In the target environment, create a new data feed with the following information: a) Select the target application (General>Target) b) Select Archer Web Services Transporter (Transport>Transport Method) c) Insert the URL to the source environment (Transport>URL) d) Select Report ID (Transport>Search Type) e) Insert the report GUID from step 2(Transport>Report) f) Insert the username and password for the account created in step 1 (Transport>User Name

and Transport>Password) g) Insert the instance name for the source environment (Transport>Instance) h) Insert the default transform on the Navigation tab

i. Click Navigation tab ii. Click Load Transform iii. With the Default radio button selected, click OK

i) Load the source fields i. Click Source Definition tab ii. Click Load Fields iii. With Execute Search radio button selected, click OK


Archer to Archer Approach Steps to implement the process: j) Map fields on the Data Map tab

i. Drag and drop your source fields to the appropriate field in the target application

ii. If you are going to be potentially copying multiple attachments into a single records attachment field, ensure the “Append” option is enabled on the target attachment field

iii. Select a key field if you wish to identify one. If you do not, it will be assumed that new records are always going to be created

iv. Choose your update/archive settings k) Click on the Schedule tab and run the feed


File Transporter Approach

The file transporter approach will still leverage the data feed manager, however there will be a need to build a source file rather than leveraging a built-in report. For the purposes of migrating attachments, we will leverage a SQL script to assist us in building the source file and the batch file required to convert file IDs back to file names.



1. The first step we must take is to generate our source file and a batch file to copy our attachments out of the repository and rename them back to their original text names (Archer stores the files in the database as a numeric.extension format ie 123.pdf)

2. A SQL script can be leveraged to assist in both the file creation and the syntax for the batch file:

/************************************** --Script Name Copy5xAttachments.sql --Author CL --Date 20141121 --Purpose Assist in the generation of a source file and batch file syntax for use in the migration of attachments --Versions Supported Archer 5.4 - 5.5 SP1 --Instructions Provide information on the file repository path, the target path, Application name, and attachment field name. Then run this script. --WARNINGS This script is not making updates to live data. It is performing only select statements against permanent tables. **************************************/


File Transporter Approach DECLARE @SourceDirectory VARCHAR(MAX), @DestinationDirectory VARCHAR(MAX), @moduleName varchar(MAX), @fieldName varchar(max), @fieldID int SELECT @SourceDirectory = 'C:\Program Files\RSA Archer\Repository', --insert source repository path @DestinationDirectory = 'C:\Attachments', --insert destination directory to store temp copies @moduleName='File Transporter Source', --insert application name @fieldName='Attachment' --insert field name SET @fieldID=( select fd.field_id from tblivfielddef fd join tbllevel l on fd.level_id=l.level_id join tblmodule m on m.module_id=l.module_id join tblFieldTranslation ft on fd.field_id=ft.field_id where m.module_name=@moduleName and ft.field_name=@fieldName ) IF OBJECT_ID('tempdb..#content') IS NOT NULL BEGIN DROP TABLE #content END IF SUBSTRING(REVERSE(@DestinationDirectory), 1, 1) <> '\' BEGIN SELECT @DestinationDirectory = @DestinationDirectory + '\' END CREATE TABLE #content ( [content_id] int, [filename] VARCHAR(MAX) , copy_cmd VARCHAR(MAX) , [file_id] INT , is_dup_file_name BIT DEFAULT ( 0 ) )


File Transporter Approach INSERT INTO #content ( content_id, filename , copy_cmd , file_id ) SELECT c.content_id [content_id], rf.file_name [filename], 'COPY "' + @SourceDirectory + '\' + CAST(rf.[file_id]as varchar(max)) + '.' + + SUBSTRING(rf.file_name, LEN(rf.file_name)-(CHARINDEX('.', reverse(rf.file_name))-2), 8000) + '" "' + @DestinationDirectory, rf.[file_id] from tblRepositoryFile rf join tblXIVContentRepositoryLinks crl on rf.file_id=crl.file_id join tblIVContent c on crl.content_id=c.content_id join tblModule m on m.module_id=c.module_id where m.module_name=@moduleName and crl.field_id=@fieldID UPDATE c SET c.is_dup_file_name = 1 FROM #content c JOIN ( SELECT c.filename FROM #content c GROUP BY c.filename HAVING COUNT(*) > 1 ) a ON c.filename = a.filename UPDATE c SET c.[filename] = SUBSTRING(filename, 1, LEN(filename) - PATINDEX('%.%', REVERSE(c.filename))) + '-' + CAST(c.file_id AS VARCHAR(20)) + REVERSE(SUBSTRING(REVERSE(c.filename), 1, PATINDEX('%.%', REVERSE(c.filename)))) FROM #content c WHERE c.is_dup_file_name = 1 UPDATE c SET copy_cmd = copy_cmd + c.[filename] + '"' FROM #content c SELECT c.content_id as 'Old Tracking ID', c.filename as 'Attachment', c.copy_cmd FROM #content c ORDER BY c.filename



3. This script will generate 3 columns worth of results: a) The first two columns will be leveraged to build the source file b) The third column will be leveraged to build the batch file for copying

4. Copy columns 1 and 2 and paste them into an Excel spreadsheet(ensure you copy the headers

5. Save the Excel as a .csv 6. Copy column 3 and paste it into a notepad file and save it with a .bat

extension 7. Place the file created in step 5 in a location that your job engine can get

to. This will be used as the source of the feed 8. Run the batch file from step 6 to create your attachment copies 9. Create a file transporter data feed:

a) Select the target application (General>Target) b) Select File Transporter (Transport>Transport Method) c) Populate the path to the source file that you set up in step 7


File Transporter Approach d) Leave the defaults on the Navigation tab e) Load the source fields

i. Click Load Fields ii. Browse to the source file iii. Click Open

f) Map fields on the Data Map tab i. Drag and drop your source fields into the appropriate field in the target

application ii. If you are going to be potentially copying multiple attachments into a single

records attachment field, ensure the “Append” option is enabled on the target application field

iii. Select a key field if you wish to identify one. If you do not, it will be assumed that new records are always going to be created

iv. Choose your update/archive settings g) Click on the Schedule tab and run the feed h) Assuming the plan is to drop the attachment copies into the data feed folder in the

repository, the first run will go to a warning state. This is expected as we have not moved the attachments into position yet

i) Run the batch file from step 6 to create the new copies. Move those copies into the folder in the repository that was created on feed run in step g ({Repository path}\datafeeds\{Feed name}


File Transporter Approach j) Now that the attachments are placed within the source directory of the feed, run it again k) This time the attachments will be found and linked to the record content


Common “Gotchas”

If you are leveraging a file transporter feed to link the attachments to content you will be subject to all of the File Definition configurations within Data Feed Manager. This includes the Escape Sequence identifier which is Backslash(\) by default. If you have chosen to identify your attachments in the source CSV file using a full path, this will come into play as a typical path name would most definitely leverage the backslash character: \\server\share\file.doc In this case, you would generate an error in the feed as the backslashes would generate the escape sequence. There are two solutions to this process. The first would be to change the escape sequence to a character that would not be encountered in your source file(ie “***”). The second solution is to double all of the backslash characters in your source file. The previous path would become: \\\\server\\share\\file.doc

//server/share/file.doc








DEMO TIME


Questions?


Free Customer Training • Join the ‘customer only’ weekly Free Friday Tech Huddle

live webcasts @ 12 ET

• Customer Support team addresses how to troubleshoot common issues & utilize new functionality delivered by the RSA Archer products

• Join us! https://community.emc.com/docs/DOC-18975


RSA Archer Course Offerings

System Administrator elearning Introduction to GRC Getting Started with the RSA Archer Platform Navigating the RSA Archer Platform

Business Owner

End User

Instructor-Led RSA Archer Administration RSA Archer Advanced Administration

elearning Introduction to GRC Getting Started with the RSA Archer Platform Navigating the RSA Archer Platform Archer Solution Fundamentals

Instructor-Led RSA Archer Administration Getting Started with Enterprise Risk Management Getting Started with Policy & Compliance Management

elearning Introduction to GRC Getting Started with the RSA Archer Platform Navigating the RSA Archer Platform

Customized RSA Archer End User training is available upon request

www.emc.com/rsa-training


• RSA Archer Community Newsletter – The monthly RSA Archer Community Newsletter is packed with everything you want to know for

RSA products, services, events and GRC industry trends. Never miss an issue and receive the Community Newsletter each month, delivered to your Inbox!

• RSA SecurCare Online – SecurCare Online, a state-of-the-art e-support system, is now available to RSA Archer customers.

With SecurCare Online you’ll be able to plan an installation, upgrade or migration, diagnose and repair problems, find answers to support questions, download current platform patches and bug fixes, manage your support cases, and more – 24 hours a day.

• RSA Archer Live Webcasts, for the public – Live product webcasts are held on Thursdays at 2:00 ET. Also, these webcasts are recorded and

placed on the Archer Community.


Contacting Support

• Email: [email protected]

• Phone:

USA: 1-800-995-5095 (option 5 for Archer)

Global numbers: http://www.emc.com/support/rsa/contact/phone-numbers.htm

mailto:[email protected]

http://www.emc.com/support/rsa/contact/phone-numbers.htm





THANK YOU

migrating attachments 2014

Documents

archer feed

archer data feed

emc corporation

dfm archer

archer approach items

rsa archer free

feed process

file transporter data