world 2010 - migration process

Improving Object Migration Management with Ease

powered by

Ryder’s Enterprise Data Warehouse

Bryan BrandowMicroStrategy ArchitectRyder System Inc.

January 27th, 2010

Ryder System, Inc. is a FORTUNE® provider of leading-edge transportation, logistics and supply chain management solutions worldwide. Ryder's product offerings include: Fleet Management Solutions (FMS), which provides leasing, rental and programmed maintenance of trucks, tractors and trailers to commercial customers; Supply Chain Solutions (SCS), which manages the movement of materials and related information from the acquisition of raw materials to the delivery of finished products to end-users; and Dedicated Contract Carriage (DCC), which provides a turn-key transportation service that includes vehicles, drivers, routing and scheduling. Ryder serves customer needs throughout North America and in Latin America, Europe and Asia.

For 75 years, Ryder has earned high marks for serving customers in a timely, cost-effective and reliable manner. That reputation has helped Ryder to continually rank among the industry's leading companies in many publications, including the FORTUNE® Most Admired Companies, the InformationWeek 500 of leading business users of information technology and InternetWeek's top 100 U.S. companies for effectiveness in using the Internet to achieve tangible business benefits.

Ryder's stock (NYSE: R) is a component of the Dow Jones Transportation Average and the Standard & Poor's 500 Index.

04/08/2023 Proprietary and Confidential 2

EDW Platform Statistics


Usage 8 – 14,000 reports run each day

Over 9 million reports generated in last 8 years

Reports > 250 templates with over 35,000 user saved reports

Users 14,000 (7,000 Internal, 7,000 External)

Applications 20 Subject Area/Applications leveraging the MicroStrategy Suite

Database 3 database platformsNearly 3,000 staging, development, and production tablesOver 4.0 Terabytes of data

Awards 2008 MicroStrategy World Winner Best Practices in BI Individual Excellence

Operations and Maintenance, Customer Fleet Care applications considered “World Class” – Hackett Group 2007

Importance of Data Governance

04/08/2023 4

Importance of Data Governance

04/08/2023 5

• Single Version of the Truth► Ensure unified business rules throughout the

environment.

• Reduced Support► Standardized objects promote reusability and

faster development.

• Gatekeepers not Decision Makers► EDW ensures a unified and consistent

environment based on Business Rules.

“How we decide how to decide”

Ryder’s Change Management Strategy

• Development occurs in a dedicated environment, and after passing Developer Unit Testing, is promoted to a Test environment which mirrors Production.

• Business Users utilize the Test environment to approve projects and data. Further testing from the EDW team is performed as well as a check to make sure the migration details are in order.

• Once signed off, the application is promoted to Production.


Object Manager Overview

• Object Manager provides a single graphical drag and drop interface for migrating objects between environments such as Development, Test and Production.


Object Manager Overview

• Dependencies are determined at the time of the migration and decisions on objects with different versions between environments must be made.



Ryder’s Original Migration Process

How we got here.

What worked well.What didn’t work so well.

Original Migration Process

• Basic documentation of migration actions► This Excel file is passed around between developers and administrators, each appending to

the log.



• Change Tickets for future tracking




• Separate migrations for each folder

► Unrelated objects that were in different folders had to be migrated individually. This would extremely lengthen the amount of time required for each migration depending on how many objects were involved.

► In our larger projects, the processing of a single migration takes 10-15 minutes. A large migration involving several folders could easily take an hour or two of an administrator’s time, and the environment is unavailable during this process.



• Migrations performed in 2-Tier

► Due to past experiences, all of our migrations are performed in 2-Tier. This eliminates potential object caching issues between the active IServer and the Metadata.

► After a 2-Tier migration is complete and the schema is updated, the project must be reloaded. For Production projects, we reload as soon as there are no users in the project. But for our large projects that have long reload times or have constant user traffic, we have scheduled tasks that reload the projects at night.



• Did it work?

► We had no way of knowing if a migration was successful unless we

tested the reports manually or a user called us with an issue.

► Since we migrate in 2-Tier and reloaded the production projects at

night, it was common that we would not catch an issue until users

saw the reports the following morning. By this time, fixes that

required an additional migration would now have to wait another

day due to the project reload requirement.

Original Migration Process – Goals to Improve

• Each Object location was migrated individually, lengthening the migration process

► Find a way to consolidate objects so that migrations were more efficient.

• Logs were manual and time consuming► Find a way to assist or automate the creation of logs for Developers.

• Results of the migration were unknown until further testing or often discovered by the end users

► Find a way to validate migrations to ensure that they were completed correctly.


New Process Ideas


► Use Object Prompts to serve as containers for Migrations.

• Each Object location was migrated individually, lengthening the migration process

► Find a way to consolidate objects so that migrations were more efficient.

New Process Ideas


• Logs were manual and time consuming► Find a way to assist or automate the creation of logs for Developers.

► Query the Metadata to check the Target project BEFORE the migrations to determine if the Migration Package contains all of the necessary elements

► Create the Migration Log automatically.

New Process Ideas


• Results of the migration were unknown until further testing or often discovered by the end users

► Find a way to validate migrations to ensure that they were completed correctly.

► Parse the log files and compare the results to the Metadata to immediately determine the accuracy of the migration.

► Determine if the migration was successful IMMEDIATELY and without ambiguity.

New Process Ideas


• Additional enhancements

► Automatically check the Security of Reports and Documents that were involved in the migration to ensure that their security is properly set compared to the parent folder.

► Export results to HTML to serve as a receipt to attach to Change Control tickets.

Demonstration



Migration Packages

To expedite the migration process, developers can now create a Migration Package by adding all of their desired objects into a single Object Prompt.

Step1:

Create Object Prompt

Step2:

Add all desired objects for migration

Step3:

Save with your name & date in Public Objects\Migrations

Migration Package Builder

Select your Source/Target Projects and Migration Package from the drop down lists.


The Source Projects are a list of all Dev and Int projects from our Development Project Source. The Target Projects are a list of all Int projects and Production projects from both our Development and Production Project Sources. The Object Prompt list is a list of all Prompts (of type Object selection) located in the Public Objects\Migrations folder.


The tool will tell you if there are dependencies missing from your Migration Package but different in the Target.

Review the list, and modify your Migration Package as needed.


The tool will scan the contents of the Migration Package and compare to a recursive list of dependencies from the metadata. Any dependencies that are not included in the Migration Package and do not exist in the Target Project are marked as required. Dependencies that exist with a different version ID are noted and optional to include. Checking the box and clicking Next uses the SDK to insert that object into your Object Prompt.


You will then see a list of items in your Migration Package that are conflicts with the target.

Review this list and check the box for any objects that need to be marked as Replace.


This list is comprised of objects that are included in the Migration Package but exist differently in the Target. The developer is required to check all boxes, but this screen serves as a sign off that they acknowledge everything that they want to replace.


If there are any additional special handling instructions, please list them in the space provided.



The final output text should be copied and pasted into the Remedy ticket. These instructions will help expedite the migration process.


The tool generates an overview of the migration and a list of all of the objects for the administrator that need to be marked as replace (including a hint at the object type). The number of new objects is noted, but since individual decisions aren’t made on these (at least in 8), the count is FYI.


Migration Process – Object Audit Tool

After the migration is complete, we will run an audit on the results of the migration against the Migration Package. In the event of a discrepancy, we will reconcile the difference with the developer. These could include objects that were moved that were not part of the Migration Package, or objects that were part of the package but not moved.

The OMLog is parsed and compared to the metadata to check for errors. Checks include objects moved that weren’t in the prompt (red), objects that were in the prompt but didn’t get moved (purple) objects that moved as expected (green) and the Migration Package itself (blue).

Attach “Receipt” to Remedy Ticket


After the migration is verified, we will export a receipt of everything that occurred and attach this report to the Remedy ticket. In addition to this transactional log in the ticket, the server copy will retain all history of all migrations in a single location. The OM Tool also has built in backup functionality so that we can periodically back up migration logs ourselves. Any logs can be independently loaded and viewed in the OM Tool.

The receipt contains an html output of the log analysis and is attached in our Change Control system for a complete record of every operation that occurred during the migration. Any items that “sneak” over unintended are deleted before the migration is marked as complete. Any errors are immediately resolved.


Migration Process – Object Security Audit

Once complete to production, a Security Audit will be performed to verify the permissions set on each object. By default, they will inherit the security of the parent folder. The Migration Team will raise a question in the event that there are obvious errors and will reconcile to any special instructions included with the Remedy Ticket.

This option checks the security of any Reports in the Migration and compares to their parent folder. This gives the administrator an easy view at who has access to the reports. Differences are highlighted in Red, but the primary decisions are human based. (Note: After the above migration was originally completed, the group “Finance – Vehicle “ did not have access to the report, so this highlighted that issue and we resolved before any users reported not being able to see it.)

Overall Benefit

Historically, Ryder averages 3 migrations per week between Dev->Int and Int->Prod

New Method Dev->Int Int->Prod

Developers create Migration Logs with MPB 1 1

Admin moves Migration Package 8 8 8 each

Errors are eliminated from the process 0 0

QA Testing is shorter due to higher confidence 10 10

Total (hrs) 1 1 2

15 hrs saved per week

Original Method Dev->Int Int->Prod

Developers creating Migration Logs manually 60 15

Admin average 7 migration steps (Tables, Facts, Attributes, Metrics, Filters Prompts, Reports) 56 56 8 each

Admins redo 1/3 of migrations due to error 19 19

QA Testing required to best of our ability in 2-Tier 30 30

Total (hrs) 11 6 17

Future Plans with 9: Automated Package Creation


Extract Change Journaling data to automatically build Migration Logs, further reducing the time it takes for a developer to create a migration package.

Future Plans with 9: Off Hours Scheduling with Update Packages


The final step of the MPB could create an Update Package that can be placed in a monitored folder. A separate process would check this folder nightly and apply the migrations automatically, only alerting administrators in the event of an error. This would greatly minimize administrator involvement and effectively reduce the administrator’s role to approvals instead of tasks. This would also allow backup administrators to easily manage the environment with minimal technical involvement.

Future Plans with 9: Automated Integrity Manager Testing


After the migration is complete, Integrity Manager could be executed to verify that the SQL and data match between Test and Prod.

IM can be called from the command line and the output stored in HTML and parsed for instant off hours alerts.

Thank You


Bryan BrandowMicroStrategy ArchitectRyder System Inc.

[email protected]

January 27th, 2010

Questions?

Build Your Own Migration Package Toolset

• The following slides will map out how to build your own set of tools.

• What you’ll need► Some basic SQL skills (joins, group by, stored procedures)► Some basic programming skills (read the contents of a file, connect to SQL, basic Web and

application GUI)

► No MicroStrategy SDK is required for any portion of these tools as shown in this presentation.




Select * from dssmdobjinfo where object_type = 32

Select * From dssmdobjinfo where object_type = 10 and subtype = 2566 and parent_id = ‘GUID of Migrations Folder’

10 = prompt2566 = object prompt



SQL Code on next slide.

SQL Code


This SQL code shows how to get the contents of an object prompt and determine if there are missing dependencies.

Continued on next slide …

SQL Code Continued


Object Manager Log Tool


1. Parse the log file (OMLog.log in 8.x, Diagnostics Logging Tool for 9x).2. Compare object results to the contents of the Migration Package (SQL Code in

previous step).

Report Security


Query from DSSMDOBJSECU table, joining on the object id’s of the Reports and Documents.

world 2010 - migration process

Technology