the case for migrating legacy email archives · the case for migrating legacy email archives ... as...

The Case for Migrating Legacy Email Archives

WHITE PAPER

Point. Click. Migrate.

2

ARCHIVE360 WHITE PAPER • THE CASE FOR MIGRATING LEGACY EMAIL ARCHIVES

Table of Contents3 Executive Summary

3 Key questions to address before you begin a migration project

4 Why migrate from a legacy email archive?

4 Potential issues to look out for with email archive migrations

6 Email archive migration strategies

8 The 9 steps to prepare for a trouble-free migration

9 Choosing the right migration solution and partner

11 Reducing the risk and cost of migration with Archive360 Archive 2-Anywhere

Table of Figures5 Figure 1. Orphaned message stubs during email archive migration

7 Figure 2. CGOC Survey of Data Value


3


Executive Summary

Over the past 25 years, email archives have become a despised but accepted norm for many organizations, especially for those in regulated industries, or those faced with ongoing eDiscovery requirements. Email archives were seen as the only workable solution to capture, secure, manage and make searchable, those emails and attachments that the government or courts wanted to see from time to time.

Prior to 2010, stand-alone, on-premise email archives were the go-to solution to ensure reduced regulatory and eDiscovery risk. Cloud-based email archiving solutions were later added to the potential offerings and have experienced continual growth. According to the Radicati Group 2013 Information Archiving Market Quadrant report, cloud archiving services market share had reached 32%, with on-premise archiving solutions with a 68% market share. That said, there are many reasons to migrate from a legacy email archive, which we will discuss in the next section.

Key questions to address before you begin a migration project

As you consider your email migration project, several questions should be addressed:

• What is the overall budget to complete this email migration?

• What is the timeframe to complete the migration process? (extraction and ingestion)

• How much data does the email archive contain?

• What email system did the original email archive data come from; Exchange, Notes, GroupWise, other?

• What kind of data does the email archive contain? (email, attachments, calendars, tasks, notes, public folders, metadata including folder structure, folder movements, opened, deleted, etc.)

• Do you have regulatory retention requirements that may apply to the currently archived email?

• Does the organization have a recently updated document retention policy?

• Are there legal hold and eDiscovery responsibilities that pertain to the archived email?

• Will you address the clean-up of old or expired content as part of the migration project?

• How will you measure success for this project?


Many Exchange admins forget all about their email archives when migrating to the cloud — at their peril.

InfoWorld: Moving to Office 365? Prepare for the huge pain of archive migration

4


Why migrate from a legacy email archive?

Many organizations are facing one or more issues forcing them to consider moving away from their legacy email archiving system to something else. Here are a few examples:

• They have outgrown their current stand-alone legacy email archiving systems due to insufficient functionality, rising costs, and general ease of use issues.

• They have decided to move to Exchange 2013 or Exchange Online/Office 365, because of the multi-functionality capability, which includes email archiving, legal hold/eDiscovery functionality, and records management.

• They have standardized on Google Apps for Business as their office productivity suite and want to migrate their current legacy email archive up to the cloud-based Google Vault companion archive.

• They are undertaking an email archiving consolidation project due to a merger or acquisition and have decided to take the opportunity to migrate away from their current underpowered legacy email archiving system.

• Their current email archiving solution has been obsoleted (EOL) and is forcing the organization to make a decision on the current archive data store.

• Their legal or regulatory compliance department has raised questions about potential risks with the current legacy solution.

Whatever the reason for going forward with an email archive migration project, the main question is not whether to migrate existing archived data, but rather, how to start, what data to migrate, and what steps are needed to ensure a smooth, low risk, successful, and cost-efficient migration.


1. Email archive vendor APIs: Many legacy email archiving solution APIs are not “fully baked” and can be extremely slow causing extraction bottlenecks. They also tend to overlook important metadata or lose data during the extraction. Email archiving vendors never expected (nor want) their customers to move data away from their platform so little thought was given for easy and complete extraction. Choose an email archive migration vendor that can bypass vendor APIs – this will speed up the data extraction and give you more granular control of the process.

2. Ingestion bottlenecks: Another issue you should plan for is the ingestion capability of the platform you will be moving the archived data into. When choosing the new platform, consider the ingestion rate. Depending on your migration strategy (discussed in the next section), the time frame of the migration project will be limited by this. For example, Archive360’s Archive 2-Anywhere can extract email archive data at a rate of 5 TB per day but if the target repository ingestion rate is limited to say 600 GB per day, the migration project’s overall speed is limited to 600 GB per day.

potential issues to look out for with email archive migrations

What causes data migration projects to fail?

1. Unknown and unplanned for data formats appear

2. Legacy data quality is worse than expected

3. Legacy system documentation is lost or insufficient

4. Legacy data does not fit well in the new system

5. Legacy data conversions are not well understood

6. Migration time and cost were drastically underestimated

Bill Tolson Tolson Communications LLC

5

ARCHIVE360 WHITE PAPER • THE CASE FOR MIGRATING LEGACY EMAIL ARCHIVESPoint. Click. Migrate.

3. Not fully understanding regulatory/eDiscovery requirements: Any time large data migrations occur, consider legal and regulatory aspects and ensure you fully understand the requirements before the migration starts. Consult your corporate legal department or outside counsel on the status of any current or pending legal actions that could trigger legal hold requirements, or eDiscovery collection and review processes for data currently in the email archive. Migrating archived data that is or may be involved in a lawsuit is a high-risk undertaking and should be well understood and approved (in writing) by your legal representatives. This is because archived data could be inadvertently lost or changed during the migration process triggering spoliation (destruction of evidence) accusations. The movement of legally relevant data without strict audit and oversight could introduce “chain of custody” questions into the proceedings causing data beneficial to your legal defense to be ruled inadmissible.

Archived data subject to regulatory retention requirements can pose additional risks during migration without strict planning and oversight. Loss of regulated data can trigger investigations, agency lawsuits, fines, and negative publicity. When migrating data subject to regulatory retention, take care to ensure the migration process does not put the data at risk. Another issue with migrating archived data subject to regulatory or legal requirements is the ability to find the data after migration. This may seem like an obvious point but many organizations have found themselves unable to respond to a regulatory information request or eDiscovery order in the time expected because they “temporarily misplaced” data during migration. Under certain circumstances, this can be seen as destruction of evidence.

4. Message stubs (pointers): One of the benefits many email archiving solutions provided was the ability to manage email box sizes by automatically removing an email message and/or attachment from the user’s Exchange mailbox and moving it to the email archive while keeping the moved emails accessible to end-users by leaving a stub (or pointer) behind pointing at the email message and attachment in the archive. When the end user clicked on the pointer in their mailbox, the email was retrieved from the archive and opened. With emails and their attachments making up the vast majority of data stored on the Exchange server, stubbing relieved the Exchange server producing a faster, more reliable email messaging system - without end user compromises.

Enterprise VaultArchive

Office 365

PST

Migration Application

Emails and Email Stubs

Archived Messages

Error Message

Message not found!

Figure 1: Orphaned message stubs during email archive migration

“There’s no such thing as a standard or simple migration to Office 365.”

Every organization has something that can create a problem during a migration.

Network World: Doing an Office 365 Migration the Right Way

6


Email archive migration strategies


Organizations can choose from several email archive migration strategies depending on their budgets and legal or regulatory situation. The four most utilized migration strategies are:

Phased Migration

In the phased or gradual migration strategy, a smaller group of employee’s (usually department’s) data is migrated to the new system instead of the entire archive at once. This approach has the benefit of affecting end-users the least while providing them access to their archived data as soon as possible. A couple of added benefits include:

• Being able to test and fine-tune the process early on with small groups of mailboxes, and

• If major problems do occur during migration, reverting to the legacy system can take place quickly with the smaller numbers of mailboxes.

The downside of this approach is that it lengthens the entire migration process so that the old and new archives reside in the enterprise at the same time for a longer period.

Parallel Systems (point forward)

This strategy employs a parallel process that keeps both the old system and new archive system running for an even longer period than the phased migration strategy. In this strategy, the new archive system kicks off and begins archiving everyone on day 1 going forward. Employees must access the new archive for data archived after day 1 and the old archive for files archived before day 1. The intention is to let the old archived data expire so it does not have to be migrated at all. The main benefit of this strategy is to make the new archive available as soon as possible while not incurring the expense of migrating the old data. The other benefit is that the legacy archive is available and accessible if problems do occur with the new system (reverting to the old system is a quick process). The downsides include end-users have two systems to access to search for content and the ongoing cost of keeping the legacy system up and running (software support and maintenance costs).

If archived emails are migrated without the Exchange stubs being accounted for and removed from the individual mailboxes, errors would be generated every time a stub is clicked on (Figure 1).

5. Incomplete metadata migration: Archived email can contain a great deal of information about the “life” of the email in its metadata. This metadata can contain data such as To, CC, BCC, forward, the various folders it resided in including - was it moved to the deleted items folder, was it opened, etc. Metadata can be discoverable in legal cases and is expected to be included in an eDiscovery response so should not be left behind in an email migration or blindly deleted. Not including email archive metadata as part of an eDiscovery response can raise the risk of spoliation or destruction of evidence.

6. Exception handling and reconciliation: Emails can become corrupt during the archiving process or prior to the archiving process. Many have the attitude of “so what, no one will miss it.” No matter where an email became corrupt, it is important to flag and report on any corrupt emails during the extraction process. This exception report can become an important piece of evidence later if that specific email was relevant to a regulatory inquiry or eDiscovery request. Without the detailed exception report, the Judge could rule that the email was destroyed on purpose – spoliation.

7


1 CGOC: Information Economics and Understanding the Relationships Between Cost and Value https://www.cgoc.com/files/CGOC_Workshop_Nov2012_NYC_Info_Econ.pdf

Business Value25%

Regulatory5%

Legal1%

Enterprise Information

UnnecessaryInformation

69%

ValuableInformation

31%

Figure 2: CGOC Survey of Data Value

“All or Nothing” Migration (immediate cutover)

This strategy is considered the “let’s just get it over with” strategy where all data in the old archive is migrated into the new system all at once, verified and then the old system is shut down. The benefit of the immediate cutover strategy is to complete the migration as soon as possible. The downside is the higher risk involved. If something goes wrong in any part of the process, it can be difficult and expensive to go back.

2-Step Migration

The last three migration strategies all involve moving content from the email archive directly into the new archive or repository. The 2-step strategy incorporates an additional process to allow for content classification, tagging, and disposition before final ingestion into the final repository. The main reasons for this additional step are:

• To find and dispose of email that is beyond all retention periods (expired) and is not relevant in any legal proceedings

• To identify, classify and tag all archive content that is subject to regulatory retention requirements and/or is subject to legal hold/eDiscovery before it’s lost or misplaced in a huge new repository

Many legacy email archiving systems relied on capturing and blindly archiving all “journaled” email content… just in case. Because of this “capture everything” strategy, large portions of legacy email archives are populated with unnecessary or “junk” content that doesn’t need to be migrated into the new repository. The Compliance, Governance and Oversight Counsel (CGOC)1 conducted a survey in 2012 that showed that on the average, 1% of organizational data is subject to litigation hold, 5% is subject to regulatory retention requirements and 25% had some business value (figure 2). This means that approximately 69% of any organization’s retained data has no business value and could possibly be disposed of without legal, regulatory or business consequences.

The 2-step strategy can be used in conjunction with any of the first three strategies listed above.

Best practices for all data migration projects include backing up all data to be migrated and verifying the migration is successful before shutting down the legacy system.

8


The 9 steps to prepare for a trouble-free Migration

1. Understand the legacy archive environment: Having an in-depth understanding of the legacy archive can save you from problems further into the migration project. For example; how far does the archive go back, what types of Exchange data did it capture – emails, attachments, calendars, appointments, contacts, tasks, types of content, stubs, and what types of metadata did it capture such as folder movements, was it opened, was it deleted and when. This data will tell you what content you should consider migrating.

2. Understand the target environment: What types of data formats can it accept, can it take and effectively utilize the additional metadata from the legacy archive environment and/or from the staging area mentioned above from the 2-step migration process, and what is the maximum ingestion rate for the target system.

3. Conduct a full pre-project assessment: Get a detailed idea of data points like how much data exists in the archive, the types of data, are there “stubs” in the Exchange server pointing at individual emails in the archive server, are there PSTs to include and where are they. Are there any legal or regulatory risks to be aware of and how will the migration affect end-users during and after the project is complete?

4. Choose the right migration strategy: Choosing the best migration strategy for your situation will help in securing executive sponsors. The migration strategy is also a prerequisite for determining project timeline, project planning, and overall budget.

5. Develop a detailed migration plan: Creating a detailed migration project plan is a best practice to force you to consider all possible aspects of the migration and identify where potential issues can occur.

The migration plan should include:

1. Purpose statement – why is the migration happening2. An overview of the project – what will happen3. A Timeline – when it will happen4. A listing of responsible parties – who to contact with questions5. A breakdown of estimated resources needed6. Risks and assumptions – including potential “gotchas” and work-arounds7. Projected savings (dollars, productivity) and reduced risks (regulatory and legal)

6. Share the migration strategy: Discuss the migration project with the General Counsel and Chief Compliance Officer and their respective departments to solicit guidance and approvals for chosen migration strategy and plan.

7. Communicate the migration project: Communicate with all affected parties so they know ahead of time what will happen and when, so they can alert you when something is not right and minimize disruptions.

8. Create a recovery plan: Migration projects, especially larger ones, can have issues arise so be prepared to “go back” if the problem is not easily fixed; not being able to continue with the project and not being able to return to a known good state can cost a company large amounts of money and end-user productivity.

9. Partner with the best email migration provider: The surest way to head off costly migration issues is to collaborate with the migration partner with the best software and most appropriate experience for your particular situation. Take the time to find the solution provider that best meets your needs.

9


Choosing the right migration solution and partner

As has already been mentioned, a legacy email archive migration can be a complicated process with many challenges and risks. To reduce the risk of a failed or problematic email archive migration, choosing the best migration solution and partner is an absolute must.

Choosing the right email archive migration solution

There are many email archive migration tools available from both the email archive providers and third-party vendors. Many will automatically gravitate to the original email archive solution provider thinking, “Who would know better how to migrate out of a particular email archive?” The fact is, email archive providers don’t really want their customers moving out of their solutions so don’t necessarily spend a great deal of time or dollars providing migration capability to leave their solutions. If they do have some ability to actually do it, it may be positioned as a “basic” capability that may not address many of the questions already posed earlier in this paper, or it may be positioned as a professional services solution requiring custom coding and a great deal of time (and money) to complete. This is not to say that email archive solution providers cannot or will not offer viable migration capability, just compare their actual capability with third-party providers to understand what is possible from whom.

In choosing the best email archiving migration solution, reviewing the questions from page one, Key questions to address before you begin a migration project, will help in choosing the best solution for your organization.

• What is the overall budget to complete this email migration? The estimated (and/or approved) budget will help establish the ceiling for what the project can afford. As we’ve already mentioned, some migration solution vendors will rely on a mostly manual professional services engagement dramatically driving up the cost.

• What is the timeline to complete the migration process? The approved timeline for project completion could be driven by business challenges such as competitive situations or a consolidation or migration due to M&A activity. Other situations that could affect your timeline include regulatory information requests or legal hold/eDiscovery requirements. For any of these situations, the migration solution could prove to be a limiting factor if a hard and fast time requirement is involved. Your estimated timeline may vary based on the solution’s extraction performance and the target repository’s ingestion rate.

• How much data does the email archive contain? The total amount of data to migrate along with the migration solution’s maximum extraction performance and the target repository’s maximum ingestion rate will provide an estimate on the amount of time required to complete the migration project. For example, if your organization has a total of 100 TB in the email archive, the migration solution has a maximum extraction rate of 1 TB per day (8 hr. day), and the repository can ingest over 5 TB per day, then the timeframe required to complete the migration project is 100 days (assuming no unforeseen issues). Looking at that same example but using a migration solution with a maximum extraction rate of 5 TB per day, the total time to complete the migration project is 20 days (again assuming no unforeseen issues) – quite a difference, especially when you have a very tight timeframe for project completion.

• What email system did the original email archive data come from? Over the last 15-20 years, numerous email systems have appeared (and disappeared). The most popular and longest lasting for businesses are Microsoft Exchange, Lotus Notes, and Novell GroupWise. Email migration solutions are not a universal “one solution works with all email systems” so knowing what type of email system was archived will help narrow down those migration solutions you should spend time looking at.

10


• What kind of data does the email archive contain? Some legacy email archiving solutions were somewhat basic in that they only captured emails and their attachments. Other legacy solutions captured additional email system content such as calendars/appointments, tasks, notes, public folders, and detailed metadata including legal hold status, folder structure, email movements, To/From/CC/BCC, status as whether it was opened, or the fact that the original email (or stub) was deleted from within the email system. If your email archive did capture this level of information, then your migration solution should be able to address it as well.

• Are there regulatory retention requirements that may apply to the currently archived email? Compliance with Federal, State and local laws requiring the retention and availability of specific “records” are a fact of life for all organizations (whether they know it or not). Companies in highly regulated industries such as financial services, energy, healthcare, and pharmaceutical manufacturers have highly prescriptive laws (regulations) pertaining to the types of records they must maintain and make available for inspection for specific periods. If an organization employs anyone, pays corporate taxes, or has investors, they have federal regulatory retention requirements. Regulatory agencies can ask an organization for specific records that should have been retained. This information request is usually unscheduled and carries with it the expectation that the organization will be able to respond fully in a relatively short amount of time. The inability to respond because the requested content was lost or deleted during migration, or left on the legacy email archive and was later wiped, can trigger fines, penalties, and lawsuits by the regulatory agencies. If you plan to migrate the entire email archive to a new repository, then you will need to be able to find the data later if asked for. If your migration strategy includes the “2-step” cleaning/culling step, then ensuring regulated content is not disposed of during that process by being able to search for and recognize it, before or during the migration process, is very important.

• Does the organization have an up to date document retention policy? Closely related to the regulatory discussion above is the need for the organization to have an up to date information retention schedule. This would include retention periods for regulated information but also contain descriptions and retention periods for content regarded as important to the business – has business value for some length of time.

• Are there legal hold and eDiscovery responsibilities that affect email in the archive? Much like regulated information, content potentially relevant to litigation must be protected under a legal hold (or litigation hold) and eventually reviewed and produced based on an eDiscovery request – no matter how old it is or where it’s stored. A litigation hold is a provision in the Federal Rules of Civil Procedure (FRCP) requiring any party to an anticipated or existing lawsuit preserve all data that may relate to a legal action. A litigation hold ensures that the documents (evidence) relating to the lawsuit are not destroyed and are available for the discovery process prior to actual court dates. Those organizations that ignore or don’t take the legal hold seriously run the risk of destruction of evidence charges and loss of the case. A legacy email archive can hold thousands or millions of emails and other data that could be subject to legal hold and eDiscovery. When planning for an email migration, you should always check with your corporate legal department to see if there are current or anticipated lawsuits that could affect the migration.

• Will you address old or expired content as part of the migration project? Many organizations incorporate a “defensible disposition” process (including the 2-step migration strategy) as part of the email migration project to show a much higher ROI to ensure the project is approved. In most cases, the defensible disposition will add additional requirements to the migration. These include building policies to classify and tag content that is expired or valueless, as well as potentially deleting that content either during the migration or later, after it has been reviewed by legal and regulatory compliance groups.

11


• How will you measure success for this project? A valuable step in an email migration process is determining success levels after completion. Some potential questions to address to determine success are:

1. Did the project stay within the estimated time allotted?

2. Did the project stay within the estimated budget?

3. Were there any major unforeseen issues that arose?

4. Did the migration vendor meet or exceed expectations?

5. Did the project meet or exceed objectives developed prior to the start of the project?

Reducing the risk and cost of migration with Archive360 Archive 2-Anywhere

Archive360 has a deep understanding of email archiving technology and is the expert in working with many of the most popular email archiving platforms such as Symantec Enterprise Vault and HP/Autonomy Zantaz and NearPoint. Archive360 can ensure regulatory and eDiscovery compliance during migration and have the required expertise to extract and migrate legacy email archive data in a legally defensible manner to all of the most sought after destination platforms such as Exchange 2010/2013, Office 365, as well as other well-known cloud archive solutions.

Key Benefits of Enterprise Vault Edition

• Install and deploy quickly. The typical Archive 2-Anywhere installation is up and running in less than 30 minutes.

• Data extractions are extremely fast, limited only by the ingestion rate of the receiving repository. Archive 2-Anywhere is capable of running unlimited parallel mailbox extractions on a single server, while utilizing multiple processing threads for maximum throughput.

• Archive 2-Anywhere extracts raw messages and attachments, including all metadata, and reconstitutes them into the original Exchange MSG format for complete data extraction.

• Archive 2-Anywhere is capable of complete, original message fidelity providing a defensible response to eDiscovery and regulatory information requests.

• Archive 2-Anywhere can restore corrupted emails and create a valid MSG file. Exceptions can result from corrupt folder structures, corrupt or lost message bodies and corrupt or missing sender information.

• Archive 2-Anywhere maintains a detailed audit trail for compliance and eDiscovery reporting. A single user-friendly dashboard allows full control and management of the entire extraction process including extraction rules, export targets, and reporting.

Message Stub Management

Many email archive solutions provided the ability to manage Exchange email sizes by removing email messages and/or attachment from the user’s mailbox to the email archive while keeping them accessible to end-users by leaving a pointer (or stub) behind for the end-user to click on to access the full email and attachment stored in the archive. Over time, this stubbing capability created hundreds of thousands, or millions, of stubs within Exchange triggering a major issue. When it comes time to migrate the email archive, the presence of stubs greatly complicates the migration process. To ensure all stubs in Exchange are matched and restored with the archived messages in the email archive, a Stub Rehydration process is critical. Archive 2-Anywhere is the only email archive migration solution that can systematically match-up and restore stubs with original archived messages, and remove leftover stubs from the Exchange server in a forensically sound process.

12


About Archive360

Archive360, Inc. is setting the standard for technologically advanced software tools for email archive migrations. As an Independent Software Vendor (ISV) the company works directly with customers and a global network of specialist partners. Archive360 designs all solutions to protect records with complete chain of custody reporting, and meet the most demanding archive migration projects in the world. Archive360 recently acquired Archive1024. For more information,, please visit http://www.archive360.com.

Archive360 Global Headquarters:1737 S Naperville RoadSuite 101Wheaton, IL 60189 USA

P: +1 (630) 358-4448E: [email protected] @Archive360ARCHIVE360.COM Point. Click. Migrate.

© Copyright 2014 Archive360, Inc.Archive360, Archive 2-Anywhere, Stub Finder and Stub Rehydration are trademarks of Archive360, Inc.

http://www.archive360.com

the case for migrating legacy email archives · the case for migrating legacy email archives ... as...

Documents