sap data archiving-decision making guide

4/8/2023

SAP DATA ARCHIVING

A Guide to decision making

TABLE OF CONTENTS

1. INTRODUCTION....................................................................................................................4

1.1. DOCUMENT OBJECTIVES....................................................................................................4

1.2. SCOPE..............................................................................................................................4

1.3. LIMITATIONS......................................................................................................................4

1.4. PRE-REQUISITES...............................................................................................................4

2. WHY DATA ARCHIVING.......................................................................................................4

2.1. IMPROVED SYSTEM PERFORMANCE....................................................................................4

2.2. INCREASED SYSTEM AVAILABILITY......................................................................................4

2.3. COMPLIANCE WITH RETENTION POLICIES............................................................................5

2.4. EFFICIENT USE OF RESOURCES........................................................................................5

2.5. EASIER DISASTER RECOVERY............................................................................................5

2.6. LEGAL COMPLIANCE..........................................................................................................5

2.7. UPGRADE COMPATIBILITY...................................................................................................5

2.8. MOST RELIABLE METHOD FOR DATA RETIREMENT................................................................5

3. CONSIDERATIONS IN ARCHIVING DATA..........................................................................6

3.1. DATABASE SIZE AND GROWTH............................................................................................6

3.2. CORPORATE DATA RETENTION POLICY...............................................................................6

3.3. DATA MANAGEMENT STRATEGY.........................................................................................6

3.4. MAINTENANCE ISSUES.......................................................................................................6

3.5. SUPPLEMENTARY SOLUTIONS.............................................................................................7

3.6. INTEGRITY OF DATA...........................................................................................................7

3.7. LEGAL AND TAX REQUIREMENTS........................................................................................7

3.8. PROJECT COST CONSIDERATIONS.....................................................................................8

4. THE RIGHT MOMENT...........................................................................................................8

4.1. EARLY ARCHIVING.............................................................................................................8

4.2. LATE ARCHIVING...............................................................................................................9

4.3. THE RIGHT MOMENT.........................................................................................................9

5. ARCHIVING PROJECT – WHAT DOES IT TAKE ?...........................................................10

5.1. BUILDING THE PROJECT TEAM..........................................................................................10

5.2. PROJECT TASKS AND TIME LINES....................................................................................11

5.3. RESOURCE REQUIREMENTS.............................................................................................12

5.4. IMPLEMENTATION COSTS.................................................................................................13

5.5. MAINTENANCE COSTS.....................................................................................................13

5.6. SAVINGS AND COST REDUCTION.......................................................................................13

6. CRITICAL SUCCESS FACTORS........................................................................................14

6.1. EXPERIENCED PROJECT TEAM.........................................................................................14

4/8/2023

6.2. INVOLVING ALL EFFECTED PEOPLE....................................................................................14

6.3. COMMUNICATION AND CHANGE MANAGEMENT..................................................................14

6.4. TOP MANAGEMENT COMMITMENT.....................................................................................15

6.5. PROJECT MANAGEMENT EXPERTISE................................................................................15

6.6. RIGHT RESIDENCE AND RETENTION TIMES........................................................................15

6.7. CORRECT SEQUENCE OF OBJECTS...................................................................................15

7. LIMITATIONS OF DATA ARCHIVING................................................................................15

7.1. DATA ARCHIVING IS COMPONENT SPECIFIC.......................................................................15

7.2. NOT SUITABLE FOR REMOVING ORGANIZATIONAL UNITS.....................................................15

8. Glossary..............................................................................................................................16

4/8/2023

1. Introduction

1.1. Document ObjectivesThe process of decision making involved in selecting the right Data archiving solution can get complex with the multitude of variables effecting the decision as well as the options available. This document is intended to assist in analyzing and evaluating different issues involved in selecting an archiving solution for SAP R/3 Data. It is also intended to assist in understanding Key challenges involved in Data Archiving projects.

1.2. ScopeThis document discusses the main drivers in deciding to go for SAP data archiving. An attempt is made to highlight various considerations for and against Data Archiving. It also highlights the benefits of archiving on the one hand and the costs involved in implementing it and the limitations of SAP data archiving.

1.3. LimitationsEvery organization operates in a unique environment and hence the considerations would also be different. It is not possible to set a standard cost benefit equation that would be applicable to all situations. However, the issues discussed in the document could help as a guideline to develop one.

Data Archiving in the context of SAP R/3 system is only addressed in this document. Technical issues and solutions related SAP data archiving are not covered in the document.

1.4. Pre-requisitesBasic understanding of SAP R/3 system, it’s functioning, components and architecture is essential to understand the issues discussed in document. It would also be useful for the reader to have awareness of the SAP Data Archiving.

2. Why Data Archiving

2.1. Improved system performanceBy data archiving, you avoid the gradual impairment of system performance and poor response times that comes with keeping too much data. One of the major reasons for deterioration in system performance is the increase in database size. Data Archiving helps to address this issue in a systematic way.

2.2. Increased System availabilityThere is an inverse relationship between the availability of system to users and the size of the database i.e. the smaller the database, more the time it is available to users.

4/8/2023

Reduced database size would make it easier to maintain the database. The stop and re-start time would be less and the maintenance windows could be smaller.

This is mainly due to the operations that require the application system to be shut down, or the need to deny end user access to certain areas temporarily. These delays could be worse when upgrading to a newer software release or when recovering data after a system crash.

2.3. Compliance with Retention policiesData archiving helps to comply with the corporate retention policies regarding business data. SAP data that has long retention time could be moved to external storage systems or, if expired, could be purged using the archiving programs.

Certain data like Financial Accounting documents need to be retained for a long time to comply with legal requirements. This could be easily achieved without effecting system performance by archiving the data and storing it outside SAP system.

2.4. Efficient Use of ResourcesResources, such as hard drives, memory, CPU and networks are used more efficiently. As an extra benefit, reduced data volume reduces administrative work, which keeps your overall IT costs down

Less data would mean less programming, less testing and not having to worry about getting every bit of computer horsepower working together to get it moved as fast as possible.

2.5. Easier Disaster RecoveryHuge database carry higher risk of program terminations and system crashes compared to smaller databases. A lien and healthy database ensures easy recovery in case of any disasters. If the database is wiped out or corrupted for some reason, it takes longer time to lay it down from tape with large databases.

2.6. Legal ComplianceSAP provided features like DART helps to comply with audits and legal requirements. DART was developed by SAP in coordination with ASUG primarily for complying with the IRS requirement for retention of financial accounting data.

2.7. Upgrade compatibilityData archiving is “Future-proof”, that is independent of hardware and software release levels. Metadata is saved in the archive file along with the actual application data so that the archive file can be read long after it was created and even after the system is upgraded.

2.8. Most reliable method for data retirementData archiving is the only method supported by SAP for removing data from SAP system consistently. Functionality required for archiving SAP data is available in the standard

4/8/2023

system and is supported by SAP. Standard SAP system consists of archiving objects to remove data from almost every application area, especially those with high growth rates.SAP also ensures integration of data retrieval with standard business transactions in many cases. This helps to accomplish the goals with least or even no development work.

3. Considerations in archiving data

3.1. Database size and growthSystem performance is seriously impacted by the growth in database size. The smaller the database, the better the system performance. Larger database directly impacts dialog transactions and reports resulting in longer wait time for users.

Inefficiencies in software code may not show up in smaller databases, but my cause huge problems in large SAP databases, where you are trying to squeeze every performance efficiency possible.

Performance issues and performance tuning activities take up significantly more time if the database size is huge.

3.2. Corporate Data retention policyMore and more organizations are considering it important to have a clearly spelt out Information Management and Retention policy. This policy needs to be drafted considering all relevant aspects including the Audit, Legal, Tax and business requirements.

One of the important considerations in implementation of Data archiving projects is the corporate data retention policy. Data archiving helps organizations to ensure compliance with the corporate data retention policies.

3.3. Data Management strategyGiven the importance of data in today’s workplace, more and more companies are expanding the Data archiving project to become a total data management project.

The requirement for data may vary from management reporting to Legal requirement to keep the data. Different options are available to suit these varying requirements like Data warehouse, Near-line storage systems etc., and a comprehensive data management strategy helps to address these in totality.

3.4. Maintenance issues

The more the data, the longer it takes to stop and restart the database. Consequently the down time would increase in proportion to the growth in database size.

Large tables like COEP, BSIS are difficult to be reorganized because they cannot be exported/imported in the allowed maintenance windows. This causes slower response on the system because the tables cannot be reorganized.

4/8/2023

There is more maintenance involved if the database volume is large. For example, you may have to work with many disk packs instead of a few during any kind of file system maintenance.

The volume of SAP data directly relates to the amount of time it takes to refresh production test environments. Additionally, it takes much longer to apply new functionality to the database.

Steady growth of database would require frequent upgrade in system resources like servers, networks etc. It would be hard to move to a different hardware platform, since a different hardware platform means you have to export the database and import it. Even with faster hardware it would still take multiple days to move.

3.5. Supplementary solutionsThird party solutions are available for supporting the process of archiving, storage and retrieval of SAP data.

Archiving functionality provided by SAP addresses the requirement in most cases. Custom objects could be created for archiving custom tables, LIS structures etc.

Storing Archived data could be done using SAP Content Management Service (CMS), HSM systems etc. Archive link interface facilitates storing of archived data in external storage systems. Storage solutions provided by IBM’s Commonstore are highly compatible with SAP, also satisfy the legal requirements for storing archived data.

Retrieval of archived data is another area where many supplementary solutions are available. Solutions are available from vendors like PBS software, which highly integrate the archived data with the SAP standard retrieval transactions. Other solutions like CDART enhance the functionality provided by SAP, especially in data retrieval.

3.6. Integrity of dataSAP Data Archiving is highly integrated with application modules and automatically ensures integrity of remaining data in the system. Archiving objects are designed taking into account the technical dependency of data in R/3 tables for consistently removing data from the database.

Using SAP Data Archiving, important business objects such as accounting documents and material master records can be selectively removed from the database in such a way that the user does not have to worry about the underlying physical table design.

3.7. Legal and Tax requirementsAnother important consideration while deciding to archive data is the Legal , audit and tax requirements for the data. A good archiving system must help to ensure that the archived data complies with this multitude of requirements. DART was co-developed by SAP in association with ASUG for supporting some important requirements specified by IRS.

4/8/2023

Generally auditing or controlling department helps to identify the legal requirements of the archived data from the legal point of view. Country-specific considerations and additional requirements for external audits must also be taken into account.

3.8. Project Cost ConsiderationsAnother important consideration in evaluating an archiving project is the cost involved in implementing it. Even though a cost benefit analysis is not always the deciding factor in an Archiving project, it certainly plays an important role in the decision making process.

Each and every organization has unique dynamics, external and internal, and it is not pragmatic to apply a common yardstick to all situations. Various components affecting the cost of an Archiving project is explained in Point 5.

4. The right MomentData archiving should not be seen as the last resort to prevent system collapse after all possible hardware upgrades have been implemented. Instead, data archiving should be part of the regular maintenance procedures, such as backup or database reorganization.

Even before a system or a process is live, the cumulative data quantities should be estimated for the future. Conversely, unnecessary problems and costs arise if archiving is only carried out to resolve performance and administrative bottlenecks.

Generally two scenarios can be identified:

Early implementation of the archiving project

Late implementation of the archiving project

4.1. Early ArchivingIn this scenario, the need to archive data is addressed in the early stage of a system or process. The aim is to keep database volumes to a minimum, at an early stage, by removing unnecessary data from the database.

Ideally, Data Archiving should be considered in the early stage of the SAP R/3 implementation during the sizing phase. Data Archiving is primarily a preventive tool for keeping the database healthy and fit. It cannot be expected to return a database to high performance state after all other resources are exhausted.

The employee responsible for the application or process knows the business processes and is familiar with the data objects that are linked with those processes in R/3. In almost all cases, it is possible to estimate the number of documents and quantity of accompanying data. Together with knowledge of the existing system, the decision to implement data archiving can therefore be made at an early stage and the necessary steps implemented, such as setting up a plan for regular archiving.

4/8/2023

4.2. Late ArchivingIn this scenario, the aim is to stabilize and reduce database volumes. This situation occurs when performance or administration problems arise due to large data volumes. If you are already experiencing system bottlenecks, you will experience the following additional problems when archiving data:

Longer runtimes for the archiving programs

Additional system load caused by archiving

Data archiving requires more than just the physical removal of data from the database. It is necessary to involve all of the groups involved in carrying out a joint analysis and creating a requirements catalog.

In the first instance, this involves gathering information on the size and growth rate of database tables. A second step involves identifying the archiving objects that are assigned to these tables. Next, the data objects are then checked for archival status and to determine what requirements are to be placed on the archived data.

Designing the archive involves incorporating the requirements that have been identified during the analysis phase in a uniform archiving concept and setting up a concrete archiving plan.

During implementation phase, the data objects that are no longer required are removed from the database according to the previously created implementation plan.

Issues Faced in Late Archiving

It is highly essential to start archiving before exhausting all other resources

Could be beneficial to monitor the growth for a while after implementation and start archiving

Project gets more complex as the database grows bigger

Data archiving programs need more resources

Longer recovery time in case of disaster

Maintenance windows for “data hygiene tasks“ might become insufficient

Increased down time for upgrade of software releases

Slower system response time makes all online users less efficient

4.3. The Right MomentData Archiving is often thought to be worthwhile only after a system has reached a certain size or has been in life for a certain period. However, this statement is not true for two reasons.

4/8/2023

First, from a business point of view , there are many archiving objects that can be used as soon as the initial implementation is completed e.g. WORKITEM, IDOC etc Second , application areas or tables can grow very quickly and reach a significant size in a very short time, even if the database grows moderately.

It is usually the case that within 1-2 years from the time of R/3 system going live, the data required online in the R/3 system nearly stagnates. This is primarily because most of the data stored in the system on closed business transactions are not required to be accessed except in exceptional circumstances.

The following Chart highlights the volume of data created by the R/3 system which is not required to be maintained online. This is based on the assumption that:

SAP R/3 was implemented in Year 1 Annual Database growth 200 GB Online retention for data 18 Months

Within 3 years from going live about 50 % of the data is generally not required online and by the sixth year 3 out of four records is found to be achievable. The equation might vary based on addition of functionalities implemented, change in volume of business and specific requirement for online access to data.

5. Archiving Project – What does it take?

5.1. Building the project TeamDuring data archiving, it is not sufficient just to remove the relevant data objects from the database. It is much more important to take into account the business environment in

4/8/2023

which the data objects are embedded in order to understand the consequences of archiving. As some application objects are used in cross-application process chains, the persons responsible for individual applications should be involved in the archiving project.

The exact composition of the groups will vary according to the size and the internal organization of the enterprise. The following groups are possible participants in archiving projects.

Database administration R/3 System administration / BASIS Finance and Controlling Internal auditing department / External auditing Information management and retention team Persons responsible for the application External service providers, such as consultancies SAP Remote Archiving Service

5.2. Project Tasks and Time Lines While it is neither possible nor pragmatic to prescribe a schedule to fit all projects, it may often be useful to know the tasks and time involved in an archiving project. The following table lists the tasks involved and the time required in a typical project. It is assumed that the organization has only the main SAP modules implemented and the business complexity is average.

Project Phase Duration Key Tasks MilestonesProgram set up 1 Weeks Assemble project team

Identify project objectives Create Outline concept

Outline concept created

Business blue print 3-4 Weeks

Requirement analysis Data Prevention possibilities Retention/Residence times Storage concept Implementation plan

Business Blueprint sign off

Realization 4-6 Weeks

Archive and Retrieval design Basis customizing Application customizing ArchiveLink customizing Enhancements System testing Integration testing User training

User acceptance of system

Go Live 2-4 Weeks

Transport settings Create variants Pre-processing Execute archiving Post processing Database re-organization

Project Close

4/8/2023

Project close

5.3. Resource requirementsResource requirements on a Data Archiving project will generally depend on the modules that have been in use in the system. It will also depend on the complexity of each business scenario and technical environment. Customization of Data retrieval or enhancement requirements would also effect the resource and effort required in a Data Archiving project. It is assumed that the organization has only the main SAP modules implemented and the business complexity is average.

Resource Project Phase Skill requiredProject Manager – Customer (1)

All Phases - Project Management- Internal dynamics of the organization- Key contacts for different divisions, groups etc- SAP R/3 application being used- Basic understanding of SAP Data Archiving

Project Manager – Consulting Partner (1)

All Phases - Project Management- Change Management- SAP R/3 application - SAP Data Archiving- Storage systems

Data Archiving Consultants

Business blue printRealizationGo Live

- Business Analytical skills- Communication skills- SAP R/3 Functional Expertise (Including

configuration and customization)- SAP R/3 Technical Knowledge- SAP R/3 Data Archiving expertise- SAP R/3 ArchiveLink knowledge- Data Base systems and storage systems- Project management

Technical consultants

RealizationGo Live

- R/3 ABAP development experience- Data Archiving knowledge - Working knowledge of basic UNIX functions- Knowledge of ALE/IDOC/Workflow- Excellent verbal and written communication - Team experience working with off-shore model

BASIS consultants

RealizationGo Live

- R/3 BASIS experience- DBA knowledge- Knowledge of storage systems- ArchiveLink experience- Data Archiving knowledge preferred- Working knowledge of basic UNIX functions

Stake holders ( to be available through project on part time basis )

Business blue printRealizationGo Live

- Business specific knowledge- Data retrieval needs- SAP User experience- Legal and tax requirements

4/8/2023

5.4. Implementation Costs

Costs Deciding factorsHardware Costs - Server sizing

- Type of storage ( e.g. HSM )Software Costs – SAP R/3 Data Archiving / DART / ArchiveLink

- Nil ( Part of standard SAP application )

Software Costs – Storage system - DBMS Solution - Net working - Storage system software ( e.g.

IXOS )

Installation costs - Service Charges- Consulting fees- Training costs

Consulting costs - Project resource requirements - Project timelines- Number of modules in use- Complexity of business environment- Custom development for Data

Retrieval

5.5. Maintenance Costs

Costs Influencing factors Server upgrade costs Network upgrade costs Personnel costs

Hardware costs / installation Software upgrade costs Consulting resources Employee / outsourcing expense

5.6. Savings and cost reduction

Considering the importance of data in today’s business , Data Archiving projects in most cases considered as part of a enterprise wide Data Management Strategy. Corporate Information management and retention policy also plays an important role in deciding archiving of Data.

In a financial Cost Benefit analysis , the following are the key benefits accruing from data archiving projects.

4/8/2023

Savings Key factorsServer upgrade costs( Avoidance / Postponement )

- Hardware costs- Installation costs- Network upgrade costs- DBA expenses- Consulting expenses

Maintenance costs- DBA costs- Storage media costs- Consulting resources

Back up Costs - Back up tapes- DBA expenses

Improved Performance and system availability

- Reduced down times- Reduced wait times for users- Less wait time for customers

6. Critical Success Factors

6.1. Experienced Project TeamArchiving of business data is a highly critical and complex process and one of the major factor in ensuring it’s success is the experience of the project team in implementing Archiving projects.

Data Archiving team has to have the right combination of SAP application, technical and data archiving knowledge in addition to the business functional knowledge. Understanding of the multitude of issues involved becomes critical in understanding dependencies of data, retention times, archive triggers and deciding implementation plan.

6.2. Involving all effected peopleAnother key to successful implementation of Data Archiving projects is to turn the people effected into people involved from the beginning of the project. It is important to take all the stakeholders into confidence and involve them in different project phases. Stake holders could valuable inputs in developing Business blueprint, designing and testing of the system.

6.3. Communication and Change ManagementData Archiving projects effects almost every department in an organization, especially with complex objects in CO like CO_COSTCTR. With far reaching implications, it is important to give due priority to change management and communication aspects.

Deciding of retention times for data, introduction of new user interfaces to access archived data are some areas where the change management and communication skill of the project team would be put to test.

4/8/2023

6.4. Top Management commitmentThe importance of commitment from top management for any successful project cannot be over emphasized, and more so in a Data Archiving project. Many times data archiving effects the way middle and senior management does reporting from the SAP system , which adds to the importance of their commitment.

Another factor that adds importance to the commitment of top management is the fact that, often, the stakeholders effected by an archiving project does not find anything in it motivating them to give their commitment.

6.5. Project Management ExpertiseExperience in managing complex information system projects is an important factor that will contribute to the success of archiving projects. The project manager should have expertise in managing SAP projects and preferably Data Archiving projects.

6.6. Right residence and Retention timesTo ensure the availability of data for internal and external requirements of an organization, it is important to have the correct residence and retention time assigned to it. This assumes enormous importance when we consider the fact that the data being archived are business critical information, non availability of which could effect business functioning.

6.7. Correct Sequence of objectsDependency of business processes with each other also results in dependency of the documents created by the processes. While archiving business data it is critical to take this factor into account to ensure integrity and consistency of data.

7. Limitations of data archiving

7.1. Data Archiving is component specificAfter archiving, the data remain linked to the corresponding component. Data cannot be transferred or processed by other components. One business that involves several application components cannot be archived as one over all process.

7.2. Not suitable for removing organizational unitsData archiving is not suitable for deleting organizational units from the system. This operation could be requested if a plant or a company division is sold and the corresponding data was no longer required to be maintained. Data archiving cannot simply remove specific sub-areas from a system, because the archiving objects rarely contain the exact data that needs to be removed.

8. GlossarySAP Data Archiving

4/8/2023

SAP Data Archiving is the process in which business objects such as accounting documents and material master records are selectively removed from the database in such a way that the user does not have to worry about the underlying physical table design.

Archiving object

Set of interdependent business data that is periodically extracted from the current system and archived according to individual criteria. E.g. FI_DOCUMNT (Financial Accounting Document)

Optical archiving

Electronic storage and management of documents such as original documents, outgoing documents, print lists in an external storage system that is usually based on optical media (CDs, WORMS, and so on).

Reorganization

Reorganization refers to the restructuring of database tables with the aim of using memory space efficiently.

Residence period

Period that must elapse before application data can be archived. The residence period can be based on various application-specific data, such as creation data, posting period, goods issue date. The period can be specified in days, weeks, months or years.

Retention period

Total period of time that data is held in the database including in an Archive. Data that pass the retention time is considered expired and hence destroyed.

Restore

Rewriting the backup copy to the database to recreate the original sate of the database after a system termination.

SAP ArchiveLink

Data interface that is embedded in the Basis component and that controls communication with an external storage system. Enables access to stored documents from the SAP application.

4/8/2023

sap data archiving-decision making guide

Documents