real time replication in the real world - perforce ?· real time replication in the real world...

Download Real Time Replication in the Real World - Perforce ?· Real Time Replication in the Real World Richard…

Post on 10-Mar-2019

213 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

Doc Revision 1.1

Real Time Replication in the Real World

Richard E. Baum

and

C. Thomas Tyler

Perforce Software, Inc.

Copyright 2010 Perforce Software i

Table of Contents 1 Overview ........................................................................................................................ 1 2 Definitions ...................................................................................................................... 1

3 Solutions Involving Replication ..................................................................................... 2 3.1 High Availability Solutions ..................................................................................... 2

3.1.1 High Availability Thinking .............................................................................. 2 3.2 Disaster Recovery Solutions ................................................................................... 2

3.2.1 Disaster Recovery Thinking ............................................................................. 3

3.3 Read-only Replicas .................................................................................................. 3

3.3.1 Read-only Replica Thinking ............................................................................ 3

4 Perforce Tools that Support Metadata Replication ........................................................ 4 4.1 Journal Truncation ................................................................................................... 4 4.2 p4jrep (deprecated) .................................................................................................. 4

4.3 p4 replicate .................................................................................................... 4

4.3.1 Replication to Journal File ............................................................................... 5

4.3.2 Replication to live DB files .............................................................................. 5 4.3.3 Filtering Example ............................................................................................. 6

4.4 p4 export ........................................................................................................... 6

4.4.1 Report Generation from Scripts ....................................................................... 6 4.4.2 Report Generation from SQL Databases.......................................................... 6

4.4.3 Full Replication (Metadata and Archive Files) ................................................ 6

4.4.4 Daily Security Reports ..................................................................................... 6 4.5 Built-in Replication Tools - Summary .................................................................... 7

5 Tools that Support Archive File Replication ................................................................. 7

5.1 Rsync / Robocopy ................................................................................................... 7 5.2 Filesystem or Block-Level Replication ................................................................... 7

5.3 Perfect Consistency vs. Minimum Data Loss ......................................................... 7 6 Putting it All Together ................................................................................................... 8

6.1 Classic DR: Truncation + Commercial WAN Replicator ...................................... 8

6.2 The SDP Solution Using p4 replicate ........................................................... 9

6.2.1 Failover .......................................................................................................... 10

6.3 A Read-only Replica for Continuous Integration .................................................. 10

6.3.1 Define how Users Will Connect to the Replica. ............................................ 10 6.3.2 Use Filtered Replication ................................................................................. 10 6.3.3 Make Archive Files Accessible (read-only) to the Replica ........................... 11

7 Operational Issues with Replication............................................................................. 11 7.1 Obliterates: Replication vs. Classic Journal Truncation ...................................... 12

8 Summary ...................................................................................................................... 12 9 References & Resources .............................................................................................. 13

Copyright 2010 Perforce Software Page 1

1 Overview There are myriad ways to configure a Perforce environment to allow for multiple,

replicated servers. Configurations are chosen for a wide variety of reasons. Some provide

high availability solutions. Some provide disaster recovery. Some provide read-only

replicas that take workload off of a main server. There are also combinations of these.

What you should deploy depends on your desired business goals, the availability of

hardware and network infrastructure, and other a number of factors.

The 2009.2 release of Perforce Server provides built-in tools to allow for near-real-time

replication of metadata. These tools allow for much easier implementation of both read-

only replica servers and high-availability/disaster-recovery solutions. This paper discusses

some of the most common replication configurations, the ways they are supported by the

Perforce Server, and characteristics of each.

2 Definitions A number of terms are used throughout this document. They are defined as follows:

HA High Availability A system design protocol and associated implementation that ensures a certain degree of operational continuity during a given measurement

period, even in the event of certain failures of hardware or software components.

DR Disaster Recovery The process, policies and procedures related to preparing for recovery or continuation of technology infrastructure critical to an organization

after a natural or human-induced disaster.

RPO Recovery Point Objective Describes an acceptable amount of data loss measured in time.

RTO Recovery Time Objective The duration of time and a service level within which service must be restored after a disaster.

Metadata Data contained in the Perforce database files (db.* files in the P4ROOT).

Archive Files All revisions of all files submitted to the Perforce Server and currently shelved files.

Read-only Replica A Perforce Server instance that operates using a copy of the Perforce metadata.

Copyright 2010 Perforce Software Page 2

DRBD (Distributed Replicated Block Device) a distributed storage system available as of the 2.6.33 kernel of Linux. DRBD runs over a network and works

very much like RAID 1.

3 Solutions Involving Replication

3.1 High Availability Solutions High Availability solutions keep Perforce servers available to users, despite failures of

hardware components. HA solutions are typically deployed in environments where there is

little tolerance for unplanned downtime. Large sites with 24x7 uptime requirements due to

globally distributed development environments strive for HA.

Perforce has excellent built-in journaling capabilities. It is fairly easy to implement a

solution that is tolerant to faults, prevents data loss in any single point of failure situation,

and that limits data loss in more significant failures. With HA, prevention of data loss for

any single point of failure is assumed to be accounted for, and the focus is on limiting

downtime.

HA solutions generally offer a short RTO and low RPO. They offer the fastest recovery

from a strict hardware failure scenario. They also cost more, as they involve additional

standby hardware at the same location as the main server. Full metadata replication is also

used, and the backup server is typically located on the same local LAN as the main server.

This results in good performance of replication processes. Due to the proximity of the

primary and backup servers, these solutions offer do not offer much in the way of disaster

recovery. A site-wide disaster or regional problem (earthquake, hurricane, etc.) can result

in a total outage.

3.1.1 High Availability Thinking

The following are sample thoughts that lead to the deployment of HA solutions:

Were willing to invest in a more sophisticated deployment architecture to reduce unplanned downtime.

We will not accept data loss for any Single Point of Failure (SPOF).

Downtime is extremely expensive for us. We are willing to spend a lot to reduce the likelihood of downtime, and minimize it when it is unavoidable.

3.2 Disaster Recovery Solutions In order to offer a true disaster recovery solution, a secondary server needs to be located in

a site that is physically separate from the main server. Full metadata replication provides a

reliable failover server in a geographically separate area. Thus, if one site becomes

unavailable due to a natural disaster, another can take its place. As WAN connections are

Copyright 2010 Perforce Software Page 3

often considerably slower than local area network connections, these solutions tend to have

a higher RTO and a longer R

Recommended

View more >