year in review: perforce 2014 product updates

Year in Review: Perforce 2014 Product Updates

Charles McLouthDirector of Solutions Engineering

Presented By:

Charlie McLouthDirector of Solutions Engineering

Charlie has over 15 years of

experience with technologies such

as RDBMS, web applications, back

office applications, and web services.

He has held positions ranging from

Developer to Product Manager

before joining Perforce 8 years ago.

Agenda

• Server Optimizations

• Features for End Users

• Collaboration Features with Swarm

SERVER OPTIMIZATIONS

Server Performance and Scalability

• Concurrency Optimizations – aka Lockless

Reads

• Federated Architecture for Distributed Teams

• Clustering for Horizontal Scalability

• Clustering for High Availability

• Sync Performance

SERVER OPTIMIZATIONS

Concurrency – aka Lockless Reads

SERVER OPTIMIZATIONS

Reader

Writer

Database

p4 submit

p4 changes

Lockless Reads

Writer does not block

Reader

✗

VMware

Before After

Total Commands 3,518,462 3,211,786

Total write-wait-time 58,468s

Total read-wait-time 1,033,957s

SERVER OPTIMIZATIONS

VMware with 5,000 users(24 hour period)

3,430s

992s 1000X

17X

VMware – Command Delays

SERVER OPTIMIZATIONS

1,749

00

500

1,000

1,500

2,000

2,500

3,000

Seconds ofdelay

1 2 4 8 16 32 64 128 256

Commands affected by delays

Commands affected, before Lockless

Commands affected, after Lockless

Federated Architecture for Distributed Teams

• Commit/Edge modes for server

• Local metadata with edge

services

• Submits propagated to commit

service

• Other commands execute locally

Commit

Edge Edge

SERVER OPTIMIZATIONS

Performance Gains with Federation

56.6

8.7

61

9.8

0.9 0.7 1.8 0.20

10

20

30

40

50

60

70

Sync Revert Integrate Edit

Direct Edge

SERVER OPTIMIZATIONSPerforce Lab Benchmarks

Clustering for Horizontal Scalability

• Distribute workload to multiple nodes

• Better performance and experience for all users

From This…

100%Activity

P4D

To This…

Commit

2%

Workspace

Server

34% 33% 33%User Activity User Activity User ActivitySystem Activity

Workspace

Server

Workspace

Server

SERVER OPTIMIZATIONS

Horizontal Scalability Results

• 100,000 simulated developers

• 742 commands/sec

• 8 node cluster

Commit

WS WS WS WS

WS WS WS WS

SERVER OPTIMIZATIONS

Clustering – Failover

SERVER OPTIMIZATIONS

P4D P4D

NAS

Router

P4DDepot

MasterDepot

Standby

Depot

Master

LAN

Sync Performance

• Sync Performance Increased

– Network window size optimization

– Transfer multiple times at a time – aka Parallel Sync

SERVER OPTIMIZATIONS

Network Window Size Optimization

SERVER OPTIMIZATIONS

Network Window Size Optimization

SERVER OPTIMIZATIONS

Parallel Sync

SERVER OPTIMIZATIONS

Sync Performance Results

• Network window size optimization

– 1/16th the amount of network round trips

• Parallel Sync

– 1/4th the amount of time

• Results of both

– Syncs up to 20 times faster

SERVER OPTIMIZATIONS

LDAP For User Authentication

• Perforce provides Authentication Trigger entry points for lookup into

third party user database (including LDAP)

• auth_check trigger works well, but ...

– Needs to be installed separately

– No standard (Python, Perl, C++ implementations)

– One more headache for administrators

• Most common request on P4Ideax:

– Perforce should provide built-in LDAP integration

SERVER OPTIMIZATIONS

Native LDAP support

SERVER OPTIMIZATIONS

LDAP

p4 loginuser-login

client-PromptEnter Password:

<password> dm-login

LDAP Query

<accepted>

client-SetPassword

User logged in.

Native LDAP support

• Increases Security

– Consistent Authentication and Authorization

• Simplifies Policy Management

– Access granted/revoked by LDAP group membership

SERVER OPTIMIZATIONS

P4Search

SERVER OPTIMIZATIONS

???

File names,

Changes ...

File content?

C#

.h

JAVA

PPTX

PDF

P4 Search

Solution: External Index

SERVER OPTIMIZATIONS

storesearch

Search engine indexes content

Stores it in its own database

Users search the index first

Index returns a depot path

Permissions applied to index

results

Returns results to user

P4Search

SERVER OPTIMIZATIONS

• File content

• P4 Metadata

• P4 attributes

• And the common

metadata if

desired

END USERS

P4V – 2014 New Features

• User interface refresh

• Remove excess baggage

• Simplify experience

• Faster rendering

• Faster for large file sets

• “Snappier” responsiveness

END USERS

P4V – 2014 New Features

END USERS

P4V – 2014 New Features

END USERS

P4V – 2014 New Features

END USERS

Streams

END USERS

• Model the flow of change for

the entire product lifecycle

• Visual roadmap for users

• Remove the fear/overhead

of branching and merging

Streams – 2014 Features

END USERS

• Streams allows “import”

paths to include

components from one

Stream into another.

– Import is by reference (not a

copy)

– Import path is Read Only

Streams – 2014 Features

END USERS

Top Requests of Streams to

facilitate managing Components

with Streams:

– “I want to lock an import a

specific version of my

component.”

– “I want to submit changes to a

referenced component.”

• Makes managing components

simpler

Streams – Frozen Imports

Paths:

import import/... //depot/import/…@1000

• Freezes the import path (component) at a point in time.

• p4 sync import/…#head behaves like p4 sync import/…@1000

END USERS

Streams – Writeable Imports

Paths:

import+ import/... //depot/import/…

• Functions like standard import, enabling you to map a path or

Component from outside the stream.

• But unlike a standard import path, you can submit changes to the

files in an import+ path.

END USERS

Streams – 2014 Features

35END USERS

• Frozen Imports and Writeable

Imports

– Simplifies the management of

Components within a Stream

SWARM COLLABORATION

Swarm – 2014 Features

SWARM COLLABORATION

• Review Votes

• Required Reviewers

• Lightweight Tasks

• Attachments

• API

Swarm – 2014 Features

SWARM COLLABORATION

Swarm – 2014 Features

SWARM COLLABORATION

Swarm – 2014 Features

SWARM COLLABORATION

Swarm – 2014 Features

SWARM COLLABORATION

Swarm - Comment Attachments

SWARM COLLABORATION

Swarm – Rest API

SWARM COLLABORATION

Questions?

Charlie McLouth

cmclouth@perforce.com

• Resources

• Videos of all MERGE 2014 presentations:

perforce.com/MERGE

• Sign up for Monthly Product Updates:

perforce.com/communication-center

• Blog posts from Perforce product teams:

perforce.com/blog