research business technology pfizer enterprise elastic hpc mike miller pfizer research business...

Research Business Technology

Pfizer Enterprise Elastic HPC

Mike Miller

Pfizer Research Business Technology

May 18th Prism Meeting

Stockholm Sweden


How do we define HPC?

2

• Simply summarized as the computational laboratory• Consists of:

• Desktop/Services, integrated with• Global high performance cached file system • Centralized large capacity/capability compute resources

• Used by:• Direct

• 300-400 expert computational scientists in chemistry, biology, DMPK, stats, pharm sci & clin pharm

• Indirect• >2000 lab scientists using desktop apps that utilize HPC compute


The Evolution of HPC at Pfizer

3

2004 150 blades (300 cores)

2000 SGI Origins (128 cores)

2009 6 x3950 (520 cores)

2010 on-demand Amazon VPC


4

Intersection of “The Cloud” and HPC


Pfizer VPC Overview

• The Pfizer Virtual Private Cloud (pilot effort) has been

implemented an extension of our physical data center.

• Infrastructure as a service affords rapid provisioning

without compromising on:

– Security

– Compatibility

– Accessibility

– Agility

– Utility

• Implementation

Groton DMZ AmazonWeb Services

CloudSecure VPN Connection over the Internet

Subnets

Pfizer’s isolated VPC resources

RouterVPN Gateway

AWS Virginia DC


Feature AWS Internal VM’s

Data Center

Required to be joined to the Pfizer networkSecurity Monitoring

PublicConfidentiality

$0 mid-10’s $ Low 1000’s $Provisioning Costs

AMI/Xen XenVMWare Bare Metal

Avail. Config.

1 hr 4 hrs 2-8 wksProvisioning SLA

100-1000s 10-100s 1-10sRequest Capacity/Wk

low-10’s $ high-10’s$ Low-100’s $Runtime/Depreciation

Support Model 8x57x24Self / incident

OS ConfigurationsSolaris,

AS 400Windows server 2003/2008

Linux REHL 5.x

Environment POC

HPC HPC

Support SLAs None 24 hrImmediate

1 hr. 1 mo. 6 mo.Min. Billable Period

Controls Black Box System root level access Qualified / Validated

Stand AloneModerate

Complexity SimpleHigh

Dev / Test ProdCom

putin

g R

equi

rem

ents

com

e in

Man

y fo

rms

low med high


Security

• Amazon practices & security measures successfully met audit criteria for Research level use

• Pfizer employed the same security systems used internally– IP-sec tunnels in to AWS

– Pfizer Global Active Directory• Joining machines and managing permissions

– Linux & Windows


Compatibility

• To get the most benefit from the cloud it was necessary to align AWS resource offerings with existing internal systems:– AMI’s (VM) Pfizer Qualified RHEL 5 image

• Centrify/AD provides identification/authorization • Kerberos credentials via AD

– File cache (storage) OpenAFS volumes accessible– IP mappings Pfizer DNS

• AMI’s have Pfizer network identities & are discoverable– Allows AMI’s to be part of our LSF cluster– Users can do development work accessing the full range of Pfizer

resources• e.g. Software licenses utilize the pfizer flexlm server


Availability

• AD & DNS give us full range of access to internal systems– LSF for job scheduling

– Oracle / mySQL instances for accessing structured data

– AFS for secure access to unstructured data• High performance via local caching

– Access to licensed and internally developed software


Agility

• The $50M decision– Required completion of a time sensitive

chemoinformatics task• Workload was diverted from internal resources so they

could be dedicated.

• Within 30 min 64 cores were spun up and joined to LSF

• For 4 days >50,000 jobs were executed

• Total cost <$1,500

– Results were obtained on-time and the decision taken


Utility

• Internal Application Development– Tomcat web applications– Nightly builds & regression testing

• HPC capacity– Over 250 apps are accessible– LSF uses resource specifications to determine

suitability and schedules jobs accordingly• Over 100,000 jobs run

– QM, ab initio

– Virtual screening

– Systems biology


Implementation

• From PoC Production– Provisioning, exploring commercial solutions that

enable:• One-time actions

– Integrate with our procurement system• Move to a debit (pre-allocated funding) model

– Standard configurations

• Repeatable actions– Start/ Stop instances via a user centric dashboard

• User’s manage / are accountable for the resources they use

• LSF– Custom code

• detect workload• Start / Stop AMI’s• Leverage accounting

research business technology pfizer enterprise elastic hpc mike miller pfizer research business...

Documents