ipsec add-on for pcf - resources.docs.pivotal.iogoogle cloud platform (gcp), vsphere, azure, amazon...

IPsecAdd-onforPCF®

Version1.6

User'sGuide

©2018PivotalSoftware,Inc.

234

11121315212325

TableofContents

TableofContentsIPsecAdd-onforPCFInstallingtheIPsecAdd-onforPCFUpgradingtheIPsecAdd-onforPCFUninstallingtheIPsecAdd-onforPCFCheckingCertificateDatesTroubleshootingtheIPsecAdd-onforPCFRotatingActiveIPsecCertificatesRenewingExpiredIPsecCertificatesReleaseNotes

©CopyrightPivotalSoftwareInc,2013-2018 2 1.6

IPsec Add-on for PCFPage last updated:

ThisguidedescribestheIPsecAdd-onforPCF,whichsecuresdatatransmissionsinsidePivotalCloudFoundry (PCF).TopicscoveredinthisguideincludeIPsecAdd-onforPCFinstallationandconfiguration,troubleshooting,andcertificaterotation.

YourorganizationmayrequireIPsecifyoutransmitsensitivedata.

OverviewTheIPsecAdd-onforPCFprovidessecuritytothenetworklayeroftheOSImodelwithastrongSwan implementationofIPsec.TheIPsecAdd-onprovidesastrongSwanjobtoeachBOSH-deployedvirtualmachine(VM).

IPsecencryptsIPdataflowbetweenhosts,betweensecuritygateways,andbetweensecuritygatewaysandhosts.TheIPsecAdd-onforPCFsecuresnetworktrafficwithinaCloudFoundrydeploymentandprovidesinternalsystemprotectionifamaliciousactorbreachesyourfirewall.

IPsecImplementationDetailsTheIPsecAdd-onforPCFimplementsthefollowingcryptographicsuite:

Key Agreement (Diffie-Hellman) IKEv2MainMode

Bulk Encryption AES128GCM16

Hashing SHA2 256

Integrity/Authentication Tag 128bitGHASHICV

Digital Signing RSA3072/4096

Peer Authentication Method Public/PrivateKey

RefertothefollowingtopicsformoreinformationabouttheIPsecadd-on:

InstallingtheIPsecAdd-onforPCF

RotatingIPsecCertificates

RenewingExpiredIPsecCertificates

TroubleshootingtheIPsecAdd-onforPCF

UpgradingtheIPsecAdd-onforPCF

UninstallingtheIPsecAdd-onforPCF

ReleaseNotes


https://network.pivotal.io/products/pivotal-cf

https://www.strongswan.org/

Installing the IPsec Add-on for PCFPage last updated:

ThistopicdescribeshowtoprepareyournetworkforIPsec,createanIPsecmanifest,andaddIPsectoyourdeployment.

PrerequisitesTocompletetheIPsecinstallation,verifythatyouhavesatisfiedthefollowingprerequisitesbeforeyoubegin:

GoogleCloudPlatform(GCP),vSphere,Azure,AmazonWebServices(AWS),orOpenStackasyourIaaS

PivotalCloudFoundry(PCF)operatoradministrationrights

BOSHdeployedthroughOpsManager1.7orlater

SettheMTUforyourIaaSintheElasticRuntimetile,underNetworking.PivotalrecommendsMTUvaluesof1354onGCP,1438onAzure,andthedefaultvaluesonAWSandvSphere.ForOpenStack,followtherecommendationsofyourNeutron/ML2 pluginprovider,orempiricallytestthecorrectMTUforyourenvironment.

LimitationsTheIPsecadd-onv1.6doesnotworkonWindows.

BestPracticesIPsecmayaffectthefunctionalityofotherservicetiles.Asaresult,PivotalrecommendsdeployingElasticRuntimeandeachservicetiletodifferentisolatedsubnets.Alternatively,youcanminimallydeployallservicetilestoasingleisolatedsubnet,apartfromtheElasticRuntimesubnet.SomeservicetilesdonotsupportIPsecandmustbeplacedinanon-IPsecsubnet.

ForIPsec,PivotalrecommendsanyUbuntustemcellsforvSphere,OpenStack,andHVMstemcellsforAWS.ThesestemcellsareavailableonPivotalNetwork .IfyouusePVstemcellsobtainedfrombosh.io ,seethePacketLosssectionoftheTroubleshootingtheIPsecAdd-onforPCFtopictoadjustMTUvalues.

ConfigureNetworkSecurityRefertotheappropriatesectionbelowforyourIaaSnetworkconfigurationdetails.

GoogleCloudPlatformToconfigureyourGoogleCloudPlatform(GCP)environmentforIPsec,performthefollowingsteps:

1. NavigatetotheNetworkingsectionoftheGCPConsole.

2. ClickFirewall rules.

3. ClickCreate Firewall Rule.

4. ForName,enter ipsec .

5. ForNetwork,selectthenetworkwhereOpsManagerisdeployed.Forexample,opsmgr.

6. ForSource filter,selectAllow from any source (0.0.0.0/0).

7. ForAllowed protocols and ports,enter udp:500; ah; esp .

8. ClickCreate.

9. AdjusttheMTUvalueto 1354 byperformingtheprocedureinthePacketLosssectionoftheTroubleshootingtheIPsecAdd-onforPCFtopic.


https://wiki.openstack.org/wiki/Neutron/ML2

https://network.pivotal.io/products/stemcells

https://bosh.io

vSphereConfirmthatyournetworkallowstheprotocolslistedinthetablebelow.

Protocol Name Protocol Number Port(s)

AH 51 Any

ESP 50 Any

UDP 17 500

Azure1. Confirmthatyournetworkallowstheprotocolslistedinthetablebelow.

Protocol Name Protocol Number Port(s)

AH 51 Any

ESP 50 Any

UDP 17 500

2. AdjusttheMTUvalueto 1438 .Forinstructions,seeExplanation:PacketLoss.

AWSToconfigureyourAWSenvironmentforIPsec,performthefollowingsteps:

1. NavigatetoEC2 Dashboard > Security Groups.

2. SelecttheSecurityGroupwiththedescriptionPCF VMs Security GroupandclickEdit.

3. CreatethefollowingInbound Rules.

Type Protocol Name Protocol Number Port Range Source

CustomProtocol AH 51 All 10.0.0.0/16

CustomProtocol ESP 50 All 10.0.0.0/16

CustomUDPRule UDP 17 500 10.0.0.0/16

OpenStack

ToconfigureyourMirantisOpenStackenvironmentforIPsec,performthefollowingsteps:

1. NavigatetoProject / Access & Security.

2. SelectthesecuritygroupandclickManage Rules.

3. CreatethefollowingIngress and Egress Rules.AdjustthesourceCIDRasneededforyourenvironment.

Protocol Name Protocol Number Port Range Source

ESP 50 Any 0.0.0.0/0

AH 51 Any 0.0.0.0/0

Note:ThedefaultPCF VMs Security Groupistypicallyspecifiedwithasubnetof 10.0.0.0/16 .IfyourPCFsubnetisdeployedtoadifferentCIDRblock,adjustthesourceasneeded.

Note:ThefollowingnetworkconfigurationisoptimizedforMirantisOpenStack,butotherOpenStackdistributionshaveasimilarworkflow.


UDP 17 500 0.0.0.0/0

CreatetheIPsecManifestFollowthesestepstocreatetheIPsecmanifestforyourdeployment:

1. CreateanIPsecmanifestfile ipsec-addon.yml ,startingwiththecodebelowasatemplate.

releases:- {name: ipsec, version: 1.0.0}

addons:- name: ipsec-addon jobs: - name: ipsec release: ipsec include: stemcell: - os: ubuntu-trusty properties: ipsec: optional: false ipsec_subnets: - 10.0.1.1/20 no_ipsec_subnets: - 10.0.1.10/32 # bosh director - 10.0.1.4/32 # ops manager instance_certificate: | -----BEGIN CERTIFICATE----- MIIEMDCCAhigAwIBAgIRAIvrBY2TttU/LeRhO+V1t0YwDQYJKoZIhvcNAQELBQAw ... -----END CERTIFICATE----- instance_private_key: | -----BEGIN EXAMPLE RSA PRIVATE KEY----- EXAMPLExRSAxPRIVATExKEYxDATAxEXAMPLExRSAxPRIVATExKEYxDATA ... -----END EXAMPLE RSA PRIVATE KEY----- ca_certificates: - | -----BEGIN CERTIFICATE----- MIIFCTCCAvGgAwIBAgIBATANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDEwl0ZXN0 ... -----END CERTIFICATE----- - | -----BEGIN CERTIFICATE----- MIIFCTCCAvGgAwIBAgIBATAAYDVQQDEwl0ZXN0NBgkqhkiG9w0BAQsFADAUMRIwE ... -----END CERTIFICATE----- prestart_timeout: 30 esp_proposals: aes128gcm16! ike_proposals: aes128-sha256-modp2048!

2. Replacethevalueslistedinthetemplateasfollows:releases: - version :SpecifytheversionnumberofyourIPsecdownloadfromPivotalNetwork.jobs: - name :Donotchangethenameofthisjob.Itmustbe ipsec .include: stemcell - os :DonotchangetheOStype.IPsecisonlysupportedon ubuntu-trusty .optional :MakesIPsecenforcementoptional.ToaddIPsec,setthisflagto true .OnceIPsechasbeensuccessfullyinstalled,setthisflag

backto false andredeploy.ipsec_subnets :Listthesubnetsthatyouwanttobeencrypted.Youcanincludetheentiredeploymentoraportionofthenetwork.Encrypt

anynetworkthathandlesbusiness-sensitivedata.no_ipsec_subnets :ListtheIPaddressofyourBOSHDirectorandOpsManagerVM,alongwithanyotherIPaddressesinyourPCFdeployment

thatyouwanttocommunicatewithoutencryption.PivotalrecommendsthatyoulistthesubnetsthatareusedforPCFmanagedservices.SubnetsforPCFmanagedservicesthatdonotsupportIPsecmustbelistedunder no-ipsec .

instance_certificate :CopyinthesignedcertificatethatwillbeusedbyallyourinstanceVMs.YoumustuseoneoftheCAsintheca_certificatespropertytosignthiscertificate.Foradevelopmentortestenvironment,youcanuseaself-signedcertificate.SeetheGenerateaSelf-SignedCertificatesectionofthistopicformoreinformation.instance_private_key :Copyintheprivatekeythatcorrespondstotheinstance_certificateabove.Thiskeymustnotuseapassphrase.ca_certificates :CopyinCAcertificatesfortheinstanceVMtotrustduringthevalidationprocess.Inmostcases,youonlyneedtheCA

Note:InGCP,ifyouusethedefaultrouterforDNSinsteadoftheGooglepublicDNSat 8.8.8.8 ,youmustaddtheIPaddressofthedefaultrouterinyoursubnetto no_ipsec_subnets .Forexample, 10.0.0.1/32 .


certificateusedtosigntheinstancecertificate.DuringCAcertificaterotation,youneedtwoCAcertificates.prestart_timeout :Youcanmodifythe30seconddefaultprestarttimeoutvalue.ThisvaluelimitsthenumberofsecondsallowedforIPsecto

startbeforefailingtheattempt.esp_proposals :YoucanmodifytheESP(QuickMode)encryptionandintegrityalgorithms.Thedefault, aes128gcm16! ,is128bitAES-GCM

with128bitICVforbothencryptionandintegrity.ike_proposals :YoucanmodifytheIKE(MainMode)encryptionandintegrityalgorithms,andtheDiffie-Hellmangroup.Thedefault,aes128-sha256-modp2048! ,is128bitAES-CBCforencryption,SHA2_256_128HMACforintegrity,andGroup14forDiffie-Hellman.

DownloadandDeploytheIPsecAdd-onAfterdeployingOpsManager,performthefollowingstepstodownloadanddeploytheIPsecadd-on:

1. DownloadtheIPsecadd-onsoftwarebinaryfromthePivotalNetwork toyourlocalmachine.

2. CopythesoftwarebinarytoyourOpsManagerinstance.

$scp-iPATH/TO/PRIVATE/KEYipsec-release.tar.gzubuntu@YOUR-OPS-MANAGER-VM-IP:

3. CopytheIPsecmanifestfiletoyourOpsManagerinstance.

$scp-iPATH/TO/PRIVATE/KEYipsec-addon.ymlubuntu@YOUR-OPS-MANAGER-VM-IP:

4. SSHintoOpsManager.

$ssh-iPATH-TO-PRIVATE-KEYubuntu@YOUR-OPS-MANAGER-VM-IP

5. OntheOpsManagerVM,navigatetothesoftwarebinarylocationinyourworkingdirectory.

$cdPATH-TO-BINARY

6. LogintotheBOSHDirector.For Ops Manager v1.10 or earlier:

i. OntheOpsManagerVM,targettheinternalIPaddressofyourBOSHDirector.Whenprompted,enteryourBOSHDirectorcredentials.ToretrieveyourBOSHDirectorcredentials,navigatetoOpsManager,clicktheCredentialstab,andclickLink to CredentialnexttoDirector Credentials.Forexample:

$bosh--ca-cert/var/tempest/workspaces/default/root_ca_certificatetargetYOUR-BOSH-DIRECTOR-INTERNAL-IPTargetsetto'p-bosh'Yourusername:directorEnterpassword:******************Loggedinas'director'

For Ops Manager v1.11 or later:

i. OntheOpsManagerVM,createanaliasintheBOSHCLIforyourOpsManagerDirectorIPaddress.Forexample:

$bosh2alias-envmy-env-e10.0.0.3

ii. LogintotheBOSHDirector,specifyingthenewlycreatedalias.Forexample:

$bosh2-emy-envlog-in

7. Uploadyourrelease,specifyingthepathtothetarballedIPsecbinary,byrunningoneofthefollowingcommands:For Ops Manager v1.10 or earlier:

$boshuploadreleasePATH-TO-BINARY/BINARY-NAME.tar


Note:Tomodifytheconfigurationinanexistingdeployment,youmustupdatethemanifestfileandredeploy.


https://network.pivotal.io/products/p-ipsec-addon

$bosh2-emy-envupload-releasePATH-TO-BINARY/BINARY-NAME.tar

8. Listthereleasesbyrunningoneofthefollowingcommands,andconfirmthattheIPsecbinaryfileappears:For Ops Manager v1.10 or earlier:

$boshreleases


$bosh2-emy-envreleases

9. UpdateyourruntimeconfigurationtoincludetheIPsecadd-on.For Ops Manager v1.10 or earlier:

$boshupdateruntime-configPATH/bosh-manifest.yml


$bosh2-emy-envupdate-runtime-configPATH/bosh-manifest.yml

10. VerifyyourruntimeconfigurationchangesmatchwhatyouspecifiedintheIPsecmanifestfile.For Ops Manager v1.10 or earlier:

$boshruntime-config


$bosh2-emy-envruntime-config

Forexample,

$bosh2-emy-envruntime-configActingasuser'admin'on'micro'

releases:-{name:ipsec,version:1.0.0}

addons:name:ipsec-addonjobs:-name:ipsecrelease:ipsec...

11. IfyouhavenotinstalledtheElasticRuntimetile,followthesesteps:

a. NavigatetoyourInstallation DashboardinOpsManager.b. ClickApply Changesc. DeployElasticRuntimebyfollowingtheinstallationinstructionsforyourIaaS.SeetheInstallingPivotalCloudFoundry topicformore

information.

12. IfyouhavealreadydeployedElasticRuntimeandareaddingIPsectoanexistingdeployment,followthesesteps:

a. Setthe optional flagto true .b. NavigatetoyourInstallation DashboardinOpsManager.c. ClickApply Changesd. Waitfortheinstallationtocomplete.e. Setthe optional flagto false .f. Updatetheruntimeconfig.

For Ops Manager v1.10 or earlier:

$boshupdateruntime-configPATH/bosh-manifest.yml



http://docs.pivotal.io/pivotalcf/installing/index.html

$bosh2-emy-envupdate-runtime-configPATH/bosh-manifest.yml

g. NavigatetoyourInstallation Dashboard.h. ClickApply Changes.

13. Securethesensitiveinformationinthe ipsec-addon.yml file.Pivotalrecommendsencryptingthefileandmovingittoasecurelocation.

VerifyYourIPsecInstallationAfterinstallingIPsecanddeployingElasticRuntime,performthefollowingstepstoverifyyourIPsecinstallation:

1. ListthejobVMsinyourdeploymentbyrunningoneofthefollowingcommands:For Ops Manager v1.10 or earlier:bosh vms

For Ops Manager v1.11 or later:bosh2 -e BOSH_ENVIRONMENT vms

2. OpenanSSHconnectionintotheVM,usingthejobnameandindexofanyVMfoundabove,byrunningoneofthefollowingcommands:For Ops Manager v1.10 or earlier:bosh ssh JOB-NAME/INDEX

For Ops Manager v1.11 or later: bosh2 -e BOSH_ENVIRONMENT -d DEPLOYMENT_NAME ssh JOB-NAME/INDEX

3. Run sudosu- toentertherootenvironmentwithrootprivileges.

4. Run monitsummary toconfirmthatyour ipsec jobislistedasa bosh job.

TheMonitdaemon5.2.5uptime:18h32m...Process'ipsec'runningSystem'system_localhost'running

5. Run PATH-TO-IPSEC/ipsecstatusall toconfirmthatIPsecisrunning.IfIPsecisnotrunning,thiscommandproducesnooutput.

$/var/vcap/packages/strongswan-5.3.5/sbin/ipsecstatusallStatusofIKEcharondaemon(strongSwan5.3.5,Linux3.19.0-56-generic,x86_64):uptime:18hours,sinceMar1623:58:502016malloc:sbrk2314240,mmap0,used1182400,free1131840workerthreads:11of16idle,5/0/0/0working,jobqueue:0/0/0/0,scheduled:206loadedplugins:charonaessha1sha2randomnoncex509revocationconstraintspubkeypkcs1pkcs7pkcs8pkcs12pemgmpxcbccmachmacattrkernel-netlinksocket-defaultstrokeListeningIPaddresses:10.10.5.66Connections:ipsec-10.10.4.0/24:%any...%anyIKEv1/2ipsec-10.10.4.0/24:local:[CN=test-cert-1-ca-1]usespublickeyauthenticationipsec-10.10.4.0/24:cert:"CN=test-cert-1-ca-1"ipsec-10.10.4.0/24:remote:usespublickeyauthenticationipsec-10.10.9.0/24:child:10.10.5.66/32===10.10.9.0/24TRANSPORTno-ipsec-10.10.4.1/32:%any...%anyIKEv1/2no-ipsec-10.10.4.1/32:local:usespublickeyauthenticationno-ipsec-10.10.4.1/32:remote:usespublickeyauthenticationno-ipsec-10.10.4.1/32:child:dynamic===10.10.4.1/32PASSShuntedConnections:no-ipsec-10.10.4.1/32:dynamic===10.10.4.1/32PASSno-ipsec-10.10.5.1/32:dynamic===10.10.5.1/32PASSno-ipsec-10.10.6.1/32:dynamic===10.10.6.1/32PASSRoutedConnections:ipsec-10.10.9.0/24{6}:ROUTED,TRANSPORT,reqid6ipsec-10.10.9.0/24{6}:10.10.5.66/32===10.10.9.0/24ipsec-10.10.8.0/24{5}:ROUTED,TRANSPORT,reqid5ipsec-10.10.4.0/24{1}:10.10.5.66/32===10.10.4.0/24SecurityAssociations(45up,0connecting):ipsec-10.10.4.0/24[459]:ESTABLISHED13secondsago,10.10.5.66[CN=test-cert-1-ca-1]...10.10.4.38[CN=test-cert-1-ca-1]ipsec-10.10.4.0/24{1527}:10.10.5.66/32===10.10.4.38/32...

Note:TheexactVMdoesnotmatter,becauseinstallingtheIPsecadd-onloadsIPseconallVMsdeployedbyOpsManager.


GenerateaSelf-SignedCertificate

Togenerateaself-signedcertificateforyourIPsecmanifest,youcanuseeither openssl or certstrap .Followtheinstructionsforyourpreferredmethodbelow.

Rerunningthescriptsoverwritesyourcurrentkeysandthecertificates.

GenerateaSelf-SignedCertificatewithOpenSSL1. Download the openssl-create-ipsec-certs.sh bashscript.

2. Navigatetothedirectorywhereyoudownloadedthescript:

$cd~/workspace

3. Changethepermissionsofthescript:

$chmodu+xopenssl-create-ipsec-certs.sh

4. Runthescript:

$./openssl-create-ipsec-certs.sh

5. Becausethiscertificateexpiresin365days,setacalendarremindertorotatethecertificatewithintheyear.Forinstructionsonchangingcertificates,seeRotatingIPsecCertificates.

GenerateaSelf-SignedCertificatewithCertstrap1. DownloadandinstallGo .

2. Download the certstrap bashscript.

3. Navigatetothedirectorywhereyoudownloadedthescript:

$cd~/workspace

4. Changethepermissionsofthescript:

$chmodu+xcertstrap-create-ipsec-certs.sh

5. Runthescript:

$./certstrap-create-ipsec-certs.sh

Note:Useaself-signedcertificateonlyfordevelopmentortestenvironments.Donotuseaself-signedcertificateforaproductiondeployment.Thescriptsbelowgenerateprivatekeysina certs subdirectory.


https://docs.pivotal.io/addon-ipsec/1-6/scripts/openssl-create-ipsec-certs.sh

https://golang.org/dl/

https://docs.pivotal.io/addon-ipsec/1-6/scripts/certstrap-create-ipsec-certs.sh

Upgrading the IPsec Add-on for PCFPage last updated:

ThistopicdescribeshowtoupgradetheIPsecadd-on.

UpgradingtheIPsecAdd-On1. Retrievethelatestruntimeconfigbyrunningoneofthefollowingcommands:

For Ops Manager v1.10 or earlier: bosh runtime-config > PATH_TO_SAVE_THE_RUNTIME_CONFIGFor Ops Manager v1.11 or later: bosh2 -e BOSH_ENVIRONMENT runtime-config > PATH_TO_SAVE_THE_RUNTIME_CONFIG

2. Changethereleaseversion.

releases:- {name: ipsec, version: NEW_VERSION}

3. Updatetheruntimeconfigbyrunningoneofthefollowingcommands:For Ops Manager v1.10 or earlier: bosh update runtime-config PATH_TO_SAVE_THE_RUNTIME_CONFIGFor Ops Manager v1.11 or later: bosh2 -e BOSH_ENVIRONMENT update-runtime-config PATH_TO_SAVE_THE_RUNTIME_CONFIG

4. Uploadyournewrelease.

$boshuploadreleasePATH-TO-BINARY/BINARY-NAME.tar

5. NavigatetoyourInstallation DashboardinOpsManager.

6. ClickApply Changes.


Uninstalling the IPsec Add-on for PCFPage last updated:

ThistopicdescribeshowtouninstallIPsecfromyourdeployment.

UninstalltheIPsecAdd-On1. Retrievethelatestruntimeconfigbyrunningoneofthefollowingcommands:

For Ops Manager v1.10 or earlier: bosh runtime-config > PATH_TO_SAVE_THE_RUNTIME_CONFIGFor Ops Manager v1.11 or later: bosh2 -e BOSH_ENVIRONMENT runtime-config > PATH_TO_SAVE_THE_RUNTIME_CONFIG

2. Setthe optional flagto true underIPsecproperties.

3. Updatetheruntimeconfigbyrunningoneofthefollowingcommands:For Ops Manager v1.10 or earlier: bosh update runtime-config PATH/YOUR-RUNTIME-CONFIG.ymlFor Ops Manager v1.11 or later: bosh2 -e BOSH_ENVIRONMENT update-runtime-config PATH_TO_SAVE_THE_RUNTIME_CONFIG



6. Waitfortheinstallationtocomplete.

7. RemoveIPsecfromtheruntimeconfig.

8. Updatetheruntimeconfigbyrunningoneofthefollowingcommands:For Ops Manager v1.10 or earlier: bosh update runtime-config PATH/YOUR-RUNTIME-CONFIG.ymlFor Ops Manager v1.11 or later: bosh2 -e BOSH_ENVIRONMENT update-runtime-config PATH_TO_SAVE_THE_RUNTIME_CONFIG




Checking Certificate DatesPage last updated:

ThistopicdescribeshowtochecktheexpirationdatesofIPseccertificates.

ThefollowingproceduredescribeshowtodownloadtheruntimeconfigurationfileandextractthetwoIPseccertificatesintotemporaryfiles.Then,thefilesareinputtotheOpenSSLtool.TheOpenSSLtooldecodesthecertificatesanddisplaystheexpirationdates.

CheckCertificateDatesFollowthestepsbelowtodeterminetheexpirationdatesofyourIPseccertificates.

1. LogintoBOSHDirector.

2. RunoneofthefollowingcommandstodownloadyourruntimeconfigurationYAMLfile:For Ops Manager v1.10 or earlier: bosh runtime-config > PATH_TO_SAVE_THE_RUNTIME_CONFIGFor Ops Manager v1.11 or later: bosh2 -e BOSH_ENVIRONMENT runtime-config > PATH_TO_SAVE_THE_RUNTIME_CONFIG

Forexample,

bosh2runtime-config>/tmp/my-runtime-config.yml

3. DisplaytheruntimeconfigurationYAMLfilesothatyoucancopyfromit.Forexample,

$cat/tmp/my-runtime-config.yml

4. IdentifythesectionofthefilethatcontainsIPsecproperties,andlocatethecertificates:

addons:-include:stemcell:-os:ubuntu-trustyjobs:-name:ipsecrelease:ipsecname:ipsecproperties:ipsec:ca_certificates:-|-----BEGINCERTIFICATE-----MIIE/TCCAuWgAwIBAgIBATANBgkqhkiG9w0BAQsFADAOMQwwCgYDVQQDEwNjYTEwHhcNMTYwNTI2MjI1MDMzWhcNMjYwNTI2MjI1MDQyWjAOMQwwCgYDVQQDEwNjYTEw...Axu2pbEoT1PrMd3HlAZ3AH8ZrMR3ScJKCW3wQFRX/Plj-----ENDCERTIFICATE-----instance_certificate:|-----BEGINCERTIFICATE-----MIIEGTCCAgGgAwIBAgIQDlqK1V54BEknnblVPXu5lzANBgkqhkiG9w0BAQsFADAOMQwwCgYDVQQDEwNjYTEwHhcNMTYwNTI2MjI1MTAzWhcNMTgwNTI2MjI1MTAzWjAQMQ4wDAYDVQQDEwVjZXJ0MTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB...4Q6P/cDn9QvW2QbbWkApP2uuMk04jWJV7p79CfX4pipPqiSofjFyFqsjjvir-----ENDCERTIFICATE-----

5. Copytheca_certificateintoatextfile.Retaintheheaderandfooter,butdeletetheleadingwhitespacebeforethe -----BEGINCERTIFICATE----- and -----ENDCERTIFICATE----- lines.Forexample,

-----BEGINCERTIFICATE-----MIIE/TCCAuWgAwIBAgIBATANBgkqhkiG9w0BAQsFADAOMQwwCgYDVQQDEwNjYTEwHhcNMTYwNTI2MjI1MDMzWhcNMjYwNTI2MjI1MDQyWjAOMQwwCgYDVQQDEwNjYTEw...Axu2pbEoT1PrMd3HlAZ3AH8ZrMR3ScJKCW3wQFRX/Plj-----ENDCERTIFICATE-----


6. SavethefilewiththePEMextension,forexample, my-ipsec-ca-cert.pem .

7. Runthefollowingcommand:opensslx509-text-informpem-in/PATH/FILENAME.pem|grep"NotAfter"

Where /PATH/FILENAME.pem isthepathtoandfilenameofthefileyousavedinthestepabove.Forexample,

$opensslx509-text-informpem-in/tmp/my-ipsec-ca-cert.pem|grep"NotAfter"NotAfter:May2622:50:422026GMT

IfthePEMfileiscorrectlyformatted,theoutputshowsalinewiththe NotAfter date.IfthePEMfileisnotcorrectlyformatted,Theoutputshows unabletoloadcertificate .

8. Repeatsteps5–7fortheinstance_certificate.

9. Reviewthe NotAfter dateandplantoreplacethecertificatesaccordingly.Keepinmindtheleadtimetoobtainnewcertificatesandthetimetoperformadeploymenttoapplythem.Forinformationaboutrotatingcertificates,seeAbouttheProceduresabove.

10. Forsecurityhygiene,deletethreetemporaryfilesthatyoucreated:thedownloadedcopyofthe runtime-config.yml whichcontainstheprivatekeyandthetwoPEMfilesthatcontainthecertificates.


Troubleshooting the IPsec Add-on for PCFPage last updated:

ThistopicprovidesinstructionstoverifythatstrongSwan-basedIPsecworkswithyourPivotalCloudFoundry(PCF)deploymentandgeneralrecommendationsfortroubleshootingIPsec.

VerifythatIPsecWorkswithPCFToverifythatIPsecworksbetweentwohosts,youcancheckthattrafficisencryptedinthedeploymentwith tcpdump ,performthepingtest,andcheckthelogswiththestepsbelow.

1. Checktrafficencryptionandperformthepingtest.SelecttwohostsinyourdeploymentwithIPsecenabledandnotetheirIPaddresses.Thesearereferencedbelowas IP-ADDRESS-1 and IP-ADDRESS-2 .

a. SSHinto IP-ADDRESS-1 .

$sshIP-ADDRESS-1

b. Onthefirsthost,runthefollowing,andallowittocontinuerunning.

$tcpdumphostIP-ADDRESS-2

c. Fromaseparatecommandline,runthefollowing:

$sshIP-ADDRESS-2

d. Onthesecondhost,runthefollowing:

$pingIP-ADDRESS-1

e. VerifythatthepackettypeisESP.Ifthetrafficshows ESP asthepackettype,trafficissuccessfullyencrypted.Theoutputfrom tcpdump willlooksimilartothefollowing:

03:01:15.242731IPIP-ADDRESS-2>IP-ADDRESS-1:ESP(spi=0xcfdbb261,seq=0x3),length100

2. Openthe /var/log/daemon.log filetoobtainadetailedreport,includinginformationpertainingtothetypeofcertificatesyouuse,andtoverifyanestablishedconnectionexists.

3. NavigatetoyourInstallationDashboard,andclickRecent Install Logstoviewinformationregardingyourmostrecentdeployment.Searchfor“ipsec”andthestatusoftheIPsecjob.

4. Run ipsec statusall toreturnadetailedstatusreportregardingyourconnections.Thetypicalpathforthisbinary:/var/vcap/packages/strongswan-x.x.x/sbin . x.x.x representstheversionofstrongSwanpackagedintotheIPsec.

IfyouexperiencesymptomsthatIPsecdoesnotestablishasecureconnection,returntotheInstallingtheIPsecAdd-onforPCFtopicandreviewyourinstallation.

IfyouencounterissueswithinstallingIPsec,refertotheTroubleshootingIPsecsectionofthistopic.

TroubleshootIPsec

IPsecInstallationIssues

Symptom

Unresponsiveapplicationsorincompleteresponses,particularlyforlargepayloads


Explanation:PacketLoss

IPsecpacketencryptionincreasesthesizeofpacketpayloadsonhostVMs.Ifthesizeofthelargerpacketsexceedsthemaximumtransmissionunit(MTU)sizeofthehostVM,packetlossmayoccurwhentheVMforwardsthosepackets.

IfyourVMswerecreatedwithanAmazonPVstemcell,thedefaultMTUvalueis1500forbothhostVMsandtheapplicationcontainers.IfyourVMswerecreatedwithAmazonHVMstemcells,thedefaultMTUvalueis9001.Gardencontainersdefaultto1500MTU.

Solution

Implementa100MTUdifferencebetweenhostVMandthecontainedapplicationcontainer,usingoneofthefollowingapproaches:

DecreasetheMTUoftheapplicationcontainerstoavaluelowerthantheMTUoftheVMforthatcontainer.IntheElasticRuntimetileconfiguration,clickNetworkingandmodifyApplications Network Maximum Transmission Unit (MTU) (in bytes)beforeyoudeploy.Decreaseitfromthedefaultvalueof1454to1354.

IncreasetheMTUoftheapplicationcontainerVMstoavaluegreaterthan1500.Pivotalrecommendsaheadroomof100.Run ifconfigNETWORK-INTERFACEmtuMTU-VALUE tomakethischange.ReplaceNETWORK-INTERFACEwiththenetworkinterfaceusedtocommunicatewithotherVMsForexample: $ifconfigNETWORK-INTERFACEmtu1600

Symptom

Unresponsiveapplicationsorincompleteresponses,particularlyforlargepayloads

Explanation:NetworkDegradation

IPsecdataencryptionincreasesthesizeofpacketpayloads.Ifthenumberofrequestsandthesizeofyourfilesarelarge,thenetworkmaydegrade.

Solution

ScaleyourdeploymentbyallocatingmoreprocessingpowertoyourVMCPUorGPUs,which,additionally,decreasesthepacketencryptiontime.Onewaytoincreasenetworkperformanceistocompressthedatapriortoencryption.Thisapproachincreasesperformancebyreducingtheamountofdatatransferred.

IPsecUpgradeIssues

Symptom

OpsManagerreturnsanerrorwhenyouclickApply ChangestoUpgradethePCFIPsecAdd-On.

Explanation:Consulclusterfailure

WhenBOSHappliestheIPsecAdd-ontheVMsintheConsulcluster,theVMscannotcommunicatewitheachother,whichcausestheclustertofail.Thiscanresultinmanydifferenterrors.Forinstance,youmightseeanerrorinOpsManagerthatsaysaDiegocellisfailing.Inthiscase,itislikelythattheDiegocellisfailingbecauseConsulisnotrunning.

Solution1. ScaledownthenumberofConsulinstancesinElasticRuntimeto 1 .Formoreinformation,seeScalingElasticRuntime .


3. Aftertheinstallsucceeds,scaleupthenumberofConsulinstancestothepreviousvalue.


http://docs.pivotal.io/pivotalcf/opsguide/scaling-ert-components.html

IPsecRuntimeIssues

Symptom

ErrorsrelatingtoIPsec,includingsymptomsofnetworkpartition.YoumayreceiveanerrorindicatingthatIPsechasstoppedworking.

Forexample,thiserrorshowsasymptomofIPsecfailure,afailed clock_global-partition :

Failedupdatingjobclock_global-partition-abf4378108ba40fd9a43>clock_global-partition-abf4378108ba40fd9a43/0(ddb1fbfa-71b1-4114-a82c-fd75867d54fc)(canary):ActionFailedget_task:Task044424f7-c5f2-4382-5d81-57bacefbc238result:StoppingMonitoredServices:Stoppingserviceipsec:SendingstoprequesttoMonit:Requestfailed,response:Response{StatusCode:503,Status:'503ServiceUnavailable'}(00:05:22)..

Explanation:Asynchronous monit JobPriorities

WhenamonitstopcommandisissuedtotheNFSmounterjob,ithangs,preventingashutdownofthePCFcluster.

ThisisnotaproblemwiththeIPsecadd-onreleaseitself.Rather,itisaknownissuewiththeNFSmounterjobandthemonitstopscriptthatcanmanifestitselfafterIPsecisdeployedwithPCFv1.7.

Thisissueoccurswhenmonitjobprioritiesareasynchronous.Becausetheorderofjobshutdownisarbitrary,itispossiblethattheIPsecjobwillbestoppedfirst.Afterthishappens,thenetworkconnectivityforthatVMgoesaway,andtheNFSmounterjoblosesvisibilitytotheassociatedstorage.ThiscausestheNFSmounterjobtohang,anditblocksthemonitstopfromcompleting.SeetheMonitjobGithubdetails forfurtherinformation.

Solution1. BOSH ssh intothestuckinstancebyrunningoneofthefollowingcommands:

For Ops Manager v1.10 or earlier:bosh ssh VM_INDEX

For Ops Manager v1.11 or later: bosh2 -e BOSH_ENVIRONMENT -d DEPLOYMENT_NAME ssh VM_INDEX

2. Authenticateasrootandusethe svstopagent commandtokilltheBOSHAgent:

$sudosu#svstopagent

3. RunthefollowingcommandtodetectthemissingmonitjobVM.For Ops Manager v1.10 or earlier:bosh cloudcheck

For Ops Manager v1.11 or later: bosh2 -e ENVIRONMENT_NAME -d DEPLOYMENT_NAME cloud-check

Forexample,

#boshcloudcheckVMwithcloudID`vm-3e37133c-bc33-450e-98b1-f86d5b63502a'missing:

-Ignoreproblem-RecreateVMusinglastknownapplyspec-DeleteVMreference(DANGEROUS!)

4. Choose RecreateVMusinglastknownapplyspec .

5. Continuewithyourdeployprocedure.

Note:ThisissueaffectsdeploymentsusingCFv231orearlier,butinCFv232thereleaseusesannginxblobstoreinsteadoftheNFSblobstore.TheerrordoesnotexistforPCFdeploymentsusingCFreleasesgreaterthanCFv231.TheerroralsodoesnotapplytoPCFdeploymentsthatuseWebDAVastheirCloudControllerblobstore.


https://github.com/cloudfoundry/cf-release/blob/v231/jobs/nfs_mounter/monit

SymptomAppfailstostartwiththefollowingmessage:

FAILEDServererror,statuscode:500,errorcode:10001,message:Anunknownerroroccurred.

TheCloudControllerlogshowsitisunabletocommunicatewithDiegodueto getaddrinfo failing.

Deploymentfailswithasimilarerrormessage: diego_database-partition-620982d595434269a96a/0(a643c6c0-bc43-411b-b011-58f49fb61a6f)'isnotrunningafterupdate.Reviewlogsforfailedjobs:etcd

Explanation:SplitBrain consul

Thiserrorindicatesa“splitbrain”issuewithConsul.

Solution

Confirmthisdiagnosisbycheckingthe peers.json filefrom/var/vcap/store/consul_agent/raft.Ifitisnull,thentheremaybeasplitbrain.Tofixthisproblem,followthesesteps:

1. Run monit stop onallConsulservers:

2. Run rm -rf /var/vcap/store/consul_agent/ onallConsulservers.

3. Run monit start consul_agent onallConsulserversoneatatime.

4. Restarttheconsul_agentprocessontheCloudControllerVM.Youmayneedtorestartconsul_agentonotherVMs,aswell.

Symptom

YouseethatcommunicationisnotencryptedbetweentwoVMs.

Explanation:ErrorinNetworkConfiguration

TheIPsecBOSHjobisnotrunningoneitherVM.ThisproblemcouldhappenifbothIPsecjobscrash,bothIPsecjobsfailtostart,orthesubnetconfigurationisincorrect.ThereisamomentarygapbetweenthetimewhenaninstanceiscreatedandwhenBOSHsetsupIPsec.Duringthistime,datacanbesentunencrypted.Thislengthoftimedependsontheinstancetype,IAAS,andotherfactors.Forexample,onat2.microonAWS,thetimefromnetworkingstarttoIPsecconnectionwasmeasuredat95.45seconds.

Solution

SetupanetworkingrestrictiononhostVMstoonlyallowIPsecprotocolandblockthenormalTCP/UDPtraffic.Forexample,inAWS,configureanetworksecuritygroupwiththeminimalnetworkingsettingasshownbelowandblockallotherTCPandUDPports.

AdditionalAWSConfiguration

Type Protocol Port Range Source

CustomProtocol AH(51) All 10.0.0.0/16

CustomProtocol ESP(50) All 10.0.0.0/16

CustomUDPRule UDP 500 10.0.0.0/16

Note:Whenconfiguringanetworksecuritygroup,IPsecaddsanadditionallayertotheoriginalcommunicationprotocol.Ifacertainconnectionistargetingaportnumber,forexampleport8080withTCP,itactuallyusesIPprotocol50/51instead.Duetothisdetail,traffictargetedatablockedportmaybeabletogothrough.


Symptom

Youseeunencryptedappmessagesinthelogs.

Explanation: etcd SplitBrain

Solution1. CheckforsplitbrainetcdbyconnectingwithBOSH ssh intoeachetcdnode:

$curllocalhost:4001/v2/members

2. Checkifthemembersareconsistentonallofetcd.Ifanodehasonlyitselfasamember,ithasformeditsownclusteranddeveloped"splitbrain."Tofixthisissue,SSHintothesplitbrainVMandrunthefollowingcommands:

a. $sudosu-

b. #monitstopetcd

c. #rm-r/var/vcap/store/etcd

d. #monitstartetcd

3. Checkthelogstoconfirmthenoderejoinedtheexistingcluster.

Symptom

IPsecdeploymentfailswith Errorfillingintemplate'pre-start.erb'

Error100:Unabletorenderinstancegroupsfordeployment.Errorsare:-Unabletorenderjobsforinstancegroup'consul_server-partition-f9c4b18fd83cf3114d7f'.Errorsare:-Unabletorendertemplatesforjob'ipsec'.Errorsare:-Errorfillingintemplate'pre-start.erb'(line12:undefinedmethod`each_with_index'for#)-Unabletorenderjobsforinstancegroup'nats-partition-f9c4b18fd83cf3114d7f'.Errorsare:-Unabletorendertemplatesforjob'ipsec'.Errorsare:-Errorfillingintemplate'pre-start.erb'(line12:undefinedmethod`each_with_index'for#)

Explanation:TypographicalorsyntaxerrorindeploymentdescritorYAMLsyntax

Solution

CheckthedeploymentdescriptorYAMLsyntaxfortheCAcertificatesentry:


releases:-{name:ipsec,version:1.0.0}

addons:-name:ipsec-addonjobs:-name:ipsecrelease:ipsecproperties:ipsec:ipsec_subnets:-10.0.1.1/20no_ipsec_subnets:-10.0.1.10/32#boshdirectorinstance_certificate:|-----BEGINCERTIFICATE-----MIIEMDCCAhigAwIBAgIRAIvrBY2TttU/LeRhO+V1t0YwDQYJKoZIhvcNAQELBQAw...-----ENDCERTIFICATE-----instance_private_key:|-----BEGINEXAMPLERSAPRIVATEKEY-----MIIEogIBAAKCAQEAtAkBjrzr5x9g0aWgyDEmLd7m9u/ZzpK7UScfANLaN7JiNz3c...-----ENDEXAMPLERSAPRIVATEKEY-----ca_certificates:-|-----BEGINCERTIFICATE-----MIIEUDCCArigAwIBAgIJAJVLBeJ9Wm3TMA0GCSqGSIb3DQEBCwUAMB0xGzAZBgNVBAMMElBDRiBJUHNlYyBBZGRPbiBDQTAeFw0xNjA4MTUxNzQwNDVaFw0xOTA4MTUx...-----ENDCERTIFICATE-----

Intheexampleabove,thevaluesthatappearafterthe ca_certificates :keyarecontainedwithinalistandarenotjustasinglecertificate.Thisentrymustbefollowedbyalinestartingwith - ,andendingwith | .ThelinesfollowingthiscontainthePEMencodedcertificate(s).

Theerrormessageshownaboveindicatingaproblemwiththe each_with_index methodprovidesahintthatthe -| YAMLsyntaxsequenceismissing.UsethissyntaxeveninsituationswherethereisonlyoneCAcertificate,forexamplealistofoneentry.

Symptom

Completesystemoutagewithnowarning.

Explanation:IPsecCertificatesMightHaveExpired

ExpiredIPseccertificatescancauseasuddensystemoutage.Forexample,theself-signedcertificatesgeneratedbythescriptprovidedintheinstallationinstructionshavealifetimeof365days.IPseccertificatesexpireifyoudonotrotatethemwithintheirlifetime.

Solution

RenewexpiredIPseccertificates.ToavoidfuturedowntimeduetoexpiredIPseccertificates,setacalendarremindertorotatethecertificatesbeforetheyexpire.

Forhowtorenewcertificates,seeRenewingExpiredIPsecCertificates.Forhowtorotatethem,seeRotatingIPsecCertificates.


Rotating Active IPsec CertificatesPage last updated:

ThistopicdescribestheprocessPivotalrecommendstoincreasedeploymentsecuritybyrotatingcertificatesintheIPsecmanifest.

WhyYouNeedtoRotateCredentialsThesearecommonreasonsforrotatingcredentials:

Yourorganizationalsecuritypolicymayspecifyhowoftenyoushouldapplythesechanges.

Yourcertificatesaregoingtoexpire.Tofindtheexpirationdatesonyourcertificates,seeCheckingCertificateDates.

AbouttheProceduresTherearetwoproceduresforcertificaterotationdescribedinthistopic:

ThefirstproceduredescribesrotatingtheinstancecertificateandinstanceprivatekeyspecifiedinyourIPsecmanifest,andrequiresupdatingBOSH.Thisproceduredoesnotincluderotatingthecertificateauthority(CA)certificate.

ThesecondproceduredescribesrotatingyourCAcertificateinadditiontoyourinstancecertificateandinstanceprivatekey.ThisprocedurerequiresupdatingBOSHthreetimes.

Procedure1:RotatetheInstanceCertificateandInstancePrivateKeyFollowthestepsbelowtorotatetheinstancecertificateandinstanceprivatekey.

1. GenerateanewcertificateanduseyourexistingIPsecCAcertificatetosignthenewcertificate.

2. Updatetheinstancecertificateandtheprivatekeyfieldsinyour ipsec-addon.yml filewithnewvaluesfromthepreviousstep.


4. NavigatetoyourOpsManagerinterfaceinabrowser,andclickApply Changes.

Procedure2:RotatetheCACertificate,theInstanceCertificate,andInstancePrivateKeyFollowthesestepsbelowtorotatetheCAcertificate,theinstancecertificate,andinstanceprivatekey.

1. GenerateanewCAcertificate.

2. AppendthenewlygeneratedCAcertificateundertheexistingcertificateasanewyamllistelementinyour ipsec-addon.yml .Forexample:

ca_certificates: - | -----BEGIN CERTIFICATE----- ... -----END CERTIFICATE----- - | -----BEGIN CERTIFICATE-----

Note:Therollingdeploysduringtheseproceduresresultinminimaldeploymentdowntime.

Note:Thisstepresultsinafewminutesofappdowntime.


... -----END CERTIFICATE----- . . .



5. GenerateanewcertificateanduseyournewIPsecCAcertificatetosignthenewcertificate.

6. UpdatetheinstancecertificateandtheprivatekeyfieldsintheYAMLfilewithnewvaluesfromabove.

7. Repeatstep3toupdatetheruntimeconfig.


9. DeletetheolderCAcertificateinthe ipsec-addon.yml file.

10. Repeatstep3toupdatetheruntimeconfig.


Note:Thisstepresultsinafewminutesofappdowntime.


Renewing Expired IPsec CertificatesPage last updated:

Thistopicdescribesthebasicprocessthatdeployersmayusetorenewanysoon-to-be-expiringcertificatescontainedintheIPsecmanifest.

AboutCertificateExpirationTheIPsecAdd-onreliesuponX.509certificatestosecurethecommunicationsbetweencommunicatingpeers.

Likeallcertificates,theIPseccertificateshaveafinitelifetimeandeventuallyexpire.Thecertificatesgeneratedbytheprocedureprovidedintheinstallationinstructions,GenerateaSelf-SignedCertificatehaveadefaultlifetimeofoneyear.Regardlessoftheirspecificlifetime,allcertificatesmusteventuallyberotated,andsoitisimportantfortheoperationsteamtoplanaccordinglyandremembertorotatetheIPseccertificatesbeforetheyactuallyexpire.

RenewExpiredIPsecCertificatesTorenewexpiringIPseccertificates,dothefollowing:

1. Retrievethelatestruntimeconfigbyrunningoneofthefollowingcommands:For Ops Manager v1.10 or earlier: bosh runtime-config > PATH-TO-SAVE-THE-RUNTIME-CONFIGFor Ops Manager v1.11 or later: bosh2 -e BOSH-ENVIRONMENT runtime-config > PATH-TO-SAVE-THE-RUNTIME-CONFIG

2. Generateanewsetofcertificates.Fordevelopmentortestenvironments,youcanuseself-signedcertificates.Forinformationaboutself-signedcertificates,seeGenerateaSelf-SignedCertificate.

3. Intheruntime config.yml filesavedfromstep1,updatethe optional fieldto true andupdatethecertificatefieldswithnewcertificates.Formoreinformationaboutthesefields,seethefielddescriptionsunderCreatetheIPsecManifest.

properties: ipsec: optional: true instance_certificate: | -----BEGIN CERTIFICATE----- EXAMPLEAhigAwIBAgIRAIvrBY2TttU/LeRhO+V1t0YwDQYJKoZIhvcNAQELBQAw ... -----END CERTIFICATE----- instance_private_key: | -----BEGIN EXAMPLE RSA PRIVATE KEY----- EXAMPLExRSAxPRIVATExKEYxDATAxEXAMPLExRSAxPRIVATExKEYxDATA ... -----END EXAMPLE RSA PRIVATE KEY----- ca_certificates: - | -----BEGIN CERTIFICATE----- ExampleAvGgAwIBAgIBATANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDEwl0ZXN0 ... -----END CERTIFICATE-----

4. Updatetheruntimeconfigbyrunningoneofthefollowingcommands:For Ops Manager v1.10 or earlier: bosh update runtime-config PATH-TO-SAVE-THE-RUNTIME-CONFIGFor Ops Manager v1.11 or later: bosh2 -e BOSH-ENVIRONMENT update-runtime-config PATH-TO-SAVE-THE-RUNTIME-CONFIG



7. Removethe optional:true setinstep3.

8. Repeatsteps4to6.

IMPORTANT:RotatingthecertificateswhiletheyarestillvalidensuresthemaximumavailabilityoftheCloudFoundryplatformandavoidsanyunscheduledinterruptioninservice.


Release NotesPage last updated:

ThistopiccontainsreleasenotesfortheIPsecAdd-onforPCF.

v1.6.15Release Date:August22,2017

FixesacertificatevalidationissueonLinuxVMs-ValidateinstancecertificateissignedbyoneoftheCAcertificates

v1.6.14Release Date:August1,2017

FixestheOptionalflagforapplyingIPsectoanexistingcluster

Fixeslogfolderandfilespermissions(noimpacttousers)

v1.6.12Release Date:July28,2017

Restoressyslogforlogging

IPsecloglocationisnowconfigurable(defaultissyslog)

Usesboshstopinsteadofboshdrain

v1.6.9Release Date:June26,2017

InterimpatchtoavoidRabbitMQSyslogtimingconflict.Ifyoudon’thavethistimingconflict,youcanignorethispatch.

UpdateStrongswanto5.5.3,fixUSN-3301-1.

Updategolangto1.8.1

Updateopensslto1.0.2k

Updateopenssl-fipsto2.0.14.

Updategmpto6.1.2.

v1.6.3

Note:IfyouhaveconflictswiththeRabbitMQHAProxy,checktheKnowledgebaseatsupport.pivotal.io



Note:ThisreleasedoesNOTwritetoSyslog;ifyourequireSyslog,waitforthenextpatchrelease.


Release Date:March10,2017

SupportsAzure(forLinuxVMs).

EnablesconfigurationofIKEandESPproposals.

ReducesdowntimewhenapplyingIPsectoexistingdeploymentthroughoptionalflag.

Updates“xfrm_acq_expires”from165secs(default)to6secstospeedupretries.

FixestheorderingofIPsecandno-IPsecsubnets.

Usesaes128-sha256-modp2048!asthedefaultIKEproposal.

KnownIssuesSpurious Configuration Warning:AspartoftheupgradetoStrongSwanversion5.4.0,thisversionoftheIPsecadd-onwillemitasequenceofspuriousconfigurationwarningmessages.Themessageswillappearsimilartothefollowing:

!!Yourstrongswan.confcontainsmanualpluginloadoptionsforcharon.!!Thisisrecommendedforexpertsonly,see!!http://wiki.strongswan.org/projects/strongswan/wiki/PluginLoad

Thesemessagesarebothexpected,andharmless.Asacautiontoendusers,theStrongSwansoftwarenowemitsawarningmessagewhenitdetectsthattheinstallationincludesamanuallyconfiguredsetofplug-ins.Asamatterofsecurityhygienebestpractices,theIPsecadd-onhasalwaysusedamanual(explicit)configuration,andloadsarestrictedsetofStrongSwanplug-ins.Anyunusedplug-insarenotloaded.ThenewestversionofStrongSwannowissuesthiswarningmessagewhenitdetectsthatsituation.Theactuallistofplug-insinusehasbeendeterminedtobeappropriateforuseofStrongSwaninthePCFenvironment.Thiswarningisexpected,andshouldbeignored.

Certificate Verification:ThereisaknownissuewiththeCAcertificatevalidation.TheIPsecadd-onsupportscredentialrotationwithminimaldowntime.Thehostinstancecertificatecanberotatedatanytimebydoingadeployment.Inaddition,theCAcertificatethatisusedtoverifytrustinthehostcertificatescanberotatedwithminimaldowntimebydoingmultipledeployments.

However,becauseallVMstypicallysharethesameinstancecertificate,theywilltrusteachotherwithoutrelyingupontheCAcertificate.TheCAcertificateisnotactuallyneededuntiltheoperatordoesadeploymenttorotatetheinstancecertificate(s).Whilethatdeploymentisrunning,someoftheVMswillhavereceivedanewinstancecertificate,whileotherVMsarestilloperatingusingthepriorinstancecertificate.Duringthistime,whiletheinstancecertificatesaredifferent,thevalidationofthepeerinstancecertificatewillrelyuponthecommonCAcertificateinordertoestablishtrustinthecounterparty.

IftheCAcertificateismalformed,orotherwiseinvalid,thisproblemwillremainlatentuntilthetimewhentheinstancecertificateisbeingrotated.ItisonlyduringthatdeploymentwhentheoperatorwilldiscoverthattheCAcertificateisnotvalid.Ofcourse,aslongastheCAcertificateisvalid,thereisnoproblem.

ItisrecommendedthatoperatorsuseatoolsuchasOpenSSLtoverifythattheCAcertificatetheyarechoosingtoconfigureisinfactvalid,andcontainstheappropriatedetailsforproperend-entityauthenticationoftheVMinthedeployment(suchassubjectName,issuerName,andvaliditydates,etc).

Operatorscanusetheirfavoritecertificatemanagementtooltoconfirmthattheircertificatematcheswhattheyexpect.UsingOpenSSL,onecanissuethecommand:

$opensslx509-inmyCA.crt-text

Ifthiscommandproducesvalidoutput,thenthecertificatewillbeOKwhenconfiguredforIPsec.

MTU Sizing:Use1354onOpenStack.KeepthedefaultonAWSandvSphere.


ipsec add-on for pcf - resources.docs.pivotal.iogoogle cloud platform (gcp), vsphere, azure, amazon...

Documents