ibm hyper protect virtual servers user s guide - version 1.2.x...

IBM Hyper Protect Virtual Servers User’sGuide - Version 1.2.x 2021-04-02

IBM

Contents

About this documentation......................................................................................1Intended audience.......................................................................................................................................1Prerequisite and related information.......................................................................................................... 2

Release notes........................................................................................................2What's new in version 1.2.3.........................................................................................................................2

The imagecache parameter....................................................................................................................2License acceptance................................................................................................................................ 2Non-default SSH port............................................................................................................................. 2New CLI commands............................................................................................................................... 2Networking with Hipersockets............................................................................................................... 3Added Linux capabilities........................................................................................................................ 3Schnorr signature support......................................................................................................................3

What's new in version 1.2.2.1......................................................................................................................3Fix Packs are available only on IBM Fix Central.................................................................................... 3BIP32 support........................................................................................................................................ 3SLIP-0010 support.................................................................................................................................3Upgrade IBM Hyper Protect Virtual Servers.......................................................................................... 4

What's new in version 1.2.2.........................................................................................................................4JSON format for the output of the Command Line Interface (CLI)....................................................... 4The hpvs undeploy command................................................................................................................ 4Git Large File Storage (LFS) support...................................................................................................... 4Ed25519 support....................................................................................................................................4Updated hpvs deploy command............................................................................................................ 4Updated the topic on GREP11 virtual servers....................................................................................... 4Base64 format for the SSH key.............................................................................................................. 5Setup script changes.............................................................................................................................. 5RUNQ_ROOTDISK...................................................................................................................................5Upgrade IBM Hyper Protect Virtual Servers.......................................................................................... 5Enabling ports.........................................................................................................................................5

What's new in version 1.2.1.1......................................................................................................................5Fix Packs are available only on IBM Fix Central.................................................................................... 5Changes in the images for Hyper Protect Virtual Servers..................................................................... 5Fix Pack installation instructions........................................................................................................... 6

What's new in version 1.2.1.........................................................................................................................6The IBM Hyper Protect Virtual Servers Command Line Interface (CLI)............................................... 6Setup script to set up the environment................................................................................................. 6The mustgather script............................................................................................................................ 6

What's new in version 1.2.0.1......................................................................................................................6Fix Packs only available on IBM Fix Central.......................................................................................... 6Changes in the Hyper Protect Virtual Servers configuration.................................................................6Other improvements.............................................................................................................................. 7Fix Packs upgrade instructions.............................................................................................................. 7

What's new in version 1.2.0.........................................................................................................................7Support of IBM z15 and LinuxONE III................................................................................................... 7Support of configuring storage on the Secure Service Container partition by using commands.........7Support of creating networks for Secure Build Container on-the-fly................................................... 7Support of creating the repository registration file on-the-fly..............................................................7Support of generating the signing key pair when creating and encrypting the repository

registration files.................................................................................................................................8

ii

Support of updating the Secure Build container and its base image....................................................8Support of using personal certificate for the Secure Build container...................................................8Support of specifying the Dockerfile name and location for the Secure Build container.....................8Support of monitoring............................................................................................................................ 8Support of Enterprise PKCS #11 (EP11) integration.............................................................................8More configuration example files...........................................................................................................8Federal Information Processing Standards (FIPS)................................................................................9

Known issues and limitations...................................................................................................................... 9Known issues and limitations with IBM Hyper Protect Virtual Servers Version 1.2.2......................... 9Known issues and limitations with IBM Hyper Protect Virtual Servers Version 1.2.1....................... 10The following are some limitations related to upgrading IBM Hyper Protect Virtual Servers

from 1.2.0, or 1.2.0.1 to 1.2.1........................................................................................................ 11Known issues and limitations with IBM Hyper Protect Virtual Servers 1.2.0, or 1.2.0.1.................. 11

Accessibility features for IBM® Hyper Protect Virtual Servers................................................................. 12Overview............................................................................................................................................... 12Keyboard navigation.............................................................................................................................12Vendor software................................................................................................................................... 13Related accessibility information.........................................................................................................13

Introduction........................................................................................................ 13Overview.................................................................................................................................................... 13Advantages.................................................................................................................................................14Technology at a glance.............................................................................................................................. 14

IBM Secure Service Container............................................................................................................. 14Architecture overview................................................................................................................................15Components...............................................................................................................................................18System requirements.................................................................................................................................20

Hardware requirements for the Linux management server................................................................20Hardware requirements for Secure Service Container partition.........................................................20Software requirements........................................................................................................................ 21Supported operating systems and platforms...................................................................................... 22Networking........................................................................................................................................... 22Supported Docker versions..................................................................................................................23Required ports......................................................................................................................................23

User roles................................................................................................................................................... 23FAQs........................................................................................................................................................... 24

What is IBM Hyper Protect Virtual Servers?........................................................................................24As an application developer or ISV, how can I benefit from IBM Hyper Protect Virtual Servers?.....24As a cloud administrator or system administrator, how can I benefit from IBM Hyper Protect

Virtual Servers?............................................................................................................................... 24As a solution end-user, how can I benefit from IBM Hyper Protect Virtual Servers?........................ 24What is Secure Service Container Framework?...................................................................................24What is a hosting appliance?................................................................................................................24What is a software appliance?............................................................................................................. 25Can I deploy an application as is or is containerizing my application required to use IBM Hyper

Protect Virtual Servers?.................................................................................................................. 25Can I use my own private key to sign the images for the Docker Content Trust?.............................. 25What happens when I run the docker push command against a DCT-enabled repository?..........25Where is the Secure Build container?..................................................................................................25When is the docker repo key pair generated?..................................................................................... 25What are manifest signing keys?..........................................................................................................25Can the Secure Build server container be used to build an existing docker image on the Docker

Hub?.................................................................................................................................................25

Planning for the environment...............................................................................26Before you begin........................................................................................................................................ 26Management server................................................................................................................................... 26

iii

Secure Service Container partitions..........................................................................................................27A Hyper Protect Virtual Server instance with SSH daemon......................................................................27A Secure Build virtual server..................................................................................................................... 28Monitoring.................................................................................................................................................. 29Grep11....................................................................................................................................................... 30Next............................................................................................................................................................ 30

Downloading IBM Hyper Protect Virtual Servers................................................... 30Before you begin........................................................................................................................................ 31Procedure...................................................................................................................................................31Result......................................................................................................................................................... 32Next............................................................................................................................................................ 34Downloading IBM Hyper Protect Virtual Servers Fix Pack....................................................................... 34

Before you begin...................................................................................................................................34Procedure............................................................................................................................................. 34Next.......................................................................................................................................................34

Setting up the Secure Service Container Partition.................................................34Creating the Secure Service Container partition.......................................................................................35

Before you begin...................................................................................................................................35Procedure............................................................................................................................................. 35Next.......................................................................................................................................................36

Installing the Hyper Protect hosting appliance........................................................................................ 36Before you begin...................................................................................................................................36Procedure............................................................................................................................................. 37Next.......................................................................................................................................................37

Configuring the storage on the Secure Service Container partition......................................................... 37Before you begin...................................................................................................................................37Procedure............................................................................................................................................. 37Next.......................................................................................................................................................38

Configuring the network on the Secure Service Container partition........................................................ 38Before you begin...................................................................................................................................38Procedure............................................................................................................................................. 39Next.......................................................................................................................................................40

Working with IBM Hyper Protect Virtual Servers version 1.2.1, or later................. 40Setting up the environment by using the setup script.............................................................................. 40

Before you begin...................................................................................................................................41Procedure............................................................................................................................................. 41Available language codes.....................................................................................................................44Next.......................................................................................................................................................44

Registering base images in the remote registry server............................................................................ 44Before you begin...................................................................................................................................45Procedure............................................................................................................................................. 45

Creating a Hyper Protect Virtual Server instance..................................................................................... 46Before you begin...................................................................................................................................46Procedure............................................................................................................................................. 47Next.......................................................................................................................................................51

Generating the signing keys...................................................................................................................... 51Before you begin...................................................................................................................................52Procedure............................................................................................................................................. 52

Enabling ports............................................................................................................................................ 52Building your application with the Secure Build virtual server.................................................................53

Before you begin...................................................................................................................................54Procedure............................................................................................................................................. 54

Verifying the signature of the manifest file............................................................................................... 61Procedure............................................................................................................................................. 62

iv

Rolling keys in a Secure Build container................................................................................................... 63Before you begin...................................................................................................................................63Procedure............................................................................................................................................. 63

Deploying your applications securely....................................................................................................... 63Before you begin...................................................................................................................................63Procedure............................................................................................................................................. 64

Refreshing registered repositories with a new signing key pair............................................................... 68Before you begin...................................................................................................................................68Procedure............................................................................................................................................. 68

Creating the monitoring Virtual Servers....................................................................................................70Before you begin...................................................................................................................................70Procedure............................................................................................................................................. 70Next.......................................................................................................................................................73Creating CA signed certificates for the monitoring infrastructure...................................................... 74

Creating the GREP11 container.................................................................................................................77Before you begin...................................................................................................................................77Procedure............................................................................................................................................. 78Next.......................................................................................................................................................81Creating OpenSSL certificates for GREP11 Virtual Servers................................................................ 82Testing the GREP11 virtual server....................................................................................................... 84

Backing up and restoring IBM Hyper Protect Virtual Servers...................................................................88Procedure............................................................................................................................................. 88

Undeploying virtual servers.......................................................................................................................89Before you begin...................................................................................................................................89Procedure............................................................................................................................................. 89

Updating virtual servers.............................................................................................................................91Procedure............................................................................................................................................. 91

Uninstalling IBM Hyper Protect Virtual Servers........................................................................................92Uninstalling IBM Hyper Protect Virtual Servers CLI tools...................................................................92Uninstalling Secure Service Container partitions................................................................................92

Updating Hyper Protect Virtual Server containers....................................................................................93Before you begin...................................................................................................................................93Procedure............................................................................................................................................. 93

Upgrading IBM Hyper Protect Virtual Servers.......................................................................................... 94Before you begin...................................................................................................................................95Procedure............................................................................................................................................. 95Updating the virtual servers images.................................................................................................... 96Rollback in case of update failure......................................................................................................102

Upgrading IBM Hyper Protect Virtual Servers from 1.2.0 or 1.2.0.1 to 1.2.1....................................... 103Before you begin................................................................................................................................ 103Procedure........................................................................................................................................... 103The following are some limitations related to upgrading virtual servers......................................... 105Updating the collectd and monitoring virtual servers images.......................................................... 105

Using the Docker based commands with the IBM Hyper Protect Virtual Servers...105Planning for the environment..................................................................................................................106

Before you begin................................................................................................................................ 106Management server........................................................................................................................... 107Secure Service Container partitions.................................................................................................. 107Secure Build containers..................................................................................................................... 108Repository definition files.................................................................................................................. 110Hyper Protect Virtual Server containers............................................................................................110Monitoring.......................................................................................................................................... 111Grep11................................................................................................................................................112Next.................................................................................................................................................... 112

Installing the IBM Hyper Protect Virtual Servers CLI tool......................................................................113Before you begin................................................................................................................................ 113

v

Procedure........................................................................................................................................... 113Next.................................................................................................................................................... 114

Deploying your workloads with IBM Hyper Protect Virtual Servers.......................................................114Registering base images in the remote registry server.....................................................................115Loading the Secure Build base image into the Secure Service Container partition......................... 117Building the container image for your application by using the Secure Build..................................120Creating and encrypting the repository definition file...................................................................... 131Registering the repository on the Secure Service Container............................................................ 134Deploying your application into the Hyper Protect Virtual Server container................................... 135Debugging your application in the Hyper Protect Virtual Server container......................................137Updating Hyper Protect Virtual Server containers............................................................................138Refreshing registered repositories with a new signing key pair....................................................... 139

Monitoring IBM Hyper Protect Virtual Servers....................................................................................... 140Before you begin................................................................................................................................ 141Procedure........................................................................................................................................... 141Next.................................................................................................................................................... 143

Generating certificates for secure communication................................................................................ 144Creating CA signed certificates for the monitoring infrastructure....................................................144

Integrating with the EP11 library............................................................................................................ 146Before you begin................................................................................................................................ 147Procedure........................................................................................................................................... 147Next.................................................................................................................................................... 149Creating OpenSSL certificates for GREP11 Virtual Servers.............................................................. 149Testing the GREP11 virtual server.....................................................................................................151

Backing up and restoring IBM Hyper Protect Virtual Servers................................................................ 155Procedure........................................................................................................................................... 155

Upgrading IBM Hyper Protect Virtual Servers........................................................................................ 157Before you begin................................................................................................................................ 157Procedure........................................................................................................................................... 157Next.................................................................................................................................................... 157

Uninstalling IBM Hyper Protect Virtual Servers......................................................................................157Uninstalling IBM Hyper Protect Virtual Servers CLI tools.................................................................158Uninstalling Secure Service Container partitions..............................................................................158

Working with Secure Service Container for IBM Cloud Private.............................159Architecture of Secure Service Container for IBM Cloud Private........................................................... 159System requirements for IBM Secure Service Container for IBM Cloud Private................................... 160

Hardware requirements for the 64-bit x86 or Linux on IBM Z/LinuxONE (such as s390xarchitecture) management server................................................................................................160

Hardware requirements for Secure Service Container partition...................................................... 160Networking......................................................................................................................................... 161Supported operating systems and platforms....................................................................................161Software requirements...................................................................................................................... 162Supported Docker versions................................................................................................................162Supported IBM Cloud Private versions..............................................................................................162Required ports....................................................................................................................................162

Known issues and limitations..................................................................................................................162Installing Secure Service Container for IBM Cloud Private.................................................................... 165

Planning for Secure Service Container for IBM Cloud Private.......................................................... 165Downloading IBM Hyper Protect Virtual Servers.............................................................................. 170Installing the IBM Hyper Protect Virtual Servers CLI tool................................................................ 174

Installing IBM Cloud Private cluster....................................................................................................... 175Creating Secure Service Container partitions................................................................................... 176Installing the Secure Service Container for IBM Cloud Private appliance....................................... 176Configuring Secure Service Container storage..................................................................................177Configuring the appliance network....................................................................................................179Configuring the network for worker and proxy nodes.......................................................................180

vi

Configuring the Secure Service Container for IBM Cloud Private CLI tool....................................... 182Configuring the cluster resources......................................................................................................183Deploying GlusterFS...........................................................................................................................194Creating the cluster nodes.................................................................................................................196Configuring the network on the master node....................................................................................199Deploying IBM Cloud Private............................................................................................................. 204Before you begin................................................................................................................................ 204Deploying containerized applications............................................................................................... 207Updating the cluster resources dynamically..................................................................................... 208

Upgrading Secure Service Container for IBM Cloud Private...................................................................210Planning for the upgrade....................................................................................................................210Upgrading the Secure Service Container for IBM Cloud Private appliance......................................211Upgrading the Secure Service Container for IBM Cloud Private CLI tool......................................... 211Upgrading cluster nodes with the isolated VM..................................................................................213Upgrading the IBM Cloud Private...................................................................................................... 214

Reverting the Secure Service Container for IBM Cloud Private............................................................. 215Undo upgrade preparation steps.......................................................................................................216Downgrade the Secure Service Container for IBM Cloud Private appliance.................................... 216Rollback the Secure Service Container for IBM Cloud Private CLI tool............................................217Rollback the isoldated VMs on each cluster node............................................................................ 217Rollback IBM Cloud Private............................................................................................................... 217

Uninstalling Secure Service Container for IBM Cloud Private................................................................217Uninstalling IBM Cloud Private.......................................................................................................... 217Uninstalling the Secure Service Container for IBM Cloud Private CLI tool...................................... 218Uninstalling Secure Service Container partitions..............................................................................219

Troubleshooting IBM Hyper Protect Virtual Servers............................................ 219Refer to the following information for troubleshooting issues with IBM Hyper Protect Virtual

Servers Version 1.2.1, or later............................................................................................................219Known issues with IBM Hyper Protect Virtual Servers version 1.2.3.................................................... 220

ERROR: Failed to pull image.............................................................................................................. 220ERROR: Failed to SSH to the HPVS container when both external and private networks exist.......220ERROR: HVS-VSUD003 Update virtual server failed due to wrong quotagroup configuration........220HPVS container hangs or secure shell (SSH) access fails.................................................................220The data pool is not ready..................................................................................................................220The "hpvs vs **" command failed..................................................................................................... 220standard_init_linux.go:: exec user process caused "exec format error"..... 221gpg: Invalid option errors when generating the GPG key pair...............................................221GPG hangs or "Not enough random bytes available." error when generating the GPG key pair..... 221Secure Build failed to clone the Github repository if a passphrase is associated with the

private key..................................................................................................................................... 222Hyper Protect Virtual Server instance restarting continuously when running hpvs vs list

command...................................................................................................................................... 222Known issues with IBM Hyper Protect Virtual Servers version 1.2.2.................................................... 222

ERROR: HVS-VSUD003 Update virtual server failed due to wrong quotagroup configuration........222HPVS container hangs or secure shell (SSH) access fails.................................................................223The data pool is not ready..................................................................................................................223The "hpvs vs **" command failed..................................................................................................... 223standard_init_linux.go:: exec user process caused "exec format error"..... 223gpg: Invalid option errors when generating the GPG key pair...............................................224GPG hangs or "Not enough random bytes available." error when generating the GPG key pair..... 224Secure Build failed to clone the Github repository if a passphrase is associated with the


command...................................................................................................................................... 224Known issues with IBM Hyper Protect Virtual Server 1.2.1.1, or 1.2.1.................................................225

The data pool is not ready..................................................................................................................225

vii

The "hpvs vs **" command failed..................................................................................................... 225standard_init_linux.go:: exec user process caused "exec format error"..... 225gpg: Invalid option errors when generating the GPG key pair...............................................226GPG hangs or "Not enough random bytes available." error when generating the GPG key pair..... 226Secure Build failed to clone the Github repository if a passphrase is associated with the


command...................................................................................................................................... 227Refer to the following information for troubleshooting issues with IBM Hyper Protect Virtual

Servers Version 1.2.0.1, or 1.2.0....................................................................................................... 227Known issues with IBM Hyper Protect Virtual Servers Version 1.2.0.1, or 1.2.0..................................227

standard_init_linux.go:: exec user process caused "exec format error"..... 227gpg: Invalid option errors when generating the GPG key pair...............................................228GPG hangs or "Not enough random bytes available." error when generating the GPG key pair..... 228Configuration files not found when running the command...............................................................228Malformed configuration file issues when running the command................................................... 229Secure Build failed to clone the Github repository if a passphrase is associated with the

private key..................................................................................................................................... 230Invalid IP address for the Secure Build container when running the securebuild create

command...................................................................................................................................... 230Key pair issues when creating or registering a repository definition file..........................................231"Internal Server Error" message when running the image load command..................................232Invalid password problem when running regfile create command.........................................232"root.json not found" message when running the regfile create command........................... 233The repository signing key already exists in appliance when running repository create

command...................................................................................................................................... 233Signature validation failed when running repository update command.................................. 233CHZ00106E Not enough disk space available when running hpvs create command................ 234Hyper Protect Virtual Server instance restarting continuously when running hpvs get

command...................................................................................................................................... 234Container already exists when running monitoring create command..................................... 234IBM Hyper Protect Virtual Servers CLI tool fails to work after AppArmor upgrade on the

management server...................................................................................................................... 235

Troubleshooting Secure Service Container for IBM Cloud Private........................ 235Secure Service Container for IBM Cloud Private command line tool failed when configuring IBM

Cloud Private nodes............................................................................................................................235IBM Cloud Private installation failed with the host unresolved error.................................................... 235HTTP request error when accessing the IBM Cloud Private console right after the installation.......... 236"500 internal server error" when accessing the IBM Cloud Private master node.................................236An IBM Cloud Private node on the recycled Secure Service Container partition cannot ping all of

the other IBM Cloud Private nodes in the IBM Cloud Private cluster using its network interface...236Worker or proxy nodes stop responding after the cluster has been running smoothly for a while or

restarted............................................................................................................................................. 236Gateway on the proxy node is not configured automatically after the CLI installation.........................237DSN Server loopback issue during the IBM Cloud Private installation.................................................. 238False error messages about the storage requirements when installing the IBM Cloud Private........... 238The Catalog page is empty after the IBM Cloud Private cluster is started............................................ 238Actions to perform after the restart of the Secure Service Container for IBM Cloud Private

components........................................................................................................................................239"502 Bad Gateway" when accessing the application on IBM Cloud Private v3.1.2.............................. 239OCI runtime error after GlusterFS node restarted..................................................................................240Troubleshooting upgrade and rollback................................................................................................... 241

Operation failed when cordoning the cluster nodes......................................................................... 241Errors occurred when draining the cluster nodes............................................................................. 241Errors occurred when stopping the cluster nodes............................................................................ 241Errors occurred when exporting or importing the appliance data....................................................242

viii

Errors occurred when upgrading the Secure Service Container for IBM Cloud Private appliance..242Errors occurred when upgrading the Secure Service Container for IBM Cloud Private CLI tool..... 242Errors occurred when upgrading the isolated VM.............................................................................242Errors occurred when upgrading the IBM Cloud Private.................................................................. 243

Frequently asked questions.................................................................................................................... 243Architecture limits..............................................................................................................................243IBM Cloud private...............................................................................................................................244GDPS...................................................................................................................................................244Updates.............................................................................................................................................. 244Security...............................................................................................................................................244Database.............................................................................................................................................244

References........................................................................................................ 244File and directory structure of IBM Hyper Protect Virtual Servers........................................................ 245File and directory structure of IBM Hyper Protect Virtual Servers........................................................ 247Commands in IBM Hyper Protect Virtual Servers...................................................................................248

Commands......................................................................................................................................... 248hpvs crypto......................................................................................................................................... 250hpvs deploy........................................................................................................................................ 251hpvs help............................................................................................................................................ 251hpvs host............................................................................................................................................ 252hpvs image......................................................................................................................................... 254hpvs network......................................................................................................................................256hpvs network update......................................................................................................................... 258hpvs quotagroup................................................................................................................................ 259hpvs regfile......................................................................................................................................... 261hpvs registry....................................................................................................................................... 261hpvs repository...................................................................................................................................264hpvs sb................................................................................................................................................266hpvs snapshot.................................................................................................................................... 269hpvs undeploy.................................................................................................................................... 271hpvs vs................................................................................................................................................271

Configuration files of IBM Hyper Protect Virtual Servers....................................................................... 275hosts................................................................................................................................................... 275registry................................................................................................................................................275repository........................................................................................................................................... 276Virtual server template file................................................................................................................ 276Virtual server configuration file......................................................................................................... 276Secure Build configuration.................................................................................................................279Create repository registration............................................................................................................280

Network requirements for IBM Hyper Protect Virtual Servers...............................................................280Bridge types supported on IBM Hyper Protect Virtual Servers........................................................ 280Internal or external network configuration scenarios.......................................................................280

Overview of quotagroups for IBM Hyper Protect Virtual Servers...........................................................282Quotagroup types supported on IBM Hyper Protect Virtual Servers............................................... 282

Updating the parameters of IBM Hyper Protect Virtual Servers............................................................284Parameters of virtual servers that can be updated...........................................................................284

Gathering Information for IBM Support..................................................................................................285Before you begin................................................................................................................................ 285Procedure........................................................................................................................................... 285Next.................................................................................................................................................... 286

Commands...............................................................................................................................................286Commands for crypto domains..........................................................................................................286Commands for disks...........................................................................................................................286Commands for GREP11..................................................................................................................... 287Commands for Hyper Protect Virtual Server containers...................................................................288Commands for images....................................................................................................................... 294

ix

Commands for quotagroups.............................................................................................................. 294Commands for monitoring................................................................................................................. 296Commands for repositories............................................................................................................... 297Commands for repository definition files.......................................................................................... 298Commands for snapshots.................................................................................................................. 301Commands for Secure Build containers............................................................................................ 302

Configuration files....................................................................................................................................305Hosts...................................................................................................................................................305hpvs-config.yaml................................................................................................................................ 306hpvs-env.json..................................................................................................................................... 306securebuild.yaml................................................................................................................................ 307monitoring.yaml................................................................................................................................. 311OpenSSL configuration examples......................................................................................................312grep11-config.yaml............................................................................................................................ 313

Others...................................................................................................................................................... 314Security of IBM Hyper Protect Virtual Servers.................................................................................. 314List of docker images in the IBM Hyper Protect Virtual Servers.......................................................317List of keys used during the Secure Build..........................................................................................318Metrics collected by the monitoring infrastructure...........................................................................319

About error messages in Hyper Protect Virtual Servers....................................... 322Error Code Format for IBM Hyper Protect Virtual Servers CLI............................................................... 322Messages of IBM Hyper Protect Virtual Servers.....................................................................................323

Crypto command messages.............................................................................................................. 323Deploy command messages..............................................................................................................323Host command messages..................................................................................................................327Image command messages...............................................................................................................331Network command messages........................................................................................................... 335Quotagroups command messages.................................................................................................... 337Registry command messages............................................................................................................ 342Regfile command messages.............................................................................................................. 345Repository command messages........................................................................................................346Root command messages..................................................................................................................350Secure Build command messages.....................................................................................................350Snapshot command messages..........................................................................................................357Token operations................................................................................................................................362Virtual Server command messages................................................................................................... 362

Terminology...................................................................................................... 371

x

About this documentation

This documentation describes how to use the IBM® Hyper Protect Virtual Servers to deploy and manageDocker-based workloads on IBM Z and LinuxONE servers in your environment.

The documentation is structured based on the following major workflow:-

• How to configure and start a Secure Service Container partition.• How to install Hyper Protect hosting appliance by using the Secure Service Container user interface.• How to configure and start IBM Hyper Protect Virtual Servers on IBM Z and LinuxONE servers in your

cloud environment.• How to securely build and deploy your containerized workload into the Hyper Protect Virtual Server

containers.

Note that IBM Hyper Protect Virtual Servers contains components to support the following deployment:

• Secure Service Container for IBM Cloud Private. For more information about this component, seeWorking with Secure Service Container for IBM Cloud Private.

• IBM Hyper Protect Virtual Servers and its modules. For more information about this component,continue with this documentation.

This documentation describes the version of IBM Hyper Protect Virtual Servers that is available fordeployment with:

• Hardware Management Console (HMC) / Support Element (SE) Version 2.14.0 (z14, z14 ZR1, LinuxONEEmperor II or LinuxONE Rockhopper II). For more information about Secure Service Container, seeSecure Service Container User's Guide, SC28-6978-02a.

• Hardware Management Console (HMC) / Support Element (SE) Version 2.15.0 (z15, LinuxONE III). Formore information about Secure Service Container, see Secure Service Container User's Guide,SC28-7005-01.

Figures that are included in this document illustrate concepts and are not necessarily accurate in content,appearance, or specific behavior.

Important: The PDF version of this product document is a snapshot of the content on the IBM KnowledgeCenter on 2021-04-02. To read the up-to-date documentation about IBM Hyper Protect Virtual Servers,see IBM Knowledge Center.

Intended audienceThe primary audience for this documentation is developers wanting to securely build their applications,and administrators who are responsible for installing, and managing containerized applications in thesecured cloud environment. Those containerized applications can be hosted within Hyper Protect VirtualServer containers on an IBM Z or LinuxONE server.

This documentation distinguishes the following types of user roles:

• Cloud admin• Appliance admin• System admin• ISV or App developer

The different tasks that are described in this documentation are associated to one of these user roles. Asingle user can have a single role or multiple roles. For more information about the roles, see User roles.

About this documentation 1

https://www.ibm.com/support/pages/node/6018358https://www.ibm.com/support/pages/node/6019774https://www.ibm.com/support/pages/node/6019774https://www.ibm.com/support/knowledgecenter/SSHPMH_1.2.x/kc_welcome_page.html

Prerequisite and related informationTo deploy and manage containerized workloads within Hyper Protect Virtual Server containers on IBM Zor LinuxONE servers, in addition to this documentation, system administrators also need to access thefollowing reference to accomplish specific tasks.

• For more information about IBM Secure Service Container deployment to z14, z14 ZR1, LinuxONEEmperor II or LinuxONE Rockhopper II, see Secure Service Container User's Guide, SC28-6978-02a.

• For more information about IBM Secure Service Container deployment to z15 or LinuxONE III, seeSecure Service Container User's Guide, SC28-7005-00b.

Release notes

• What's new• Known issues and limitations• Accessibility features

What's new in version 1.2.3Get a quick overview of what's added, changed, improved, or deprecated in this release.

IBM Hyper Protect Virtual Servers Version 1.2.3 introduces the following new features andenhancements:

The imagecache parameterYou can specify the imagecache parameter in the configuration yaml file that is used to create a virtualserver by using the hpvs deploy command. When the value of the imagecache parameter is set totrue, then the image from the cache is used during the deploy operation, and when the value is set tofalse, the deploy operation pulls the images and register the repositories every time the deployoperation is run. For more information, see the following topics.

• Creating a Hyper Protect Virtual Server instance• Building your application with the Secure Build virtual server• Deploying your applications securely• Working with Monitoring virtual servers• Working with GREP11 virtual servers• Configuration files of IBM Hyper Protect Virtual Servers

License acceptanceThe license must be accepted for executing the setup script. For more information, see Setting up theenvironment by using the setup script.

Non-default SSH portA non-default SSH port can be specified in the "github url" parameter. For more information, see Buildingyour application with the Secure Build virtual server.

New CLI commandsThe hpvs host show, hpvs host unset, and hpvs network update commands are nowsupported. For more information, see Commands in IBM Hyper Protect Virtual Servers.

2 IBM Hyper Protect Virtual Servers User’s Guide - Version 1.2.x 2021-04-02

https://www.ibm.com/support/pages/node/6018358https://www.ibm.com/support/pages/node/6019774

Networking with HipersocketsYou can leverage technologies that are available within the Z architecture like internal communications todrive performance, scale, and optimized use of hardware resources. IBM Z Architecture internal logicalpartition (LPAR) to LPAR communications technology using Hipersockets is now supported, therebyreducing additional hardware requirement and increasing performance. For more information, see thefollowing topics.

• System requirements• Configuring the network on the Secure Service Container partition

Added Linux capabilitiesAdded support for the cap_add parameter. For more information, see the following topics.

• Building your application with the Secure Build virtual server• Deploying your applications securely.

Schnorr signature supportThe Schnorr signature is a digital signature produced by the Schnorr signature algorithm and is known forits simplicity, efficiency, and generates short signatures. For more information, see the following topics.

• Working with GREP11 virtual servers• Testing the GREP11 virtual server

What's new in version 1.2.2.1Get a quick overview of what's added, changed, improved, or deprecated in this release.

IBM Hyper Protect Virtual Servers Version 1.2.2.1 introduces the following new features andenhancements:

Fix Packs are available only on IBM Fix CentralThe installation package of IBM Hyper Protect Virtual Servers version 1.2.2.1 is available only on IBM FixCentral.

For more information on how to download the Fix Pack, see Downloading the Fix Pack installationpackages.

BIP32 supportAddress path (BIP32) defines how to derive private and public keys of a wallet from a binary master seed(m) and an ordered set of indices. This feature is now supported. For more information, see the followingtopics.


SLIP-0010 supportSLIP-0010 describes how to derive private and public key pairs for curve types different from secp256k1.Secp256k1 refers to the parameters of the elliptic curve used in Bitcoin's public-key cryptography, and isdefined in the Standards for Efficient Cryptography (SEC). This feature is now supported. For moreinformation, see the following topics.


Release notes 3

https://www.ibm.com/support/fixcentralhttps://www.ibm.com/support/fixcentral

Upgrade IBM Hyper Protect Virtual ServersYou can upgrade IBM Hyper Protect Virtual Servers from version 1.2.2 to version 1.2.2.1 For moreinformation, see Upgrading IBM Hyper Protect Virtual Servers.



JSON format for the output of the Command Line Interface (CLI)You can use the --json flag when you want the output to be displayed in json format. For moreinformation, see Commands in IBM Hyper Protect Virtual Servers.

The hpvs undeploy commandYou can use the hpvs undeploy command to delete existing virtual server instances along withresources like networks, and quotagroups, that were allocated to that virtual server. For more information,see Undeploying virtual servers.

Git Large File Storage (LFS) supportYou can use Git LFS with the Secure Build virtual server to build your source code stored in the GitHubrepository, deploy it into the IBM Hyper Protect Virtual Servers as a Hyper Protect Virtual Server instance,and publish the built image to the remote Docker repository. For more information, see Building yourapplication with the Secure Build Virtual Server.

Ed25519 supportEd25519 is a public-key signature system with several attractive features and is now supported. Only theCEX7P card is supported with ED25519. For more information, see the following topics.


Updated hpvs deploy commandYou can update the resources or configuration of a virtual server after the completion of the deployoperation by using the -u, or the --update flag of the hpvs deploy command. For more information,see the following topics.

• Creating a Hyper Protect Virtual Server instance• Building your application with the Secure Build virtual server• Deploying your applications securely• Working with Monitoring virtual servers• Working with GREP11 virtual servers

Updated the topic on GREP11 virtual serversThe example of the configuration yaml file has been updated with a new variable and the example of thejson file is updated with the changes required for the new GREP11 image. For more information, seeWorking with GREP11 virtual servers.


Base64 format for the SSH keyThe following topics have been updated with changes for the base64 format for the SSH key.

• Creating a Hyper Protect Virtual Server instance• Building your application with the Secure Build virtual server• Virtual server configuration file

Setup script changesThe setup.sh script can be executed by a root or a non-root user. For more information, see Setting up theenvironment by using the setup script.

RUNQ_ROOTDISKA dedicated root-disk can be assigned to a virtual server in the environment variables by using themount_id of disks (mounts) that are assigned to a virtual server from the available quotagroup. You canreset this root-disk by using the --update flag of the hpvs deploy command and setting the value ofthe reset_root parameter to true in mount section of the configuration file. RUNQ_ROOTDISK works forboth passthrough and non passthrough quotagroups. The parameter reset_root:true works only fornon passthrough quotagroups.

The following topics have been updated for this feature.

• Creating a Hyper Protect Virtual Server instance• Building your application with the Secure Build virtual server• Virtual server configuration file

Upgrade IBM Hyper Protect Virtual ServersYou can upgrade IBM Hyper Protect Virtual Servers from version 1.2.1.1, or 1.2.1 to version 1.2.2. Formore information, see Upgrading IBM Hyper Protect Virtual Servers from version 1.2.1.1, or 1.2.1 toversion 1.2.2.

Enabling portsWhen you are using IBM Hyper Protect Virtual Servers version 1.2.2, or later, before you build a dockerimage by using the Hyper Protect base images, you must open the required ports for your application. Formore information, see Enabling ports.

What's new in version 1.2.1.1Get a quick overview of what's added in this Fix Pack 1 release.

Fix Packs are available only on IBM Fix CentralThe installation package of IBM Hyper Protect Virtual Servers version 1.2.1.1 is available only on IBM FixCentral.

For more information on how to download the Fix Pack, see Downloading the Fix Pack installationpackages.

Changes in the images for Hyper Protect Virtual Servers• IBM Hyper Protect Virtual Servers Version 1.2.1.1 contains updated signing keys which enable the

deployment of new images on the IBM Hyper Protect Virtual Servers platform.• The signing keys that were shipped with the IBM Hyper Protect Virtual Servers Version 1.2.1 (July 2020)

have been refreshed in IBM Hyper Protect Virtual Servers Version 1.2.1.1. It is highly recommended

Release notes 5


that you migrate to IBM Hyper Protect Virtual Servers Version 1.2.1.1 to continue using the version withrefreshed keys for the provided images.

• IBM Hyper Protect Virtual Servers Version 1.2.1.1 does not introduce any new features nor does itchange the functionality of existing features supported by IBM Hyper Protect Servers Version 1.2.1.

Fix Pack installation instructionsTo install the fix pack for IBM Hyper Protect Virtual Servers Version 1.2.1.1, delete the /usr/local/bin/hpvsdirectory and then follow the instructions from step 4 of the topic Downloading the installation package.



The IBM Hyper Protect Virtual Servers Command Line Interface (CLI)The IBM Hyper Protect Virtual Servers environment can be setup by using a new set of CLI commandsthat simplifies the process of running various commands to create the virtual servers, deploy yourworkloads, monitoring, and GREP11 library. This includes the hpvs deploy command that simplifies thecreation of the IBM Hyper Protect Virtual Servers and deployment. For more information, see Commandsin IBM Hyper Protect Virtual Servers.

Setup script to set up the environmentThe setup.sh script performs an initial environment check on the Linux Management server. The scriptalso performs the following actions:

• Checks if Docker, OpenSSL and GPG are installed. This is required to use IBM Hyper Protect VirtualServers.

• Sets up the initial infrastructure required for the new IBM Hyper Protect Virtual Servers CLI. For moreinformation, see Setting up the environment by using the setup script.

The mustgather scriptThe IBM Hyper Protect Virtual Servers Version 1.2.1 provides an automated procedure to gather usefulinformation when you want to open a support ticket. For more information, see Gathering Information forIBM Support.

What's new in version 1.2.0.1Get a quick overview of what's added, changed, improved, or deprecated in this Fix Pack 1 release.

Fix Packs only available on IBM Fix CentralThe installation package of IBM Hyper Protect Virtual Servers version 1.2.0.1 is only available on IBM FixCentral.

For more information, see Downloading the Fix Pack installation packages.

Changes in the Hyper Protect Virtual Servers configurationThe quotagroup_storage parameter in the hpvs-config.yaml file defines the quotagroup size (inGB) used by the Hyper Protect Virtual Server container. If you specify 0 to this parameter, a default 12 GBquotagroup is created for the container. Note that quotagroups being created for Hyper Protect VirtualServer containers will have 15% less storage available due to the storage filesystem overheads.



Other improvementsIn this Fix pack, some other improvements include:

• Fixing some known mistakes in the example configuration files under the /monitoring-cli/configdirectory.

• Improvement on the error messages of some commands such as hpvs delete.• Security vulnerability fixes.

Fix Packs upgrade instructionsThe instructions on how to upgrade to IBM Hyper Protect Virtual Servers version 1.2.0.1 is available onUpgrading IBM Hyper Protect Virtual Servers.



Support of IBM z15 and LinuxONE IIIYou can run IBM Hyper Protect Virtual Servers on the latest IBM z15 and IBM LinuxONE III.

Support of configuring storage on the Secure Service Container partition byusing commands

When configuring the storage pool on the Secure Service Container partition, you can use the diskcommands to manage and add disks into the storage pool.

And you can also use the quotagroup update command to configure the appliance_dataquotagroup, which is used by the Hyper Protect hosting appliance.

For more information, see Configuring the storage on the Secure Service Container partition.

Support of creating networks for Secure Build Container on-the-flyWhen configuring the network for a Secure Build container, you can have a list of options:

• Use the port mapping on the partition• Use the dedicated IP address for external access• Use an existing network on the partition• Create a network on the partition when the Secure Build container is created

You can also control whether to delete the network when the Secure Build container is deleted by usingthe delete_network_on_container_delete parameter in the securebuild.yaml file.

For more information, see Configuring the network for Secure Build containers.

Support of creating the repository registration file on-the-flyWhen creating the container image by using the Secure Build, you can retrieve the cleartext repositoryregistration file in either python or JSON format.

The JSON repository registration file can be used as direct input to generate the encrypted repositorydefinition file.

For more information, see Building the container image for your application by using the Secure Build.

Release notes 7

Support of generating the signing key pair when creating and encrypting therepository registration files

When encrypting the repository registration file, you can use the private and public key pair generated byusing IBM Hyper Protect Virtual Servers CLI tool, or generate your own key pair for the encryption.

For more information, see Creating your own public and private key pair to encrypt the repositorydefinition file and Creating and encrypting the repository definition file.

Support of updating the Secure Build container and its base imageWhen updating the Secure Build container by using a different configuration or a newer version, you canuse the securebuild update command to complete the tasks.

For more information, see Updating the base image of a Secure Build container and Updating theconfiguration of a running Secure Build Container.

Support of using personal certificate for the Secure Build containerBefore creating the Secure Build container, you can specify your own certificate to override the defaultsettings such as the certificate's Subject Name, duration, key length, signature algorithm, and so on. Youcan use your own certificate by using the cert_name key in the securebuild.yaml file.

For more information, see Using your own certificate for the Secure Build container.

Support of specifying the Dockerfile name and location for the Secure Buildcontainer

When building your application with the Secure Build, you can use the dockerfile_path key in thesecurebuild.yaml to specify the relative path of the Dockerfile in your github repository. The defaultvalue is the root directory of your github repository.

Also, the build:script and build:dir keys are not supported in this release.

For more information, see Building the container image for your application by using the Secure Build.

Support of monitoringWith the monitoring infrastructure provided in IBM Hyper Protect Virtual Servers, you can collectmonitoring metrics from Hyper Protect hosting appliance and Secure Service Container partition.

For more information, see Monitoring IBM Hyper Protect Virtual Servers and Commands for monitoring

Support of Enterprise PKCS #11 (EP11) integrationWith the Enterprise PKCS #11 over gRPC (GREP11) containers provided in IBM Hyper Protect VirtualServers, you can integrate your application with the asymmetric (public and private) key pairs generatedby the Hardware Security Modules (HSM) on the IBM z or LinuxONE servers.

For more information, see Integrating with the EP11 library.

More configuration example filesAdditional configuration example files are available under the /config directory of each CLI commandmodule, and you can refer to them when configuring your own environment for IBM Hyper Protect VirtualServers.

Those example files covers the following network topologies for Hyper Protect Virtual Server and SecureBuild containers:

• Layer 2 and Layer 3 network based on Ethernet type connections• Layer 2 network based on VLAN type connections


• Layer 3 network based on VLAN type connections

For more information about the layer 2 and layer 3 network, see this blog.

For more information about VLAN or Ethernet type connections, download Secure Service ContainerUser's Guide from the About topic.

Federal Information Processing Standards (FIPS)Federal Information Processing Standards (FIPS) are information technology standards that aredeveloped by the United States federal government. See IBM Hyper Protect Virtual Servers platformconsiderations for FIPS compliance for information about FIPS compliance in IBM Hyper Protect VirtualServers.

Known issues and limitationsThis topic lists some of the known issues and limitations of IBM Hyper Protect Virtual Servers.

Known issues and limitations with IBM Hyper Protect Virtual ServersVersion 1.2.2.

• When you are running applications on virtual servers that are using non-passthrough quotagroups, it isrecommended that you monitor the available datapool size by using the hpvs quotagroup showcommand, and update the size by 5 GB when the size of the datapool is less than 1 GB. You can use thehpvs quotagroup update command to increase the size of the datapool.

• The snapshots of a Hyper Protect Virtual Server container can be created only on the same SecureService Container partition that the server instance resides.

• You can use IBM Hyper Protect Virtual Servers only with Docker Hub or IBM Container Registry.• You must restart the Hyper Protect Virtual Server container after you revert a snapshot of the Hyper

Protect Virtual Server container.• Secure Build requires that the private key, used to secure access to the source Github repository, does

not have a passphrase.• IBM Cloud Object Storage service is supported only for archiving application manifest files.• Backup and restore of encrypted credentials used by the Secure Build container can only be supported

by using Hosting Appliance snapshots.• The monitoring infrastructure collects metrics only from the Hyper Protect hosting appliance and Secure

Service Container partition.• When using ep11 for text file encryption, the text file size architectural limit is 4MB.• You must not delete the contents of the installation directory after you have run the setup.sh script.

The setup.sh creates a working directory that contains images which are symbolic links pointing tothe images in the extracted directory. If you delete the contents of the installation directory, you cannotrun the hpvs commands and must repeat the process of downloading and extracting the images.

• You cannot create a snapshot of virtual servers that are configured with passthrough quotagroups.• You cannot retrieve snapshots from virtual servers that have been deleted. • You cannot create snapshots of a virtual server that has multiple quotagroups attached when any of

them is a passthrough quotagroup.• If you create a snapshot for a virtual server with passthrough and non-passthrough quotagroups, that

results in an error, then you cannot delete the snapshot and the virtual server.• When specifying the size for the quotagroup, you must not use decimal notation. For example, use 1800

MB instead of 1.8GB.• If you had used quotagroup parameters when creating a virtual server, then you must pr

ibm hyper protect virtual servers user s guide - version 1.2.x...

Documents