Implementing Windows Server 2016 Device Guard and Credential Guard on HPE ProLiant servers

Contents 1.0 Introduction ............................................................................................................................................................................................................................................................................................................................. 2 2.0 System preparation .......................................................................................................................................................................................................................................................................................................... 2

2.1 Server platform firmware ...................................................................................................................................................................................................................................................................................... 2 2.2 Secure Boot ....................................................................................................................................................................................................................................................................................................................... 2 2.3 Server UEFI System Access ................................................................................................................................................................................................................................................................................ 2 2.4 Trusted Platform Module (TPM) .................................................................................................................................................................................................................................................................. 2 2.5 Server platform virtualization configuration ....................................................................................................................................................................................................................................... 2

3.0 Configure Device Guard ............................................................................................................................................................................................................................................................................................... 3 3.1 Enable Virtualization-Based Security ........................................................................................................................................................................................................................................................ 3 3.2 Create and deploy Kernel Mode Code Integrity policies ......................................................................................................................................................................................................... 3 3.3 Create and deploy User Mode Code Integrity policies .............................................................................................................................................................................................................. 5 3.4 Fine-tune CI Policy for different scenarios ............................................................................................................................................................................................................................................ 7 3.5 Best practices to enable Device Guard on HPE ProLiant servers................................................................................................................................................................................... 9

4.0 Configure Credential Guard ................................................................................................................................................................................................................................................................................... 10

Technical white paper

Technical white paper Page 2

1.0 Introduction With the release of Windows Server® 2016, Microsoft® introduced the concept of Virtualization Based Security (VBS). The commonly known features such as Device Guard and Credential Guard were built around VBS. While Microsoft does a good job of providing wealthy amount of detailed information (see link), this document will provide the user with a clear overview of the fundamentals, and best practice of implementing Device Guard and Credential Guard with HPE ProLiant Servers hardware and software.

When VBS is enabled, essentially an isolated “sandbox” environment is created, so critical code and data can be executed with much less exploit surface. This isolated “sandbox” environment is referred to as Virtualized Secure Mode (VSM). VSM is achieved with virtualization hardware functionalities such as Intel® VT-x or AMD virtualization (AMD-V), and Intel VT-d or AMD IOMMU, Microsoft’s hypervisor software layer (Hyper-V), and new implementation in Windows® kernel.

Device Guard is the combination of the new feature, VBS, and an existing feature, Code Integrity (CI). CI exists prior to Windows Server 2016. CI is similar to a “white list” to allow drivers, DLL, and executable to be run depending on a set of pre-defined rule set. Such rule set is called Configurable CI Policy. Prior to Windows Server 2016, processing of the CI Policy during early OS boot happens in the normal CPU/Memory/Kernel execution path. In Windows Server 2016, if Device Guard is enabled, the processing of CI Policy is done in VSM; thus reduce the possibility of tampering or breaches on the CI Policy. The processing of CI Policy in VSM is referred to as Hypervisor-assisted Code Integrity (HVCI).

User needs to configure the CI policy that is best suited to the usage of the system. Later in this document, a proposed best practice of CI policy with Service Pack for ProLiant (SPP) is explained in more details.

Credential Guard is another feature leveraging VBS in Windows Server 2016. When Credential Guard is enabled, the mechanism of logging in user name and passwords is processed in VSM instead of less protected code and memory regions.

Currently, only Gen10 or later generation of HPE ProLiant Servers supports these features.

2.0 System preparation It is recommended to follow best practice of security described in “HPE Gen10 Security Reference Guide”.

2.1 Server platform firmware This step is mandatory. All system firmware need to be updated with Service Pack for ProLiant (SPP) version 2018.09.0 or newer before OS installation. SPP can be downloaded here.

2.2 Secure Boot This step is mandatory. System firmware “Secure Boot” feature is a requirement for Device Guard/Credential Guard to work. It is recommended to enable Secure Boot before or right after the Windows Server OS installation. For details about Secure Boot and how to enable/disable it, please refer to the “Server security” section of the document, “UEFI System Utilities User Guide for HPE ProLiant Gen10 Servers and HPE Synergy”.

2.3 Server UEFI System Access This step is optional. To maximize protection, it is highly recommend to set user name and password for UEFI System Access. Without password protection for UEFI configuration, unintended access can disable Secure Boot thus leads to disable Device Guard/Credential Guard. For details about how to configure access permission for UEFI, please refer to the “Server security” section of the document, “UEFI System Utilities User Guide for HPE ProLiant Gen10 Servers and HPE Synergy”.

2.4 Trusted Platform Module (TPM) TPM is optional, but it is highly recommended to ensure basic server system security. For details of using TPM, please refer to “Hardware options installation => HPE Trusted Platform Module 2.0 Gen10 option” section of HPE ProLiant DL380 Gen10 Server User Guide.

2.5 Server platform virtualization configuration It is mandatory to enable hardware-assisted virtualization for Device Guard and Credential Guard to work. For details of configuring platform virtualization, please refer to the “Virtualization options” section of the document, “UEFI System Utilities User Guide for HPE ProLiant Gen10 Servers and HPE Synergy”. If the server is Intel processor based, please enable both “Intel Virtualization Technology (Intel VT)” and “Intel VT-d” option. The option of “SR-IOV” is not required. If the server is AMD processor based, please enable both “AMD IOMMU” and “AMD Virtualization Technology”.

Technical white paper Page 3

3.0 Configure Device Guard 3.1 Enable Virtualization-Based Security Virtualization-Based Security (VBS) is a required feature for Device Guard. You need to enable VBS before deploying Device Guard.

You can enable the VBS from the local policy editor. Run the “gpedit.msc” in command console to launch the program. This tool can help you to manage the Device Guard setting on local machine:

1. Start “gpedit.msc” in the “Run” command console to launch Group Policy Management Console.

2. In the left panel, choose Computer Configuration\Administrative Templates\System\Device Guard.

3. In the right panel, double click Turn On Virtualization Based Security, and then click Enabled.

4. For Select Platform Security Level, choose Secure Boot and DMA Protection.

5. For Virtualization Based Protection of Code Integrity, choose Enabled with UEFI lock.

6. Press OK; now the VBS and DMA protection is enabled and that needs system reboot for enforcement.

3.2 Create and deploy Kernel Mode Code Integrity policies Before deploying Code Integrity Policy, you need a golden system (HPE ProLiant server) to create your policy. Make sure all system drivers and applications are installed from Service Pack for ProLiant (SPP) image. And install all of your applications in the golden system.

The Code Integrity Policy is created based on the golden system, and then all other drivers and applications get blocked. The target result is full locked state of leveraging HVCI; therefore, we suggest you follow the flowchart mentioned in Section 3.5 to install your system and deploy Device Guard. It can help provide maximized protection for your environment.

At this point, we assume you have prepared the golden system and started to create policy:

1. Open PowerShell and run command:

> New-CIPolicy -Level PcaCertificate -Fallback Hash -FilePath c:\MyScan.xml

This cmdlet scans whole system to create a new policy file as c:\MyScan.xml. It may take couple of hours. Always keep this XML file because you may need to use the file to change some settings in the future. The Level parameter assigns the primary file rule. For binaries that cannot meet the requirement of primary file rule criteria, use the Fallback parameter. Here, we use hash file rule as a fallback if binaries does not have a signing certificate. Note the Fallback switch is allowed to have more than two items (string array). For high security demand environment, we suggest you use the following sequence for the Fallback parameter.

> New-CIPolicy -Level WHQL -Fallback Publisher, PcaCertificate, Hash -FilePath c:\MyScan.xml

WHQL signed drivers are always preferred security measures. Test signed or unsigned drivers should be used with caution.

2. You can disable or enable some policy setting within the XML file. You need to enable the audit mode (option 3) firstly. Run PowerShell command:

> Set-RuleOption -Option 3 -FilePath c:\MyScan.xml

Setting audit mode means any violation recorded in Event Viewer but is not blocked. Use audit mode for days and make sure no violation is observed. This step is important because if critical code is blocked, then your system could lead to BSOD or malfunction.

3. Convert the generated c:\MyScan.xml to binary format. Run the following PowerShell command:

> ConvertFrom-CIPolicy c:\MyScan.xml c:\MyScan.bin

4. Run “gpedit.msc”. In the left panel, choose Computer Configuration\Administrative Templates\System\Device Guard. On the right panel, double click Deploy Code Integrity Policy, and then select Enabled.

5. For Code Integrity Policy file path, input the MyScan.bin file path at “c:\Windows\system32\CodeIntegrity\MyScan.bin”, and then press OK. A SIPolicy.p7b file is generated from this step in the same folder. Windows uses this file as policy rule.

Technical white paper Page 4

Figure 1. Group policy editor for Code Integrity Policy

6. Reboot the system.

7. Launch Event Viewer from Server Manager\Tools. In the left panel, choose Application and Services Log\Microsoft\Windows\Device Guard\Operational. In the right panel, check the Event ID 7000 of the item in the top, and click the event item and Details Tab. Check if all required functions are enabled. VBS, DMA Protection, HVCI, and Security Boot should be enabled. Check the Event ID 7010, and click the event item and Details Tab. Check if the Policy File Path is correct and CIPolicy is enabled.

Figure 2. Examples of event details in Event Viewer

8. To check if there is any program that violates the code integrity, you can check the evet log from Application and Services Log\Microsoft\Windows\CodeIntegrity\Operational. You need to resolve these problems found in event log or the system cannot boot before you exit audit mode.

9. You can also use msinfo32.exe to check the Device Guard status.

Figure 3. Result of checking Device Guard status

Technical white paper Page 5

10. Windows Server 2016 has WMI class (Win32_DeviceGuard) related to Device Guard. You can use the following PowerShell command to get Device Guard information from it.

> Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\Microsoft\Windows\DeviceGuard

Explanation of output is described here.

11. If your system passes the previous steps, then you can turn off the audit mode to enforce the Code Integrity Policy. Run the following PowerShell command to turn off audit mode.

> Set-RuleOption -Option 3 -FilePath c:\MyScan.xml -Delete

12. Convert it to binary.

> ConvertFrom-CIPolicy c:\MyScan.xml c:\MyScan_enforced.bin

13. Run Steps 4 to 6 to apply the new policy file.

14. From “msinfo32.exe”, you see the system is in Enforced mode.

Figure 4. Result of running KMCI only

Note Before enforcing your change for new policy rules, test it by the audit mode (option 3, use Set-Rule Option to enable audit mode). You should make sure there is no violation of Code Integrity Policy from the event log before enforcing it. Any violation of kernel mode driver during system startup causes BSOD. Check every violation event and resolve each problem, and then you can enforce the new policy. We recommend to enable option 9 (Advanced Boot Option Menu—F8 key is enabled at startup.) and option 10 (Boot Audit on Failure—when driver fails during startup, the Code Integrity Policy is placed in audit mode so that Windows can continue booting system.) before testing the enforced policy. These two options can help you recover system quickly. If everything is tested OK, then you can remove both items to have full protection of your system.

3.3 Create and deploy User Mode Code Integrity policies The previous section describes how to enable Code Integrity Policy for kernel mode only (KMCI). If you need to enable code integrity for user mode program (UMCI), follow these steps.

1. Based on the policy generated from “Create and deploy Kernel Mode Code Integrity policies” section, run the following PowerShell command to enable UMCI.

> Set-RuleOption -Option 0 -FilePath c:\MyScan.xml

2. You need to enable audit mode before enforcing the UMCI:

> Set-RuleOption -Option 3 -FilePath c:\MyScan.xml

3. Convert it to binary and put it in the target folder:

> ConvertFrom-CIPolicy c:\MyScan.xml c:\MyScan_UMCI.bin

4. Run Steps 5 to 6 of “Create and deploy Kernel Mode Code Integrity policies” section to apply the new policy file and reboot system.

5. Run and test every application, which you want to be allowed to run under your new policy environment. Check if there is any program that violates the code integrity by opening Event Viewer and check for the log:

“Application and Services Logs\Microsoft\Windows\CodeIntegrity\Operational”.

Technical white paper Page 6

6. If there is any violation observed in the event log after system running in audit mode for a period of time, you can create new Code Integrity Policy from the event log to make the violated program be allowed to run. Run the following PowerShell command to generate policy based on event log: many information level logs show that the execution file does not meet enterprise-level signing.

> New-CIPolicy -Level PcaCertificate -Fallback Hash -UserPEs -FilePath C:\MyAuditPolicy.xml

This generates hash values for every application that violate the policy and use it as final rule for that application.

7. Review generated MyAuditPolicy.xml file; any application that were caught as exceptions, but should be allowed to run in your environment, should be left as-is in the .xml file.

8. Any application that should not be allowed to run in your environment must be removed from the .xml file.

9. Use the following PowerShell command to merge the MyAuditPolicy.xml with the MyScan.xml:

> Merge-CIPolicy -PolicyPaths c:\MyScan.xml,c:\MyAuditPolicy.xml -OutputFilePath c:\MyMergedPolicy.xml

10. Convert the merged MyMergedPolicy.xml to binary and move it to deploy folder:

> ConvertFrom-CIPolicy c:\MyMergedPolicy.xml c:\MyMergedPolicy.bin

> Copy c:\MyMergedPolicy.bin c:\Windows\system32\CodeIntegrity

11. Run Steps 5 to 6 of Section 3.3 to apply the new policy file and reboot system.

12. From msinfo32.exe, you see that the system is in audit mode

Figure 5. Result of running CI in audit mode

13. If you launch any untrusted application, you can find it in Event Viewer.

Figure 6. An example of Code Integrity Policy violation event

14. Run Step 5 to check if there is any violation again.

15. Run Steps 10 to 12 to disable audit mode and enforce the MyMergedPolicy.bin for system.

16. You can also use msinfo32.exe to check the Device Guard status.

Figure 7. Result of running msinfo32.exe

Technical white paper Page 7

17. You can start any other application to test the Code Integrity Policy with UMCI. For example, start the application for example, PUTTY.EXE that was not part of the golden image. You see Windows pop-up message box like the following.

Figure 8. An example of unregistered application, PUTTY.EXE blocked by UMCI

18. If you want to add PUTTY.EXE in allowed policy, first you need to have a new policy file for PUTTY.EXE and then merge it with existing policy.

> New-CIPolicy -Level PcaCertificate -Fallback Hash -ScanPath C:\Users\Administrator\Documents\putty -FilePath c:\putty.xml

> Merge-CIPolicy -PolicyPaths c:\putty.xml,c:\MyMergedPolicy.xml -OutputFilePath c:\MyMergedPolicy2.xml

19. Deploy c:\MyMergedPolicy2.xml to allow PUTTY execution.

3.4 Fine-tune CI Policy for different scenarios Code integrity policies include file and policy rules, which define several level of control policies. You can use Set-RuleOption PowerShell cmdlet to set policy rule options. More details can be found at technet.microsoft.com/itpro/windows/keep-secure/deploy-code-integrity-policies-policy-rules-and-file-rules. There are 0 to 10 options that can be enabled or disabled individually by the Set-RuleOption cmdlet. For UMCI (which is option 0), it is disabled by default. We recommend to enable option 2, Required:WHQL (default), at least to protect your Windows kernel from malware attack.

3.4.1 Create and deploy catalog files for Code Integrity Policy If you have unsigned applications for which the process of signing is difficult, you can create catalog file that contains information of the trusted applications. After you sign and distribute the catalog, your applications can be handled by code integrity policies as any other signed application. You can more easily block all unsigned applications, allowing only signed applications to run with these steps. The creation of catalog file is a necessary step for adding an unsigned application to a Code Integrity Policy.

You can use Package Inspector to create catalog file. You should run Package Inspector on the system and have Code Integrity Policy deployed in audit mode. Thus, Package Inspector can detect installation files that have been removed from the computer during the installation process.

1. Start Package Inspector. To begin, start scanning local drive C.

> PackageInspector.exe Start C:

2. Copy the installation file (.zip or .msi) to drive C and start installation to drive C. Do not run any application that you don’t want to capture in the catalog.

3. Open the application and close it. This step is necessary to ensure the scan can capture all related binaries.

4. To stop Package Inspector:

> PackageInspector.exe Stop C: -Name ApplicationName.cat -cdfpath ApplicationName.cdf

Technical white paper Page 8

Two files get generated. You can view the .cdf file with a text editor.

The catalog needs to be signed to be trusted by Code Integrity Policy. The signing certificate can be added to the Code Integrity Policy, and then the catalog file can be distributed to the client computers. You need an internal certification authority (CA) code signing certificate or purchased code signing certificate. If you do not have a code signing certificate, see Create a code signing certificate for detail of the steps.

To sign the catalog file, you need the SignTool.exe. You can find it in the Windows SDK (Windows 7 or later).

1. Sign the catalog file with SignTool.exe and your code signing certificate, for example, MyDGCert:

> SignTool.exe sign /n “MyDGCert” /fd sha256 /v ApplicationName.cat

2. Verify the catalog file digital signature. Right-click the catalog file, and then click Properties. On the Digital Signatures tab, verify your signing certificate exists with a SHA-256 algorithm.

3. To deploy the catalog file, copy the catalog file to C:\Windows\System32\catroot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}.

After the catalog file is signed, you can add the signing certificate to a Code Integrity Policy by the following two options.

Option A: If you already have an XML policy file, you can use Add-SignerRule to add the signing certificate to the Code Integrity Policy:

> Add-SignerRule -FilePath <PolicyPath> -CertificatePath <CertPath> -User

The PolicyPath is the path and file name of the original XML policy file. The CertPath is the path and file name of the certificate file (.cdf). Any application signed by the certificate is allowed to run for the policy. Run Steps 4 to 5 of Section 3.2 to deploy the modified policy.

Option B: If you want to create an XML policy for this code signing certificate, then you can merge it to your original XML policy file:

> New-CIPolicy -Level PcaCertificate -FilePath ApplicationName.xml -UserPEs

Run Steps 9 to 13 of Section 3.3 to merge the new policy with your original policy and deploy the new policy. Now, you can run your application to test your new deployed catalog file to see if there is any issue.

3.4.2 Sign Code Integrity Policy and deploy it You can sign your CI Policy to have highest level of malware protection. You can use the purchased code signing certificate or create the CA as mentioned in Create a code signing certificate to sign your policy. Before you sign your policy, be sure to enable rule option 9 (Advanced Boot Option Menu) and 10 (Boot Audit on Failure) to leave troubleshooting options available to administrators:

> Set-RuleOption -FilePath c:\MyPolicy.xml -option 9

> Set-RuleOption -FilePath c:\MyPolicy.xml -option 10

You need to import the .pfx code signing certificate, which does the signing. And then export the .cer version to your desktop or working folder. We need to use the .cer file to add to your policy.

> AddSignerRule -FilePath c:\MyPolicy.xml -CertificatePath <Path to .cer certificate> -kernel -User -Update

Then, remove the unsigned policy rule option; this enforces the verification of the policy signer:

> Set-RuleOption -FilePath c:\MyPolicy.xml -option 6 -Delete

And then, convert policy to binary:

> ConvertFrom-CIPolicy c:\MyPolicy.xml c:\MyPolicy.bin

Sign the policy by SignTool.exe:

> SignTool.exe sign -v /n “MyDGCert” -p7. -p7co 1.3.6.1.4.1.311.79.1 -fd sha256 c:\MyPolicy.bin

When complete, the MyPolicy.bin.p7 is generated. You can deploy the policy file as the same way previously.

Technical white paper Page 9

3.4.3 Disable unsigned or signed Code Integrity Policy To disable a deployed unsigned code integrity is very simple. It is just to delete the policy file in the following locations:

<EFI System Partition>\Microsoft\Boot\

<OS Volume>\Windows\System32\CodeIntegrity\

If the signed Code Integrity Policy is manually enabled and copied in the CodeIntegrity folder, you must complete the following steps to remove the policy:

1. Replace the existing policy in the following folder with another signed policy that have the rule option 6 (Unsigned System Integrity Policy) enabled.

<EFI System Partition>\Microsoft\Boot\<OS Volume>\Windows\System32\CodeIntegrity\>

The new policy must be signed with a certificate previously added to your policy. We recommend administrator should prepare this policy file every time you want to deploy the signed policy for future system maintenance.

2. Restart the client computer.

3. Verify that the new signed policy exists on the client computer.

4. Delete the new policy.

5. Restart the client computer.

It is possible for signed code integrity policies to cause a boot failure. There is a way to make system recovery from this condition.

1. Disable Secure Boot feature from BIOS RBSU menu.

2. Delete the file from the following locations on the operation system disk. This allows system to boot in Windows:

<EFI System Partition>\Microsoft\Boot\

<OS Volume>\Windows\System32\CodeIntegrity\

3.5 Best practices to enable Device Guard on HPE ProLiant servers Device Guard deployment process is complicated and requires planning for end-user or IT demand. Since it is critical to have HPE servers running with latest firmware and software updates by applying SPP, Figure 9 depicts the flow of actions needed to build a base golden system of Device Guard enabled HPE ProLiant server. Stage 1 is basic process to enable Device Guard for common conditions. Stage 2 creates and deploys policy for enterprise environment based on Stage 1.

Figure 9. Best practice for deploying HPE ProLiant SPP for setting up Device Guard

Technical white paper

Share now

Get updates

© Copyright 2018–2019 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

AMD is a trademark of Advanced Micro Devices, Inc. Intel is a trademark of Intel Corporation in the U.S. and other countries. Microsoft, Windows, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other third-party marks are property of their respective owners.

a00049396ENW, February 2019, Rev. 1

Figure 10. Best practice for deploying HPE ProLiant SPP for setting up Device Guard (continued)

4.0 Configure Credential Guard 1. Start “gpedit.msc” in the “Run” command console to launch Group Policy Management Console.

2. In the left panel, choose Computer Configuration\Administrative Templates\System\Device Guard.

3. In the right panel, double click Turn On Virtualization Based Security, and then click Enabled.

4. For Select Platform Security Level, choose Secure Boot and DMA Protection.

5. In the Credential Guard Configuration box, click Enabled with UEFI lock, and then click OK. If you want to be able to turn off Windows Defender Credential Guard remotely, choose Enabled without lock.

To enforce processing of the group policy, you can run gpupdate /force.