virtual server security for vmware: administrator guide

IBM Virtual Server Security for VMware

Administrator Guide for Virtual ServerSecurity for VMware(Proventia Server for VMware)Version 1.0

��

Copyright statement

© Copyright IBM Corporation 2009.IBM Global ServicesRoute 100Somers, NY 10589U.S.A.

U.S. Government Users Restricted Rights — Use, duplication or disclosure restricted by GSA ADP Schedule Contract withIBM Corp.

Publication Date: 02 December 2009

Technical support contacts

IBM Internet Security Systems (IBM ISS) provides technical support to customers that are entitled toreceive support. Information related to Customer Support hours of operation, phone numbers, andmethods of contact is available on the IBM ISS Customer Support Web page.

The IBM ISS Web site

The IBM ISS Customer Support Web page at http://www.ibm.com/services/us/iss/support/ providesdirect access to online user documentation, current versions listings, detailed product literature, whitepapers, the Technical Support Knowledgebase, and contact information for Customer Support.

Contact information

For contact information, go to the IBM ISS Contact Technical Support Web page at http://www.ibm.com/services/us/iss/support/contacts.html.

Other documentationThis documentation for the Proventia® Server for VMware agents was designed to provide theinformation you need to perform tasks as you work with the product. Additional documentation isavailable to help you with other tasks.

Related publications

The following documents are available for download from the Product documentation link on the IBMInternet Security Systems Web site at http://www.ibm.com/services/us/iss/support/:v Installation Guide for Virtual Server Security for VMware (Proventia Server for VMware)

v SiteProtector Policies and Responses Configuration Guide

License agreement

For licensing information on IBM Internet Security Systems products, download the IBM LicensingAgreement from: http://www.ibm.com/services/us/iss/html/contracts_landing.html

Documentation feedbackYour feedback about documentation is important to IBM Internet Security Systems.

Send comments about or suggestions for improving the technical documentation to mailto://[email protected].

© Copyright IBM Corp. 2009 iii

http://www.ibm.com/services/us/iss/support/

http://www.ibm.com/services/us/iss/support/contacts.html

http://www.ibm.com/services/us/iss/support/contacts.html

http://www.ibm.com/services/us/iss/support/

http://www.ibm.com/services/us/iss/html/contracts_landing.html

mailto://[email protected]

mailto://[email protected]

iv Virtual Server Security for VMware: Administrator Guide

Contents

Technical support contacts . . . . . . iiiOther documentation . . . . . . . . . . . iiiDocumentation feedback . . . . . . . . . . iii

Chapter 1. Getting started . . . . . . . 1Fundamentals of policy management . . . . . . 1First tasks . . . . . . . . . . . . . . . 2

Chapter 2. Configuring policies. . . . . 5Configuring the Virtual Objects policy . . . . . . 5

Virtual objects . . . . . . . . . . . . . 5Adding a virtual object . . . . . . . . . . 5

Configuring the Asset Settings policy . . . . . . 6Asset settings . . . . . . . . . . . . . 6Pass-through mode . . . . . . . . . . . 7Configuring network settings . . . . . . . . 8Configuring VM Settings . . . . . . . . . 9Defining the protection scope . . . . . . . 10Excluding assets from network monitoring . . . 10Excluding assets from VM configuration. . . . 11

Configuring the Update Settings policy . . . . . 11Update settings . . . . . . . . . . . . 11Configuring update settings . . . . . . . . 12Configuring license and update servers . . . . 13Scheduling update installations . . . . . . . 15Configuring advanced parameters . . . . . . 15Uninstalling intrusion prevention updates . . . 16Automatic updates advanced parameters . . . 16

Configuring the Security Events policy . . . . . 16Security Events policy . . . . . . . . . . 17Security Events . . . . . . . . . . . . 17Response Filters . . . . . . . . . . . . 23User-Defined . . . . . . . . . . . . . 28OpenSignatures . . . . . . . . . . . . 32

Configuring the Firewall policy. . . . . . . . 33Firewall policy . . . . . . . . . . . . 33Bypass filters . . . . . . . . . . . . . 34Configuring firewall rules . . . . . . . . 34Changing the order of firewall rules . . . . . 37Configuring bypass filters . . . . . . . . 38Firewall rule actions . . . . . . . . . . 39Firewall rule syntax . . . . . . . . . . 39

Configuring the Discovery policy . . . . . . . 40Discovery scanning . . . . . . . . . . . 41Configuring global discovery settings. . . . . 41Configuring exceptions to global discoverysettings . . . . . . . . . . . . . . . 42

Configuring the VM Events policy. . . . . . . 42VM events. . . . . . . . . . . . . . 42Configuring system events . . . . . . . . 44Configuring asset-specific events . . . . . . 44Updating authentication credentials . . . . . 45

Configuring the Anti-rootkit policy . . . . . . 46Rootkit detection . . . . . . . . . . . 46Configuring global anti-rootkit settings . . . . 47

Configuring exceptions to global anti-rootkitsettings . . . . . . . . . . . . . . . 48Excluding virtual machines from rootkitdetection . . . . . . . . . . . . . . 49

Configuring the Network Access Control policy . . 50Network Access Control . . . . . . . . . 50Creating a trusted asset list . . . . . . . . 51Creating an access control list for quarantinedassets . . . . . . . . . . . . . . . 51

Configuring the Agent Settings policy . . . . . 52Agent settings . . . . . . . . . . . . 52Configuring agent alerts . . . . . . . . . 53Configuring advanced parameters . . . . . . 53Agent-specific advanced parameters . . . . . 54

Chapter 3. Configuring filters . . . . . 59Controlling table display information . . . . . . 59Event filters . . . . . . . . . . . . . . 59Configuring event filters . . . . . . . . . . 60

Chapter 4. Configuring resourcemanagement . . . . . . . . . . . . 61Resource management . . . . . . . . . . . 61Configuring resource management settings . . . . 61

Chapter 5. Configuring responses . . . 63Responses . . . . . . . . . . . . . . . 63Configuring response objects . . . . . . . . 63

Chapter 6. Administering . . . . . . . 65Working with log files . . . . . . . . . . . 65

Logging packets from intrusion attempts . . . 65Viewing system log files . . . . . . . . . 65Forwarding remote log files to SiteProtector . . 66Log files available in Proventia Server forVMware agents . . . . . . . . . . . . 66

Working with agent health information . . . . . 67Health summary . . . . . . . . . . . 67Navigating to the Health Summary pane . . . 68Working with health status . . . . . . . . 68Working with agent messages . . . . . . . 70

Viewing information for agent components . . . . 71Module status . . . . . . . . . . . . 71Navigating to the Module Status pane . . . . 72Agent status - Agent Information . . . . . . 72Module Status - Network Monitoring. . . . . 73Module Status - Engine Status . . . . . . . 73Module Status - Engine Information . . . . . 73Module Status - Anti-rootkit . . . . . . . . 74Module Status - Discovery . . . . . . . . 74Module Status - VM Events . . . . . . . . 74

Monitoring agent command jobs . . . . . . . 74Command jobs . . . . . . . . . . . . 74Navigating to the Command Jobs pane . . . . 75

© Copyright IBM Corp. 2009 v

Chapter 7. Troubleshooting . . . . . . 77Seeing alerts for allowed traffic . . . . . . . . 77Agent is showing as offline in SiteProtector. . . . 77Traffic seems to be bypassing analysis . . . . . 78Troubleshooting issues with OneTrust . . . . . 78Unable to access the security virtual machine (SVM) 79

Appendix A. Adding or editing a virtualobject . . . . . . . . . . . . . . . 81

Appendix B. Informational links fromthe product interface. . . . . . . . . 83Can I edit this VM event? . . . . . . . . . 83Can I disable the global virtual object? . . . . . 83Help me understand how to define a trusted asset 83Help me understand how to define access control 83Help me understand how to define my protectionscope . . . . . . . . . . . . . . . . 83How does the Any firewall protocol work? . . . . 84How do I ensure the agent can authenticate? . . . 84How do I use virtual objects? . . . . . . . . 84How frequently should I scan the virtual machines? 85Tell me more about logging packets that matchfirewall rules . . . . . . . . . . . . . . 85Tell me more about network monitoring . . . . . 85

Tell me more about the intrusion response . . . . 85Tell me more about the pass-through mode. . . . 85Tell me more about the IBM ISS X-Force blockingrecommendations . . . . . . . . . . . . 86What do these trust levels mean? . . . . . . . 86What is a valid parameter name? . . . . . . . 86What is a valid parameter name for an updatesettings parameter? . . . . . . . . . . . . 86What is event throttling? . . . . . . . . . . 86What regular expressions are supported inuser-defined signatures? . . . . . . . . . . 87Where can I see the information gathered bydiscovery scans? . . . . . . . . . . . . . 88Why does the Asset-Specific tab have different VMevents than the System tab? . . . . . . . . . 88Why is the order of exceptions important? . . . . 88Why should I limit the number of exceptions Iconfigure? . . . . . . . . . . . . . . . 88Why should I schedule the installation of updates? 89

Notices . . . . . . . . . . . . . . 91Trademarks . . . . . . . . . . . . . . 92

Index . . . . . . . . . . . . . . . 93

vi Virtual Server Security for VMware: Administrator Guide

Chapter 1. Getting started

Before you begin to configure your Proventia Server for VMware policies, consider the followinginformation to ensure that your virtual environment is protected appropriately.

Fundamentals of policy managementPolicies control how agents protect the assets in your environment. The Proventia Server for VMwareagent protects virtual assets in a virtual environment. This topic provides background information aboutthe policy structure of the agent to help you understand how to best protect your virtual environment.

Ensuring protection in a virtual environment

In a virtual environment, virtual machines must be protected as they move from one server to another(VMotion). The VMware VMotion technology allows running virtual machines to move between physicalservers to maximize the efficiency of your virtual environment. To ensure that a virtual machine isprotected, even as it moves between servers, the agent enforces security policy settings based on the IPaddress of the virtual machine. This asset-based approach ensures consistent and appropriate protectionfor virtual machines regardless of the server that is hosting them.

Understanding asset-based policies

Asset-based policies define the settings for every virtual asset in your virtual environment, so the agentuses virtual objects to associate the appropriate setting with the appropriate virtual asset. By default, allvirtual assets belong to the global virtual object. As you configure asset-based policies, you define thesettings that apply to the global virtual object. If these settings are not appropriate for a particular virtualasset or a particular group of virtual assets, you can create a user-defined virtual object, which groupsthose particular assets, and then refer to that virtual object when you define custom settings for thecollection of virtual assets.

Understanding agent-based policies

Agent-based policies control agent-specific configuration settings, but do not control the protection thatthe agent provides. You can deploy agent-based policies to any group in your Site that contains aProventia Server for VMware agent. If you need granular control over a particular agent, you can create aunique version of any agent-based policy that is appropriate for only that agent.

Tip: If you group agents that can use the same settings for agent-based policies, you can minimize thenumber of agent-based policies that you must maintain.

Example: If you have ten agents that can use the same Update Settings policy settings, you can group allten agents in a single group, configure one Update Settings policy, and deploy the policy to the group.You only have one policy to manage. If, however, six agents can use the same Update Settings policywhile two agents need a second configuration, and the remaining two agent need a third configuration,you can group your agents into three separate groups and deploy a customized policy to each group. Youhave three policies to manage, but each agent has the configuration that it needs.

Understanding shared object policies

Shared object policies define reusable objects that can be referenced by other policies. Because sharedobject policies manage these reusable objects centrally , you can make changes to all of your policiessimply by editing the shared object.

© Copyright IBM Corp. 2009 1

Example: You create a new virtual machine that will function as a file server. You must ensure the assetis appropriately protected. You add the IP address of the new asset to the File Server virtual object. Theasset is protected by the policy settings defined for file servers as soon as the agent receives the updateddefinition of the virtual object.

Available policies

The agent uses several policies and policy types to provide the appropriate protection for the virtualassets in your virtual environment.

Policy type Purpose Deployment Policies

Asset-based Defines the appropriateprotection for a virtualmachine, even when thatvirtual machine movesfrom one server to anotherwithin the virtualenvironment

Deployed at the DefaultRepository level of the SiteGroup

Anti-rootkit

Asset Settings

Discovery

Firewall

Network Access Control

Security Events

VM Events

Agent-based Defines configurationsettings that are specific tothe agent

Deployed at the Site Grouplevel or to any subgroupwithin the Site GroupNote: You can deploy asmany agent-based policiesas you need, in order toachieve the configurationyou want.Example: You can deployone Update Settings policyat the Site Node if all theagents can use the samesettings, or you can deployone Update Settings policyat the group level if eachgroup needs customsettings.

Agent Settings

Group Settings

Update Settings

Shared Object Defines reusable objectsthat are referenced by otherpolicies to customizeprotection for your virtualassets

These policies are notdeployed.

Response Objects

Virtual Objects

First tasksThere are certain tasks you might want to perform before you configure settings for your ProventiaServer for VMware agent. These tasks help to maintain the availability of your asset and help you toconfigure policies.

2 Virtual Server Security for VMware: Administrator Guide

Task Description Reference

Define the protection scope The protection scope defines yourvirtual environment and does thefollowing things:

v Ensures that assets outside of thevirtual environment are not addedto the quarantine list

v Eliminates duplicate inspection ofa packet as it moves through thevirtual environment

To define the protection scope, see“Asset settings” on page 6

For more information aboutquarantine, see “Network AccessControl” on page 50

Define virtual objects Virtual objects define a collection ofvirtual assets (virtual machines). Thevirtual assets are identified by theirIP addresses. These virtual assets canthen be easily referenced within otherpolicies to provide the appropriateprotection to your virtualenvironment.

For example, you can use the IPaddresses you specified for theprotection scope to define a virtualobject and then use that virtual objectto define firewall rules for yourvirtual environment.

For more information about virtualobjects, see “Virtual objects” on page5

Define trusted assets The trusted assets list ensures thatvirtual assets that are known and arecompliant with your security policyare not added to the quarantine list.

For more information about trustedassets, see “Network Access Control”on page 50

Define a local update server An update server is a repository forupdates released by IBM ISS.

By default, agents communicatedirectly with the IBM ISS UpdateServer, but it is more efficient to haveagents communicate with theintegrated update server that is partof your SiteProtector installation.

For more information about updateservers, see “Configuring license andupdate servers” on page 13

Chapter 1. Getting started 3

Chapter 2. Configuring policies

This section contains information about how to configure the policies that control the behavior of theProventia Server for VMware agent.

Configuring the Virtual Objects policyThe Virtual Objects policy for the Proventia Server for VMware agent allows you to create reusableobjects that define collections of virtual assets. You can use these virtual objects to define customizedprotection for collections of virtual assets.

Virtual objectsVirtual objects define virtual assets (virtual machines). The virtual assets can then be easily referencedwithin other policies so that Proventia Server for VMware agents can provide the appropriate protectionto your virtual environment.

How the agent uses virtual objects

Asset-based policies use virtual objects. There are two types of virtual objects. A global virtual object isautomatically defined and it includes the entire network segment that is your virtual environment. Youdefine the policy settings for the global virtual object (all virtual assets) first. If the policy settings definedfor the global virtual object are not appropriate for certain virtual assets, you can create a user-definedvirtual object. Use the IP address assigned to a virtual asset to group assets in to custom virtual objects.You can then associate custom policy settings with that virtual object so that each virtual asset isprotected appropriately.

Important: The policy settings defined for user-defined virtual objects are always processed before anypolicy settings defined for the global virtual object. This ensures that your custom configurations areenforced.

How to group virtual assets into virtual objects

If you must have customized protection for collections of virtual assets, create virtual objects based on thereason the customization is needed. This approach will help you to ensure that a virtual object is usedappropriately and that virtual assets are not protected inappropriately.

Example: Your security policy dictates that you run scheduled discovery scans more frequently onmission critical virtual assets than on non-mission critical virtual assets. You can create a virtual objectcalled ″Mission-critical scan schedule″ that contains the IP addresses of your mission critical assets andthen use that virtual object to configure a discovery exception that defines a more frequent scan schedulefor these assets. By naming the virtual object specifically, you are less likely to use this virtual object in apolicy where the collection of virtual assets it defines is not appropriate.

Navigation

Locate the Virtual Objects policy in Default Repository → Shared Objects → Virtual Objects.

Adding a virtual objectAdd a virtual object when you need to configure customized protection for specific virtual assets.Proventia Server for VMware agents can enforce customized protection for a collection of virtual assetswhen you associate the virtual object that defines the collection of virtual assets with the customizedpolicy settings.


Before you begin

Determine which virtual assets should be included in the virtual object. As virtual objects control whichpolicy settings are used to protect a virtual asset, you should group assets based on a logical associationbetween the assets, such as function.

Attention: Use virtual objects only when you must have customized protection for a collection of virtualassets.

Procedure1. In the Navigation pane, click the Site Group and open the Policy view for Proventia Server for

VMware.2. Expand the Default Repository node, and expand the Shared Objects node.3. Right-click Virtual Objects, and click Open.

4. Click the Add icon.5. Select the Enable check box.6. Complete the following options:

Option Description

Virtual object name Specifies a descriptive name for the virtual objectNote: A well chosen virtual object name can help ensurethat you select the correct object when you select thevirtual object from a list.

Description Specifies a unique description for the virtual object

IP addresses Specifies the IP addresses of the virtual assets that makeup this virtual object

7. Click OK.

Results

The new virtual object is added to the table and can now be used when you define custom policysettings.

Configuring the Asset Settings policyThe Asset Settings policy for the Proventia Server for VMware agent allows you to customize certainsettings specific to the protection of your virtual assets. The settings in this policy apply to all agents inthe repository where the policy is deployed from.

Asset settingsThe Asset Settings policy in the Proventia Server for VMware agent allows you to configure options fornetwork settings and virtual machine settings.

Asset settings apply to all agents in the repository that the policy is deployed from.


Network Settings

Component Description

Protection scope Define the protection scope:

v To ensure that you do not add assets to the quarantinelist that are outside of the virtual environment

v To eliminate duplicate inspection of a packet as itmoves through the virtual environment

Tip: You can copy and paste the contents of theProtection scope field into a virtual object and then usethe virtual object as you define firewall rules.Important: The Protection scope must include all of thevirtual assets available on the ESX server and shouldnever contain IP addresses of any asset external to theESX server.

Network Monitoring Turn Network Monitoring Off to disable the Firewallpolicy and the Security Events policy.

Intrusion Response When the agent is set to Block and Alert, the agentenforces the Block response that is set in each securityevent signature. When the agent is set to Alert only, thenintrusions are only reported, regardless of the blockstatus. By default, the agent is set to Alert only.

Pass-through Mode The Pass-through Mode option specifies how the agenthandles traffic during overload conditions. By default,the agent is set to Fail open.

VM Settings

Component Description

VM Configuration File Update By default, the agent is configured to manually pauseand resume. Therefore, you must pause and resume eachvirtual machine that needs to be updated. Considerscheduling a maintenance window.

The agent provides protection to the virtual machine bymodifying the virtual machine configuration file (VMXfile). To receive the update to the VMX file and to beginprotection, each of the virtual machines must be pausedand resumedImportant: Until each virtual machine is paused andresumed, no protection is provided by the agent.Recommendation: Create an exception to pause andresume non-mission critical virtual machinesautomatically.

Navigation

Locate the policy you want to edit in the Default Repository.

Pass-through modeThe Proventia Server for VMware agent monitors all traffic to and from the server and all traffic passingbetween virtual machines on the server. During overload conditions, traffic may be delayed as the agenttries to process the high load. You can configure the agent to allow traffic to pass-through the agent toprevent any possible delays.

Chapter 2. Configuring policies 7

Important: During overload conditions, the queue between the VMsafe interface and the netengine mightget flooded. If the fail condition is set to open, all traffic passes through without analysis. If you cannottolerate any traffic passing without analysis, you can customize your environment by quarantining thevirtual machines or setting the policy to fail closed.

Causes of overload conditions

Situations that may result in an overload condition include:v When traffic rates consume system resources to the point where the agent may drop packets.v When the number of events generated by the agent consumes excessive system resources.

Note: The number of events generated by the agent depends on the amount of traffic being processedand the settings enabled in the policy.

v When some internal agent situation causes a failure of the agent

Note: The agent has many built-in recovery processes that make an internal failure unlikely.

Choosing a mode

Each pass-through mode has risks, so determine whether your security policy requires that all trafficreach its destination (failopen) or that the security of the system is not compromised (failclosed). Certainregulations such as HIPAA, SOX, SB1386, and BASEL II require the protection of digital data and maydictate which pass-through mode you should set.

Note: If your security requirements dictate that all traffic is analyzed under any circumstance, ensure youselect fail closed as the pass-through mode and disable the engine.pamlook.enabled advanced parameterin the Agent Settings policy.

Default setting

By default, the agent fails open during overload conditions; the fail open pass-through mode allows alltraffic to reach its destination without being inspected for malicious content.

Configuring network settingsNetwork settings define how the Proventia Server for VMware agent manages certain network functions.


VMware.2. Right-click Asset Settings, and click Open.3. In the Network Settings tab, specify the IP addresses of the virtual network that should be

monitored for new virtual machines in the Protection Scope field.

Important: You must define the protection scope in the Asset settings policy before the settings inthe NAC policy can take effect.

4. Ensure that the Enable check box is selected.


5. Configure the Global Settings that you want to apply to all virtual assets:

Option Description

Network Monitoring By default, the agent processes all network traffic againstfirewall rules and security events. You can turn thisoption Off to disable the agent firewall and all intrusionprotection.Important: Allowing traffic to bypass the protectionoffered by the agent may impact the integrity of yourserver.

Intrusion Response The intrusion response controls the behavior of theSecurity Events policy.

Select from the following options:

v Block and Alert Blocks any malicious traffic and sendsan alert to the management console

v Alert only Sends an alert to the management consolewhen malicious traffic is detected, but allows thetraffic to pass

Pass-through mode The pass-through mode specifies how the agent handlestraffic during overload conditions.


v Fail open Allows traffic to pass through in the eventof traffic overload; the agent provides no protectionagainst malicious packets in traffic that passes through

v Fail closed Rejects all traffic in the event of overload;the agent does not process any packets

6. If the Network Monitoring option you selected in Step 6 does not apply to all virtual assets, create

an exception in the Exceptions area, by clicking the Add icon.7. Select the Enabled check box.8. In the Virtual object field, select the collection of virtual assets that this configuration applies to.

Note: The topic Appendix A, “Adding or editing a virtual object,” on page 81 describes how towork with virtual objects as you configure your policy.

Important: If you edit a virtual object as you configure this tab, the changes you make apply toevery policy that uses the virtual object.

9. In the Network Monitoring area, choose the appropriate setting.10. Click OK.

Configuring VM SettingsCertain situations, such as when a new virtual machine comes online require the Proventia Server forVMware agent to pause and resume the virtual machine to update the VMX file. In addition, each virtualmachine that requires rootkit detection must also be paused and resumed. The VM Settings tab defineswhether the pause and resume operation should be managed automatically or manually.


VMware.2. Right-click Asset Settings, and click Open.3. In the VM Configuration File Update area, choose how to enable protection for every virtual machine

from the following options:


v Manual: You must manually pause and resume each virtual machinev Automatic: The agent automatically pauses and resumes each of the virtual machines

Important: Until each online virtual machine is paused and resumed, no protection is provided bythe agent.

4. If the VM Configuration File Update option you selected in step 3 does not apply to all virtual assets,

create an exception in the Exceptions area, by clicking the Add icon.5. Select the Enabled check box.6. In the Virtual object field, select the collection of virtual assets that this configuration applies to.

Note: The topic Appendix A, “Adding or editing a virtual object,” on page 81 describes how to workwith virtual objects as you configure your policy.

Important: If you edit a virtual object as you configure this tab, the changes you make apply to everypolicy that uses the virtual object.

7. In the VM Configuration File Update area, choose the appropriate setting.

Important: Until each virtual machine is paused and resumed, no protection is provided by the agent.

What to do next

If you chose to manually pause and resume any virtual machine, remember you are not protected untilyou pause and resume each virtual machine.

Defining the protection scopeThe protection scope in the Proventia Server for VMware agent defines all of the IP addresses that arepart of your virtual environment and that should be protected.

About this task

You should complete this task if you have added any IP addresses to your virtual environment since youfirst defined your protection scope.


VMware.2. Right-click Asset Settings, and click Open.3. In the Network Settings tab, specify the IP addresses of the virtual network that should be monitored

for new virtual machines in the Protection Scope field.

Important: You must define the protection scope in the Asset settings policy before the settings in theNAC policy can take effect.

Excluding assets from network monitoringYou can exclude a specified group of assets from network monitoring by creating an exception in theProventia Server for VMware agent.


VMware.2. Right-click Asset Settings, and click Open.3. In the Network Settings tab, ensure that the Enable check box is selected.


4. In the Exceptions area, click the Add icon.5. Select the Enabled check box.6. In the Virtual object field, select the collection of virtual assets that this configuration applies to.



7. In the Network Monitoring area, choose the appropriate setting.8. Click OK.

Excluding assets from VM configurationYou can exclude a specified group of assets from manually updating the VM configuration file bycreating an exception in the Proventia Server for VMware agent.

About this task

You might want to create an exception to automatically pause and resume non-mission critical virtualmachines. For example, if you have a virtual machine that can tolerate being paused and resumedautomatically, you can create an exception by referencing a virtual object that contains the IP address ofthe virtual machine.


VMware.2. Right-click Asset Settings, and click Open.

3. In the VM Settings tab, click the Add icon in the Exceptions area.4. Ensure the Enabled check box is selected.5. In the Virtual object field, select the collection of virtual assets that this configuration applies to.



6. In the VM Configuration File Update area, choose the appropriate setting.7. Click OK.

Configuring the Update Settings policyIBM ISS issues frequent updates for Proventia Server for VMware agents in the form of security contentupdates and core updates. To ensure that agents protect your system effectively, you must stay currentwith these updates.

Update settingsIBM ISS issues frequent updates for Proventia Server for VMware agents. These updates can be eithercore updates, which include feature updates and product updates, or security content updates.

You can configure how frequently the agent checks for updates. You can manage security content updatesand core updates separately. By managing your updates separately you have the following advantages:


v You can automatically download and install security content updates, which ensures that your systemhas the most recent and comprehensive protection

v You can control when core updates are installed, which ensures updates are deployed at a convenienttime and after you have tested them in a nonproduction environment

Types of updates

Type Description

Security Contains intrusion prevention, discovery and anti-rootkitupdates, and other updates from the IBM® ISS X-Force®

Core Contains changes to the agent’s operating software:

v Feature updates are minor releases at the decimalrelease version. For example, upgrading from 1.5 to 1.6is a feature update.

v Product updates are major releases at the integerrelease version. For example, upgrading from 1.0 to 2.0is a product update.

Production vs nonproduction environments

Sometimes you do not want to install updates on production systems until you have tested them innonproduction environments. This is particularly true for core updates, which must frequently beapplied, but only during change control windows. You can control the update process in the followingways:v For core updates, you can use the Update Settings tab to control the version of the updates that are

downloaded and installedv For core updates and security content updates, you can use the Scheduled Installation tab to schedule

when updates should be installedv For core updates and security content updates, you can manually manage your update process

Note: This approach is not recommended as the best way to protect your assets is to keep current withupdates (particularly security content updates).

How the update process works

The agent periodically checks for available updates by connecting to the first update server on the list ofupdate servers. If the catalog file on the update server is more recent than the catalog file the agent isusing, the agent does one of the following things, depending on how you have configured the agent:v Downloads and installs the update immediatelyv Updates the Update Status column in the Agent view, if you do not want to install the update

immediately

Navigation

Locate the policy you want to edit and then click Update Settings.

Configuring update settingsYou can configure how frequently Proventia Server for VMware agents check for available security andcore updates.


Procedure1. In the Navigation pane, expand the group the agent is assigned to, right-click Update Settings, and

click Open.2. Click the Update Settings tab.3. Complete the following options:

Option Description

Automatically Check For Updates Select from the following options:

v Daily or weekly: Specifies the day of week and timeof day when the agent checks for updates

v At specified intervals: Specifies how frequently, inminutes, the agent should check for updates

Security Updates Select from the following options:

v Automatically download updates and install them:Specifies that any applicable updates are automaticallydownloaded and installed

v Do not automatically download or install updates:Specifies that any applicable updates are notautomatically downloaded and installedNote: If you select this option, you must manuallymanage the download and installation of securitycontent.

Note: These options apply to the intrusion prevention,discovery, and anti-rootkit updates.

Core Updates Select from the following options:

v Ignore core updates or feature upgrades later thanversion specified: Allows you to only download andinstall upgrades to a specified version (by ignoringany upgrades that come after that version)

v Automatically download updates and install them:Specifies that applicable updates are automaticallydownloaded and installed

v Do not automatically download or install updates:Specifies that applicable updates are not automaticallydownloaded and installedNote: If you select this option, you must manuallymanage the download and installation of core updatesand feature upgrades.

Configuring license and update serversUpdate servers are a repository for updates released by IBM ISS. By default, Proventia Server forVMware agents communicate directly with the IBM ISS X-Press Update Server, however, it is moreefficient to have agents obtain updates from the integrated update server that is part of your SiteProtectorinstallation.

About this task

The integrated update server (local update server) that is part of your SiteProtector installation candownload all available updates for any IBM ISS agent. The local update server then acts as a centralrepository for all IBM ISS agents in your Site. When you define a local update server, you increase theefficiency of retrieving updates because only one system (the local update server) has to communicatewith the IBM ISS X-Press Update Server. Without the local update server, each agent would have toconnect to the IBM ISS X-Press Update Server to retrieve updates.


You can also configure the agent to use a secondary update server if the primary update server becomesunavailable. Specify primary and secondary update servers by ordering the servers in the list.


click Open.

2. In the License and Update Servers tab, click the Add icon to add a new server or select an

existing server and click the Edit icon to edit the server settings.3. If you want to use this server after you configure the options, select the Enabled check box.4. Complete the following options:

Option Description

Name Specifies a descriptive name for the server

Host or IP Specifies the server DNS name or IP address

Port Specifies the port the server listens to for downloadrequests

v For SiteProtector X-Press Update Servers, the defaultport is 3994.

v For the Download Center (www.iss.net), the port is443.

v For the OneTrust Server (xpu.iss.net), the port is 443.

Trust level Specifies how authentication between the agent and thelicense or update server is managed.


v trust-all: This agent trusts the server. No certificatesare used for authentication.

v first-time-trust: This agent trusts the server once anduses the server’s certificate for all futureauthentication.

v explicit-trust: This agent will use the local certificate toauthenticate the server.

5. If there is a firewall or proxy server between the agent and the update or license server, select Specifya proxy for this server and complete the following options:

Option Description

Proxy host Specifies the DNS or IP address of the proxy server orthe firewall

Port Specifies the port number that the server uses tocommunicate with the proxy server or the firewall

6. If there is an authenticating proxy between the agent and the update or license server, select Enableproxy authentication and complete the following options:

Option Description

User name Specifies the user name required by the authenticatingproxy server

Password Specifies the password required by the authenticatingproxy server

7. Click OK.


Scheduling update installationsYou can schedule the installation of core and security updates in the Proventia Server for VMware agents.


click Open.

2. In the Scheduled Installations tab, click the Add icon to add an update schedule or select an

existing update schedule and click the Edit icon to edit the schedule.3. Complete the following options:

Option Description

Update type Identifies what type of update is being scheduled.


v Core

v Intrusion prevention

v Discovery and anti-rootkit

Version Specifies the version of the product this schedule appliestoNote: This field is only available for Core updates.

Update Describes the core update that gets translated into theupdate-setting.conf file on the agentNote: This field is only available for Core updates.

Update time Specifies the time and date the update should beinstalled

Description Describes the purpose of the schedule

4. Click OK.

Configuring advanced parametersIn the Proventia Server for VMware agents, you can use advanced parameters to fine-tune the updateprocess.

Before you begin

The agent does not validate the name, type, or value of advanced parameters; therefore, you must ensureyou configure advanced parameters correctly.

About this task

Advanced parameters fine-tune the performance of your agent; however, assigning inappropriate settingsto an advanced parameter might have significant negative effects on the behavior of the agent. If youenter conflicting or duplicate parameters, the parameter entered last overrides the parameter entered first.


click Open.

2. In the Advanced Parameters tab, click the Add icon to add a new parameter or select an existing

parameter and click the Edit icon to edit the parameter.


3. Complete the following options:

Option Description

Name Specifies the name of the parameter

Description Describes the purpose of the parameter

4. In the Value area, select one of the value types:

Option Description

Boolean Indicates this parameter has a Boolean valueNote: Selecting Enabled sets the Boolean value to True;clearing Enabled sets the value to False.

Number Indicates this parameter has a numeric value

String Indicates this parameter has a string value

5. Click OK.

Uninstalling intrusion prevention updatesYou can uninstall intrusion prevention updates from the Proventia Server for VMware agents.

Procedure1. In the Navigation pane, click the Site Group and open the Agent view for Proventia Server for

VMware.2. Right-click the Proventia Server for VMware agent, and click Updates → Remove Last XPU.3. Click OK.

Automatic updates advanced parametersThis topic describes advanced parameters that apply to automatic updates in the Proventia Server forVMware agent.

NameValueType

DefaultValue Description

update.update.directory String /var/spool/updates

Specifies the fully qualified path to locationwhere update packages are downloaded to

update.update.logs.directory String /var/spool/updates/logs

Specifies the fully qualified path to locationof update installation and uninstallation logfiles

update.history.events.max Number 100 Specifies the maximum number of entries tokeep in the update history fileNote: When the file reaches the maximumnumber of entries, older entries areoverwritten with newer entries.

update.download.timeout Number 900 Specifies the number of seconds allowed fora download to complete

Configuring the Security Events policyThe Security Events policy defines the security policy that determines how your Proventia Server forVMware agent responds to and reports security events that occur on your network.


Security Events policyThe Security Events policy defines the type of network activity the Proventia Server for VMware agentmonitors for. If the agent detects suspicious activity, it can block the traffic to protect your system. Youcan customize certain attributes of the security event signatures to better meet your security needs.

Tab Description

Security Events A security event is network traffic with content that canindicate an attack or other suspicious activity. Theseevents are triggered when network traffic matches one ofthe signatures in the security policy.

Response Filters Response filters control the responses the agent takesagainst malicious traffic. When a packet matches aresponse filter, the agent executes the responses specifiedin the filter; otherwise, the agent executes the responsesspecified in the security event configuration.

User-Defined User-defined signatures specify the type and part of anetwork packet that you want to monitor.

OpenSignatures Open signatures use a flexible rules language to definepattern-matching IDS signatures that detect specificthreats which are not already covered by the signaturesin the Security Events tab.

Intrusion response setting

The Intrusion Response setting and the Network Monitoring setting in the Asset Settings policy impacthow the agent implements settings in the Security Events policy. You should review the Asset Settingspolicy when you configure the Security Events policy.

Navigation


Security EventsSecurity events are predefined signatures that the Proventia Server for VMware agent uses to monitor forfrequently encountered security events that may pose a threat to your system.

Customizing security event signaturesThe Proventia Server for VMware agent comes with pre-defined signatures that analyze network traffic.You can customize certain attributes of these pre-defined signatures to better meet your security needs.

Before you begin


About this task

Quarantine responses are defined at the Site Group level in the SiteProtector Central Responses policy. Ifyou want the agent to use a quarantine response, add a quarantine response using the Central Responsespolicy in SiteProtector.

Tip: See the Central Responses Help (open the Central Responses policy and press F1) for informationabout how to configure central responses.


You can also specify whether you want to use blocking responses that are provided by the IBM ISSX-Force. You may need to disable the X-Force blocking occasionally so that you can determine whethercurrent suspicious activity on your network is valid, or so that you can protect against explicit threats toyour network.


VMware.2. Right-click Security Events, and click Open.3. To disable the X-Force blocking recommendations, clear the Enable X-Force blocking

recommendations check box.4. Click the plus sign (+) to expand the group that contains the signature you want to customize.5. Select the signature you want to customize.

6. Click the Edit icon.7. Select the Enabled check box.8. In the Virtual object field, select the collection of virtual assets that this configuration applies to.



9. Change the following options as necessary:

Option Description

Signature Specifies a unique descriptive name for the signatureNote: If you are editing an existing signature, thesignature name appears. Click More Information to viewa brief description of the event.

Severity Specifies the severity level assigned to this signature.


v High

v Medium

v Low

Note: The default severity is assigned by the X-Force;you can change the severity if this type of attack poses adifferent threat to your system.

Event throttling Sets a time window (in seconds) during which multipleevents are reported only onceTip: Use this feature to prevent your console from beingoverrun with duplicate events that might potentiallymask a more dangerous event.Note: The default value is 0 (zero), which disables eventthrottling.

10. To configure responses for this signature, change the following options:

Option Description

Ignore events Instructs the agent to ignore events that match thecriteria set for the event


Option Description

Display in console Specifies how you want to display the event in themanagement console.


v No display: Does not display the detected event

v Without Raw: Logs a summary of the event

v With Raw: Logs a summary and the associated packetcapture

Block Instructs the agent to block the attack by droppingpackets and sending resets to TCP connections

Log evidence Instructs the agent to log the packet that triggered theevent to the /var/iss/ directory

11. In the Quarantine Response area, select the appropriate Enabled check boxes to enable quarantineresponses.

Note: Quarantine responses indicate the response the agent should take when this event occurs.Each signature can have any combination of responses or no responses at all.

12. Click the Edit icon to change the properties of a quarantine response in the list.

Note: Quarantine responses are defined in the Central Responses policy. If you want the agent touse a quarantine response, add a quarantine response using the Central Responses policy inSiteProtector.


Option Description

Enabled Enables the response as part of your security policy

Duration Specifies the duration (in seconds) of the quarantine rule

v Valid range: 1 to 86400 seconds (24 hours)

v Default: 3600 seconds (1 hour)

Percentage Specifies the percentage of traffic to be quarantinedNote: The default value is 100 percent, which indicatesthat all traffic should be quarantined. You can specify alower percentage to improve agent performance.Example: You may want to quarantine a lowerpercentage of traffic for a denial of service (DoS) attackor distributed denial of service (DDoS) attack, since notall of that traffic will be malicious.

14. Click OK.

Configuring responses for security eventsUse the Quarantine Response area of the Security Events tab in the Proventia Server for VMware agent toconfigure how the agent notifies you about security events. Quarantine responses specify responses thatblock intruders, including worms and Trojans, when the agent detects connection events.

Before you begin




About this task

Quarantine responses indicate the response the agent should take when this event occurs. Each signaturecan have any combination of responses or no responses at all.


VMware.2. Right-click Security Events, and click Open.

3. In the Security Events tab, click the Add icon.4. In the Quarantine Response area, select the appropriate Enabled check boxes to enable quarantine

responses.

5. Click the Edit icon to change the properties of a quarantine response in the list.6. Change the following options as necessary:

Option Description

Enabled Enables the quarantine response as part of your securitypolicy


v Valid range: 1 to 86400 seconds



7. Click OK.

Configuring the intrusion responseYou can set one intrusion response for all enabled security event signatures in the Proventia Server forVMware agent Security Events policy. The global action setting configures all enabled signatures at thesame time, so that you do not have to configure each enabled signature separately.

Before you begin



VMware.2. Right-click Asset Settings, and click Open.


3. In the Intrusion Response area, select one of the following options:

Option Description

Block and alert Blocks any malicious traffic and sends an alert to themanagement console

Alert only Sends an alert to the management console whenmalicious traffic is detected, but allows the traffic to pass

Editing multiple security eventsYou can edit multiple security events that you have grouped and filtered on your Proventia Server forVMware agent.


VMware.2. Right-click Security Events, and click Open.3. In the Security Events tab, select the parent row for the group of events you want to edit.

Note: Selecting the parent row selects all the events in that category.

4. Click the Copy icon.

5. Click the Paste icon.

6. Click the Edit icon to edit all the selected events.

Note: A blue triangle icon appears next to any item in the events selected that has a different value. Ifyou change the value of a field for an event that has this icon, the value changes for all selectedevents and the blue triangle icon no longer appears next to the field. Example: If you select to edittwo events and one has blocking enabled and the other does not, a blue triangle appears next toBlock. If you enable the block response on the event that was originally disabled, then both eventshave blocking enabled, and the blue triangle disappears.

7. Click OK.

Configuring the IBM ISS X-Force blocking recommendationsWhen you use the IBM ISS X-Force blocking recommendations in the Proventia Server for VMware agent,the block response is enabled automatically for events (or signatures) according to the X-Forcerecommendations.

About this task

If you change the X-Force blocking recommendations setting, you must save and then reopen the SecurityEvents policy to see the changes.


VMware.2. Right-click Security Events, and click Open.3. To disable the X-Force blocking recommendations, clear the Enable X-Force blocking

recommendations check box.4. Save and then reopen the Security Events policy to see the changes take effect.

Security event signature propertiesThis topic describes the security event signature properties in the Proventia Server for VMware agent.


Property Description

Enabled Enables or disables this signature

Virtual object Specifies the virtual object that this signatureconfiguration applies toNote: The topic Appendix A, “Adding or editing avirtual object,” on page 81 describes how to work withvirtual objects as you configure your policy.

Attack/Audit Specifies the type of event detected by this signature:

v Attack: Events that match network traffic seeking toharm your network

v Audit: Events that match network traffic seekinginformation about your network

Signature Indicates the name of the signature

Severity Specifies the severity level assigned to this signature:

v High

v Medium

v Low

Note: The default severity is assigned by the IBM ISSX-Force; you can change the severity if this type of attackposes a different threat to your system.

Protocol Specifies the protocol used by this attack (ICMP=1,TCP=6, UDP=17)Note: For existing events, this field displays the protocoltype, which is uneditable.

Event throttling Sets a time window (in seconds) during which multipleevents are reported only onceNote: The default value is 0 (zero), which disables eventthrottling.Tip: Use this feature to prevent your console from beingoverrun with duplicate events that might potentiallymask a more dangerous event.

Ignore events Shows whether you have selected to ignore events thatmight occur on secure and trusted hosts on yournetwork or on hosts that you want the agent to ignorefor any reason.

Display in console Specifies how you want to display the event in themanagement console:






Quarantine Response Indicates the response the agent should take when thisevent occurs

XPU Indicates the XPU release that contained this signature

Signature Date Indicates the date IBM ISS created this signature

Default protection Displays the default protection set for the event, such as″Block″


Property Description

User overridden Indicates a custom event when you create a new eventNote: This is enabled for custom events, or forpredefined events that have been edited.

Response FiltersResponse filters add an extra layer of control to the responses the Proventia Server for VMware agenttakes against malicious traffic. If, based on specific network criteria (such as a specific IP or port), apacket matches a response filter, the agent executes the responses specified in the filter; otherwise, theagent executes the responses specified in the security event configuration.

Configuring response filtersUse the settings on the Response Filters tab in the Proventia Server for VMware agent to configureattributes such as signature name, quarantine responses, and event throttling.

Before you begin


About this task

Response filters can control the number of events the agent responds to and the number of eventsreported to the management console.

Example: If you have hosts on your network that are secure and trusted or hosts that you want the agentto ignore for any other reason, you can use a response filter with the IGNORE response enabled.

Quarantine responses are defined at the Site Group level in the SiteProtector Central Responses policy. Ifyou want the agent to use a quarantine response, you must add a quarantine response using the CentralResponses policy in SiteProtector.




3. In the Response Filters tab, click the Add icon.4. Select the Enabled check box.5. In the Virtual object field, select the collection of virtual assets that this configuration applies to.





Option Description

Signature Displays a truncated signature nameNotes:

v Click the Select button to display signatures.

v You can add multiple signatures at one time. Use thefilter settings to sort through the list.

Description Specifies a unique description for the event filter or set offilters



v High

v Medium

v Low



ICMP Type Specifies ICMP types for either side of the packetNote: Click Well Known to select often-used types.

ICMP Code Specifies ICMP codes for either side of the packetNote: Click Well Known to select often-used codes.

Ignore events Instructs the agent to ignore events that match thecriteria set for the event














Option Description

Enabled Enables the response as part of your security policy





10. Configure the Source Address tab and the Destination Address tab using the information in thefollowing table.

Option Description

Virtual object specified above Specifies the collection of virtual assets in the Virtualobject field

Single IP Address Specifies a single IP address

IP address range Specifies a range of IP addresses

Network address/CIDR format Specifies an IP address on a subnet mask the agent filtersNote: The mask is the network identifier, and is anumber from 1 to 32.Example: 192.0.2.0 /24

11. Configure the Source Port tab and the Destination Port tab using the information in the followingtable.

Option Description

Any Filters all ports

Single port Specifies a single port the agent filters

Port range Specifies a range of port numbers the agent filters

12. Click OK.

Configuring responses for response filtersUse the Responses area on the Response Filters tab in the Proventia Server for VMware agent toconfigure how the agent notifies you about events triggered by your filters.

Before you begin



About this task





3. In the Response Filters tab, click the Add icon.4. In the Quarantine Response area, select the appropriate Enabled check boxes to enable quarantine

responses.

Note: Quarantine responses indicate the response the agent should take when this event occurs. Eachsignature can have any combination of responses or no responses at all.


Note: Quarantine responses are defined in the Central Responses policy. If you want the agent to usea quarantine response, add a quarantine response using the Central Responses policy in SiteProtector.


Option Description

Enabled Enables the quarantine response as part of your securitypolicy





7. Click OK.

Changing the order of response filtersThe Proventia Server for VMware agent executes the responses specified in the filter in a specific orderwhen it processes network traffic. The agent reads the list of filters from top to bottom, so place theresponse filters in an order that provides the most effective Security Events policy for your network.

About this task

The response filters follow rule ordering. For example, if you add more than one filter for the samesecurity event, the agent executes the responses for the first match. The agent reads the list of filters fromtop to bottom.



VMware.2. Right-click Security Events, and click Open.3. In the Response Filters tab, select the response filter you want to change the order for.

4. Click the Move Up icon or the Move Down icon to move the rule higher or lower in the list.

Response filter propertiesThis topic lists the information about the Response Filters tab on your Proventia Server for VMwareagent.

Option Description

Enabled Enables or disables the response.

Virtual object Specifies the virtual object that this signatureconfiguration applies to.Note: The topic Appendix A, “Adding or editing avirtual object,” on page 81 describes how to work withvirtual objects as you configure your policy.

Signature The name of the events that have this filter applied.

Description A unique description of the event.

Severity Specifies the severity level assigned to this signature:

v High

v Medium

v Low


Event throttling Sets a time window (in seconds) during which multipleevents are reported only once.Tip: Use this feature to prevent your console from beingoverrun with duplicate events that might potentiallymask a more dangerous event.Note: The default value is 0 (zero), which disables eventthrottling.

ICMP Type The set of ICMP types for either side of the packet.

ICMP Code The set of ICMP codes for either side of the packet.

Ignore events Reduces the number of events reported to the console.Note: If you have hosts on your network that are secureand trusted or hosts that you want the agent to ignorefor any other reason, you can use a response filter withthe Ignore response enabled.

Display in console Specifies how you want to display the event in themanagement console:

v No Display: Does not display the detected event


v With Raw: Logs a summary and the entire binarycontent of a session

Block Instructs the agent to block the attack by droppingpackets and sending resets to TCP connections.

Log evidence Instructs the agent to log the packet that triggered theevent to the /var/iss/directory.


Option Description

Quarantine Response Shows whether a quarantine response is configured forthe event.

Source Address The source IP address.Note: The default is ″Any.″

Source Port The source port number of packets.

Destination Address The destination IP address.Note: The default is ″Any.″

Destination Port The destination port number of packets.

User-DefinedUser-defined signatures specify the type and part of a network packet that you want to monitor. Createuser-defined signatures in the Proventia Server for VMware agent to monitor for custom events thatmight threaten your system.

Configuring user-defined signaturesEnabled events in a policy determine what the Proventia Server for VMware detects. You createuser-defined events around contexts, which basically specify the type and part of a network packet youwant the agent to scan for events.

Before you begin


About this task





3. In the User-Defined tab, click the Add icon.4. Select the Enabled check box.5. Type a descriptive name for the user-defined event in the Name field.6. In the Virtual object field, select the collection of virtual assets that this configuration applies to.





Option Description

Description Describes the purpose of this signature



v High

v Medium

v Low


Context Indicates the type and part of the network packet thatthe agent should scan

Search string Type the text string in the packet (context) thatdetermines whether an event matches this signature. Youcan use wildcards and other expressions in strings. Youmust follow standard POSIX regular expression syntax.For example, a period is a wildcard character thatmatches any character, and any periods in a DNS namesearch must be escaped.Example:

v Incorrect:pam.userdefined.URL_Data.1000035=www.ibm.com

v Correct: pam.userdefined.URL_Data.1000035=www\.ibm\.com














10. Click OK.

Regular expressions in user-defined eventsRegular expressions (strings) are a combination of static text and variables that the Proventia Server forVMware agent uses to detect patterns in the contexts (network packets) you specify for user-definedevent signatures. Use regular expressions when you create user-defined event signatures if you need theagent to detect more than a single static text string.

You can use the these regular expression syntax in a user-defined event signature.

This meta-character... Matches...

(r) r

x x

xr x followed by r

\s either a space or a tab (not a hard break or newline)

\d a decimal digit

\″ a double quote

\’ a single quote

\\ a backslash

\n a newline (ASCII NL or LF)

\r a carriage return (ASCII CR)

\t a horizontal tab (ASCII HT)

\v a vertical tab (ASCII VT)

\f a formfeed (ASCII FF)

\b a backspace (ASCII BS)

\a a bell (ASCII BS)

\ooo a specified octal character code

\xhhh a specified hexadecimal character code

. any character except newline

\@ nothing (represents an accepting position)

″″ nothing

[xy-z] x, or anything between y and z, inclusive (characterclass)

[^xy-z] anything but x, or anything between y and z, inclusive

″text″ literal text, without regard for meta-characters

r? r or nothing

r* zero (0) or more occurrences (kleene closure)

r+ one or more occurrences of r (positive kleene closure)

r{m,n} r at least m times, at most n times (repeat operator)

r|l either r or l (alternation operator)

r/l r only if followed by l (lookahead operator)

^r r only at the beginning of a line (bol anchor)



r$ r only at the end of a line (eol anchor)

r, l any arbitrary regular expression

m, n an integer

x,y,z any printable or escaped ASCII character

text a sequence of printable or escaped ASCII characters

ooo a sequence of up to three (3) octal digits

hhh a sequence of hex digits

User-defined event contextsWhen you create a user-defined event signature, you select a context that notifies the Proventia Server forVMware agent the type and particular part of a network packet to monitor for events. After you specifythe context, you add a string that notifies the agent exactly what to look for when it scans the packet.

You can specify the following contexts when you create user-defined event signatures.

This context... Monitors...

DNS_Query The DNS name in DNS query and DNS reply packetsover UDP and TCP.

Email_Receiver Incoming and outgoing e-mail to a particular recipient(recipient in address header) using the SMTP, POP, andIMAP protocols.

Email_Sender Incoming and outgoing e-mail from a particular recipient(sender in address header) using the SMTP, POP, andIMAP protocols.

Email_Subject The subject line of an e-mail (subject in header) using theSMTP, POP, and IMAP protocols.

File_Name The file (name or type) that you specify.

News_Group The news group address that you specify.

Password The user password that you specify.

SNMP_Community The use of SNMP community strings, which areclear-text passwords in SNMP messages that authenticatethe messages. If the password is not a valid communityname, the password is rejected.

URL_Data Various security or policy issues related to HTTP_GETrequests, which occur when a client, such as a Webbrowser, requests a file from a Web server. It monitorsthe contents of a URL for particular strings.

User_Login_Name Plain-text user names in authentication requests usingthe FTP, POP, IMAP, NNTP, HTTP, Windows®, or R*protocols.

User_Probe_Name Any user name associated with FINGER, SMTP, VRFY,and SMTP EXPN to identify attempts to gain access tocomputers on your network using default programpasswords.


OpenSignaturesOpen signatures use a flexible rules language to define pattern-matching IDS signatures that theProventia Server for VMware agent uses to detect specific threats which are not already covered by thesignatures in the Security Events tab.

Configuring open signaturesThe OpenSignature feature in the Proventia Server for VMware agent uses a flexible rules language toallow you to write customized, pattern-matching IDS signatures to detect specific threats that are notalready monitored in IPS products.

Before you begin


Important: IBM ISS does not guarantee agent performance if you choose to use the OpenSignaturefeature. Use this feature at your own risk. Customer Support is not available to help you write ortroubleshoot custom rules for your environment. If you require assistance to create custom signatures,contact IBM ISS Professional Services.



3. In the OpenSignatures tab, click the Add icon or highlight the rule you want to edit, and then

click the Edit icon.

Tip: You can edit some properties directly by double-clicking the item you want to configure.4. Select the Enabled check box.5. Change the following options as necessary:

Option Description

Description Describes the purpose of this signature

Rule string Specifies the text string that tells the agent when anevent is triggered

The syntax options for each custom rule are as follows:

v <action>: alert

v <protocol>: tcp, udp, icmp, ip

v <IP and netmask>: single IP address (a.b.c.d), range ofIP addresses (a.b.c.d-w.x.y.z), network address usingCIDR notation (a.b.c.0/24)

Note: The Negation operator is indicated with an ’!’Example: alert tcp !192.168.1.0/24

Here the agent prompts you when anything other thanwhat is indicated with the ’!’ is used.Important: If you improperly format a rule string, youmight receive a PAM configuration error response.

6. Click OK.


Changing the order of OpenSignature rulesThe Proventia Server for VMware agent applies open signature rules in a specific order when it processesnetwork traffic. You can change the order of these rules to design the most effective Security Eventspolicy for your network.


VMware.2. Right-click Security Events, and click Open.3. In the OpenSignatures tab, select the OpenSignature rule you want to change the order for.

4. Click the Move Up icon or the Move Down icon to move the rule higher or lower in the list.

Configuring the Firewall policyThe Firewall policy in the Proventia Server for VMware agent defines firewall rules that filter networkpackets based on source and destination ports and IP addresses.

Firewall policyThe Firewall policy in the Proventia Server for VMware agent defines firewall rules that filter networkpackets based on source and destination ports and IP addresses. The agent applies firewall rules in aspecific order when it processes network traffic. The agent can also allow packets from certain IPaddresses or protocols to bypass the protection offered by the Firewall policy and the Security Eventspolicy.

Important: The Network Monitoring setting in the Asset Settings policy impacts how the agentimplements settings in the Firewall policy. You should review the Asset Settings policy when youconfigure the Firewall policy.

Tab Description

Firewall Rules A firewall can reduce, but not eliminate, threatsintroduced to your system by networking hosts. Firewalltechnology can prevent attacks that target networkresources by limiting access to your system. The firewallis the computer’s first line of defense against anetwork-based attack.

Bypass Filters The agent can allow packets from certain IP addresses orprotocols to bypass the protection offered by the Firewallpolicy and the Security Events policy. While bypassingpackets can improve performance, it also allows packetsto continue to the system even if the packet containsmalicious content.

Firewall rule order

The agent reads the list of firewall rules from top to bottom in the order they are listed and appliescorresponding actions. When a connection matches a firewall rule, further processing for the connectionstops, and the agent ignores any additional firewall rules you have set.

Navigation



Bypass filtersThe Proventia Server for VMware agent can allow packets from certain IP addresses or protocols tobypass the protection offered by the Firewall policy and the Security Events policy. While bypassingpackets can improve performance, it also allows packets to continue to the system even if the packetcontains malicious content.

Example: Traffic related to a system backup was inspected when it entered the system so it does notneed to be analyzed again as it is backed up. Configure a bypass filter to avoid processing known dataand slowing the backup process.

Note: Processing bypass filters impacts agent performance. The more bypass filters an agent mustprocess, the slower the agent will perform. Consider configuring no more than 32 bypass filters.

How bypass filters work

When the agent processes a packet, it checks to see if there is a bypass filter set for packets associatedwith this IP address or this protocol. If a bypass filter is configured, the agent does not process anyfirewall rules or security event signatures against the packet.

Note: For any bypass filter to take effect, the Network Monitoring setting in the Asset Settings policymust be enabled.

Bypass filter and event filter interaction

The agent processes bypass filters before it processes event filters, so it is possible to enter a bypass filterthat makes an event filter redundant. While the effect on the system is similar (traffic can circumvent theprotection offered by security event signatures), you may not see the expected behavior if you clear thebypass filter later. That is, when you clear the bypass filter, you would expect the security event rules toprovide protection against malicious traffic, but, because the event filter is still in effect, packets may stillbe circumventing the protection offered by security event rules.

Navigation


Configuring firewall rulesTo filter packets, create firewall rules based on source and destination IP addresses. Firewall rules in theProventia Server for VMware agent ensure that only authorized traffic can enter or leave a virtual asset.

Before you begin

If you want to create firewall rules for all virtual assets in your environment, you can copy the contentsof the Protection scope field in the Asset Settings policy to create a virtual object. Then you can use thatvirtual object as you define firewall rules.

About this task

As a general guideline, you should configure no more than 500 firewall rules.

Important: The Network Monitoring setting in the Asset Settings policy impacts how the agentimplements settings in the Firewall policy. You should review the Asset Settings policy when youconfigure the Firewall policy.



VMware.2. Right-click Firewall, and click Open.

3. In the Firewall Rules tab, click the Add icon.4. Select the Enabled check box.5. Change the following options as necessary:

Option Description

Description Specifies a unique description for the firewall rule

Log details of packets Specifies whether logging is enabled for the firewall rule.If enabled, packet details that match the rule are sent tothe firewall log in the /var/iss/ directory.Tip: You may want to log packets that match firewallrules when troubleshooting unexpected firewall behavior.Notes:

v Logging packets that match firewall rules may impactsystem performance.

v Only the first packet of a connection will be logged forblocked packets when the engine.pamlook.enabledadvanced parameter is enabled in the Agent Settingspolicy.

Rule action Specifies the action the agent takes when packets matcha rule.


v Drop: Drops the packets as they pass through thefirewall.

v Drop and reset: Drops the packets as they passthrough the firewall and sends a reset packet to thesource system.

v Protect: Allows matching packets to be processed bynormal responses, such as logging, the block response,and the quarantine response.

v Monitor: Functions as an IP whitelist. Allows thepackets that match the statements to bypass thequarantine response and bypass the blocking response.However, all other responses still apply to the packet.

Note: “Firewall rule actions” on page 39 describes thefirewall rule actions.

Rule type Select from the following options:

v Constructed: The agent creates the firewall rule usingthe values you specify.

v Manually entered: You create the firewall rule usingthe Rule statement field.

Rule statement Specifies the manually entered rule statementNote: “Firewall rule syntax” on page 39 explains thesyntax to use when creating firewall rules.


Option Description

Protocol Specifies the protocol this rule applies to.

If you set a Protocol value other than Any, the firewallrule is set to that protocol only.Note: If you select the Any option, the following criteriaare applied if the corresponding conditions are met:

v If you set an ICMP code, then an ICMP clause isadded to the rule.

v If you set a source or destination port, then both aUDP and a TCP clause are added to the rule.

v If you set a Protocol number greater than zero (0),then a protocol number clause is added to the rule.

v If you do not specify any protocol settings, then an IPclause is added to the rule. The source and destinationIP addresses will also be added if you have specifiedthem.

ICMP Type Specifies ICMP types for either side of the packetNote: Click Well Known to select often-used types.Tip: This option is only available if you select″Constructed″ in the Rule type field, ″Any″ or ″ICMP″ inthe Protocol field, and clear the All check box.

ICMP Code Specifies ICMP codes for either side of the packetNote: Click Well Known to select often-used codes.Tip: This option is only available if you select″Constructed″ in the Rule type field, ″Any″ or ″ICMP″ inthe Protocol field, and clear the All check box.

Protocol number Specifies the port numbers that the agent shouldassociate with a particular protocolTip: This option is only available if you select″Constructed″ in the Rule type field and ″Any″ or″Number″ in the Protocol field.

6. Configure the Source Address tab and the Destination Address tab using the information in thefollowing table.

Option Description

Any Applies the rule to any IP address

Virtual object Applies the rule to the IP addresses defined in theselected virtual objectTip: You can copy and paste the contents of theProtection scope field in the Asset Settings policy into avirtual object and then use the virtual object as youdefine firewall rules.Note: Appendix A, “Adding or editing a virtual object,”on page 81 describes how to add or edit a virtual objectto achieve the policy configuration you need.

Single IP Address Applies the rule to one IP address

IP address range Applies the rule to a range of IP addresses

Network address/CIDR format Applies the rule to a block of IP addresses specified bythe number of network bitsImportant: This option applies the firewall rule to allsystems on the identified subnet.


7. Configure the Source Port tab and the Destination Port tab using the information in the followingtable.

Option Description

Any Applies the rule to all ports

Single port Applies the rule to a single port

Port range Applies the rule to a range of port numbers

8. Click OK.

Changing the order of firewall rulesThe Proventia Server for VMware agent applies firewall rules in a specific order when it processesnetwork traffic. Order firewall rules to design the most effective firewall policy.



3. Select the firewall rule and click the Move Up icon or the Move Down icon to move the rulehigher or lower.

Disabling firewall rulesIf you want the Proventia Server for VMware agent to temporarily stop processing a firewall rule, youcan disable the rule.


VMware.2. Right-click Firewall, and click Open.3. Select the firewall rule you want to disable.4. Select the Enabled check box.5. Click OK.

Deleting firewall rulesIf you no longer need a firewall rule previously configured for your Proventia Server for VMware agent,you can delete the rule.

About this task

For example, when a system that you designed specific rules for no longer resides on your network,delete the rule to keep your firewall policy manageable.


VMware.2. Right-click Firewall, and click Open.3. Select the firewall rule you want to delete.

4. Click the Remove icon.


Configuring bypass filtersThe Proventia Server for VMware agent can allow packets from certain IP addresses or protocols tobypass the protection offered by the Firewall policy and the Security Events policy. Create bypass filterswhen you do not need to inspect trusted data packets.

Before you begin

For any bypass filter to take effect, the Network Monitoring setting in the Asset Settings policy must beenabled.

About this task

While bypassing packets can improve performance, it also allows packets to continue to the system evenif the packet contains malicious content.



3. In the Bypass Filters tab, click the Add icon.4. Select the Enabled check box.5. Complete the following options:

Option Description

Description Specifies the purpose of the filter

Rule protocol Specifies the protocol this rule applies to.


v Any: Applies the filter to any protocol

v TCP: Applies the filter to packets using theTransmission Control Protocol

v UDP: Applies the filter to packets using the UserDatagram Protocol

v ICMP: Applies the filter to packets using the InternetControl Message Protocol

v Number: Applies the filter to packets using thespecified protocol

Protocol number Specifies the protocol to use with an Any or Numberfilter


Option Description

Source or Destination Address Specifies the source or destination IP address.


v Any: Applies the filter to any IP address

v Virtual object: Applies the filter to the IP addressesassociated with the selected virtual objectNote: Appendix A, “Adding or editing a virtualobject,” on page 81 describes how to add or edit avirtual object to achieve the policy configuration youneed.

v Single IP address: Applies the filter to one IP address

v IP address range: Applies the filter to a range of IPaddresses

v Network address/CIDR format: Applies the filter to ablock of IP addresses

6. Click OK. The filter is added to the Bypass filters table.

Firewall rule actionsThis topic describes the firewall rule actions in the Proventia Server for VMware agent.

Option Description

Drop Drops the packets as they pass through the firewall.Because the firewall is inline, this action prevents thepackets from reaching the target system. To the sourcesystem, it appears as if the target system has notresponded. Several retry attempts are likely and then theconnection will eventually time out.

Drop and reset Drops the packets as they pass through the firewall andsends a reset packet to the source system. The connectionterminates more quickly (because it is automaticallyreset) than with the drop response.

Protect Enables matching packets to be processed by normalresponses, such as logging, the block response, and thequarantine response.

Monitor Functions as an IP whitelist. Allows the packets thatmatch the statements to bypass the quarantine responseand bypass the blocking response. However, all otherresponses still apply to the packet.

Firewall rule syntaxThis topic explains the syntax to use when creating firewall rules for your Proventia Server for VMwareagent.

Rule syntax examples

The following statements are examples of complete firewall rules. If you do not specify a protocol, therule assumes and uses the “Any” protocol.

Note: x is a number in the IP address.v ip src addr xxx.xxx.x.x

v ip src addr xxx.xxx.x dst addr any tcp src port 20 dst port 80


v ip src addr any dst addr xxx.xxx.xx.x

v ip src addr any dst addr any icmp type 8 tcp

v icmp

v udp

Firewall rule syntax

A firewall rule consists of several statements (or clauses) that define the traffic for which the rule applies.When you manually create firewall rules for the agent to use, you can use the following syntax.

Syntax rule Description Examples

IP clause Indicates the version of IP protocoland the conditions in the header thatmust be satisfied for the statement tomatch the rule

ipIP-source-address-conditionIP-destination-address-condition

IP datagram clause Indicates the protocol and theprotocol-specific conditions that mustbe satisfied for the statement tomatchNote: The supported protocols areICMP, TCP, and UDP. You can alsospecify a set of IP protocol numbers.

v icmpICMP-type-conditionICMP-code-condition

v tcpTCP-source-port-number-condition TCP-destination-port-number-condition

v udpUDP-source-port-number-condition UDP-destination-port-number-condition

v protoprotocol-number-expression

Source and target address conditions Indicates the set of allowable IPaddresses for the source orestablishment of a TCP-basedconnection, UDP packet, or ICMPpacket

v srcaddrIP-source-address-expression

v dstaddrIP-destination-address-expression

TCP/UDP source and target portconditions

Indicates the set of TCP or UDP portsfor the source or target of theestablishment of a (TCP) connectionor a (UDP) packet

v src portport-number-expression

v dst portport-number-expression

ICMP type and code conditions Indicates the set of ICMP types orcodes for either side of the packet

v typeICMP-type-expression

v codeICMP-code-expression

Using ranges Indicates a range of values for IPaddresses, port numbers, ICMPmessage types and codes, andprotocol numbers using a dash (-)between the first and last values inthe range

v ip src addrxxx.xxx.x.x -xxx.xxx.x.xxNote: x is a number in the IPaddress.

v tcp dst port 20 - 80

Using ″any″ Specifies ″any″ in all expressions v ipdstaddr any

v icmp type any

Configuring the Discovery policyThe Proventia Server for VMware agent uses a discovery process to locate active network interfaces onvirtual assets and identify the type of device associated with each active network interface through OSidentification.


Discovery scanningDiscovery scanning in the Proventia Server for VMware agent uses a combination of fingerprintingmethodologies to gather information about the virtual assets on your network.

How discovery scanning works

When a new virtual machine comes online, a VM event is created and a preliminary scan of the virtualmachine gathers information such as the IP address. After the initial scan is complete, the agent reportsany information it can find to SiteProtector. To identify the operating system details of each virtualmachine including information about ports and services that are running on the virtual machine, youshould initiate a scheduled scan.

Note: Policy settings specified in the Exceptions tab are enforced before the policy settings in the GlobalSettings tab.

Scheduling detailed scans

Schedule regular scans to keep information about the virtual assets up to date. Any time a machinemoves or a change has occurred to a virtual machine on your network, the virtual asset inventory inSiteProtector becomes outdated. By scheduling your scans on a regular basis, you can ensure that thevirtual machine details reported to SiteProtector are current.

Note: The minimum frequency you can scan a virtual machine is one hour.

Note: The scan interval is calculated by the last scheduled scan time plus the duration of the scan. Forexample, if you have scheduled a scan to run every hour and the scan takes 15 minutes, the next scanwill begin one hour and 15 minutes later. If any scanning occurs within a given hour, no additionalscanning will occur until the next hour block, even if you change the scan start time within the policy.

Viewing scan detail report

The information acquired by the discovery scan is reported to SiteProtector. For compliance auditing, youcan view details about your virtual assets in the Asset view.

Navigation


Configuring global discovery settingsThe global settings configuration in the Proventia Server for VMware agent defines the discovery scanoptions for all virtual assets. If the global settings are not appropriate for a specific collection of virtualassets, you can create an exception to define a custom configuration.

About this task

When you perform a scheduled scan, the operating system details of the virtual assets are updated in theSiteProtector Asset View. The scheduled scan also gathers information about the services running on thevirtual asset that you can view in the SiteProtector Analysis View.


VMware.2. Right-click Discovery, and click Open.3. Select the Enable check box.


4. In the Global Settings tab, select the Scheduled scan check box to schedule the time and frequency offuture scans.

5. Select the Scan TCP ports check box to specify additional ports to scan.6. Select the Scan UDP ports check box to specify additional ports to scan.

Configuring exceptions to global discovery settingsAn exception defines a custom discovery scan configuration for a specific collection of virtual assets inthe Proventia Server for VMware agent. Create an exception if the global settings are not appropriate fora specific collection of virtual assets.


VMware.2. Right-click Discovery, and click Open.

3. In the Exceptions tab, click the Add icon.4. Select the Enable check box.5. In the Virtual object field, select the collection of virtual assets that this configuration applies to.



6. Select the Scheduled scan check box.7. In the Discovery Scan Ports area, select the ports and specify the ranges.8. Click OK.

Configuring the VM Events policyVM (virtual machine) events in the Proventia Server for VMware agent alert you to certain events thatare specific to your virtual environment.

VM eventsVM events alert you to certain events that are specific to your virtual environment. You can configure theProventia Server for VMware agent to forward these events to the management console.

How the VM Events policy works

A VM event is generated when certain user actions or system actions occur in the virtual environment.Based on the settings in the policy, the agent filters out the events you are not interested in and forwardsthe information for the other events to the management console.

Ensuring the agent can authenticate

To have access to the VM events that are generated in your virtual environment, the agent must be ableto authenticate with the ESX server. To authenticate, the agent must have the IP address of the host ESXserver and the user name and password for the administrator account on the server. This informationwas provided when the agent was installed; however, if the account credentials have changed, you mustupdate the information.


If an agent cannot authenticate with its host ESX server, a system event is generated and displayed in theAgent Properties view in SiteProtector; this informs you that the authentication credentials need to beupdated.

Event categories

VM events provide information about user actions and system actions in the virtual environment. Forexample, events are generated when a host connection is broken or when a virtual machine is turned onor off. Each event contains a description of the event and information such as the user who generated theevent and the time the event occurred. The different events are categorized so that you can easily locatethe events you want to forward to the management console.

VM Events forwarded by default

The following table lists the VM events that are forwarded to the management console by default. Theseevents are selected by default because they are significant events that you might want to access from themanagement console. If you do not want event information for these event forwarded to the managementconsole, clear the check box for the event.

Event name Description

NoAccessUserEvent Records a failed user logon due to insufficient accesspermission

UserLoginSessionEvent Records a user logon

UserLogoutSessionEvent Records a user logoff, disconnection, or session timeout

HostShutdownEvent Records the shutdown of a host

VmCreatedEvent Records that a virtual machine was successfully created

VmPoweredOffEvent Records that a virtual machine has finished powering off

VmPoweredOnEvent Records that a virtual machine has finished powering on

VmReconfiguredEvent Records a reconfiguration of the virtual machine

VmRemovedEvent Records that a virtual machine was removed fromVirtualCenter management

VmStartingEvent Records that a virtual machine has powered on

VmSuspendedEvent Records that a virtual machine has finished suspending

VM Event severities

Each VM event has an assigned severity based on the impact the event has to your virtual environment.You can use the event severity to determine which events the agent forwards to the management console.

Severity Description

High An event that would cause the SiteProtectoradministrator to change the security policies or preparepolicy changes for virtual assets that are coming onlineshortly. Also includes changes to roles or permissions forVMware authenticated users.

Medium An event that will increase your awareness that theassets (VMs) that are being protected might change theirprofiles because of changes to the template


Severity Description

Low An event that indicates failures for entities andconstructs that can only be manipulated with VMware(for example, Resource Pools, Clusters, Templates, loginsessions)Note: While failures are important, keep in mind theSiteProtector Administrator cannot control theseconstructs from within SiteProtector.

Navigation


Configuring system eventsSystem events are events related to your virtual environment that are generated by VMware vSphere orthe ESX server. The Proventia Server for VMware agent can notify you when these events occur bysending an alert to the management console.

About this task

The agent must be able to authenticate with the ESX server before it can enforce the settings in the VMEvents policy.


VMware.2. Right-click VM Events, and click Open.3. Ensure that the Enable check box is selected.4. Click the System tab.5. Click the plus sign (+) to expand the event category that contains the event you want reported to the

management console.6. In the Selected column, select the check box for the appropriate event.

Results

When a selected event occurs, the agent sends an alert to the management console to notify you of theactivity in your virtual environment.

Configuring asset-specific eventsAsset-specific events are events generated by the virtual assets in your virtual environment. TheProventia Server for VMware agent can notify you of when these events occur by sending an alert to themanagement console.

About this task

The agent must be able to authenticate with the ESX server before it can enforce the settings in the VMEvents policy.


VMware.2. Right-click VM Events, and click Open.


3. Ensure that the Enable check box is selected.

4. In the Asset-Specific tab, click the Add icon.5. Select the Enable check box.6. In the Virtual object field, select the collection of virtual assets that this configuration applies to.



7. Click the plus sign (+) to expand the event category that contains the event you want reported to themanagement console.

8. In the Selected column, select the check box for the appropriate event.9. Click OK.

Results

When a selected event occurs, the agent sends an alert to the management console to notify you of theactivity of interest in your virtual environment.

Updating authentication credentialsTo have access to the VM events generated in your virtual environment, the Proventia Server for VMwareagent must be able to authenticate with its host ESX server. If the account credentials change, you mustupdate the account information so that the agent can continue to access the event data.

Before you begin

You need the following information to complete this task:

Information Description

SVM IP address IP address of the security virtual machine where theagent is installed

Proventia Manager password Password for the Proventia Local Management Interface,which was specified during the installation of the agent

ESX server IP address IP address of the ESX server that is hosting the securityvirtual machine where the agent is installed

ESX server user name Current (new) user name for the administrator accountof the ESX server

ESX server password Current (new) password for the administrator account ofthe ESX server

Procedure1. Open Internet Explorer.2. In the Address box, type https://svm_ip_address.3. Log in using the user name admin and the Proventia Manager password.4. Select No when asked if you want to use the Getting Started procedures.5. Click Launch Proventia Manager.6. Select System → VMware.7. Type the IP address of the host ESX server.8. Type the current user name and the current password of the host ESX server.


Configuring the Anti-rootkit policyThe anti-rootkit component in the Proventia Server for VMware agent detects malicious rootkits in thekernel data structure and reports the unauthorized activity.

Rootkit detectionThe Proventia Server for VMware agent monitors virtual machines for malware exhibiting rootkitbehavior by detecting unauthorized changes to the System Service Descriptor Table (SSDT) and theInterrupt Descriptor Table (IDT).

Note: Each virtual machine that requires rootkit detection must be paused and resumed.

When the Anti-rootkit policy is enabled, the agent monitors for known malicious rootkits by default.TheVM Settings tab defines whether the pause and resume operation should be managed automatically ormanually. You can customize detection of specific rootkits by creating a list of inclusions. For assistancewith adding rootkits to the detection list, contact IBM ISS Customer Support.

Important: If you add any hardware devices to a virtual machine with rootkit detection enabled, rootkitdetection will stop.

Note: Policy settings specified in the Exceptions tab are enforced before the policy settings in the GlobalSettings tab.

Windows platform support for anti-rootkit component

The anti-rootkit component is currently supported by the following Windows systems.

Operating system Service Pack

Windows XP 32-bit SP0, SP1, SP2, SP3

Windows XP 64-bit SP1, SP2

Windows 2003 32-bit SP0, SP1, SP2

Windows 2003 64-bit SP1, SP2

Linux® platform support for anti-rootkit component

The anti-rootkit component is currently supported by the following Linux systems.

Operating system Update/Service Pack

RHEL 5 32-bit 5.3, 5.4

SLES 10 32-bit SP1, SP2

SLES 11 32-bit SP0

How rootkit detection works

The anti-rootkit module locates the System Service Descriptor Table (SSDT) and Interrupt DescriptorTable (IDT) to detect the kernel rootkit on the virtual machine. When the virtual machine is running andthe tables are stabilized, the anti-rootkit module locks down the tables and begins monitoring for knownrootkits. You can customize the default blacklist by editing existing entries or adding new rootkits thatyou want the agent to detect.


Viewing events

The anti-rootkit component generates events when there are changes made to the SSDT or the IDT entriesby unauthorized applications. You can view events in the /var/log/iss-IM/IM_pid.log and in theAnalysis view in SiteProtector.

Navigation


Configuring global anti-rootkit settingsThe anti-rootkit component in the Proventia Server for VMware agent scans all processes used andinvoked by the operating system. You can add any rootkit that is found to be harmful to the inclusionlist.

About this task

The anti-rootkit component contains a blacklist of known malicious rootkits. To enhance the detection list,you can include additional rootkits to the blacklist that the anti-rootkit module is actively looking for. Youmust have valid entries in every required field. Partial definitions are not valid and will not provide therequired protection.


VMware.2. Right-click Anti-rootkit, and click Open.3. Select the Enable check box to enable the anti-rootkit policy.4. Click the Global Settings tab.5. In the Anti-rootkit Detection area, determine if you want to detect rootkits for all virtual assets.

Tip: If you only have a few virtual machines that you want to monitor, you can select Off for GlobalSettings and then create an exception for the virtual machines that you want to monitor.

6. Click the Add icon.7. Select the Enabled check box and complete the following fields:

Option Description

Description Specify a unique description for the inclusion

Operating system Select from the following options:

v Windows XP SP0, SP1, SP2, SP3

v Windows 2003 SP0, SP1, SP2

v Linux SLES 10 32-bit


v Linux RHEL 5 32-bit

Note: Not all operating system and operating systemarchitecture combinations are currently supported. For acomplete list of supported platforms, see “Rootkitdetection” on page 46.

Operating system architecture Select from the following options:

v 32-bit

v 64-bit


Option Description

Table entry type Select from the following options:

v IDT (Interrupt Descriptor Table)

v SSDT (System Service Descriptor Table)

Table entry number Specify a number from 0-512 that indicates the row inthe selected SSDT or IDT table

SHA256 hash Specify a valid SHA256 hash expressed as hexadecimaldigits (64 hexadecimal digits)

Module name Specify the name of the module. The module name alsocan be represented with fullpath.Example: \Windows\system32\drivers\InvisibleDrvNT.sysNote: The module name for Linux is ″Unknown″.

Module owner Specify the owner of the moduleExample: Microsoft® Corporation

8. Click OK.

Configuring exceptions to global anti-rootkit settingsCreating exceptions to global settings allows you to customize protection for a specific collection ofvirtual assets in the Proventia Server for VMware agent.


VMware.2. Right-click Anti-Rootkit, and click Open.3. Select the Enable check box to enable global anti-rootkit settings.

4. In the Exceptions tab, click the Add icon.5. Select the Enable check box.6. In the Virtual object field, select the collection of virtual assets that this configuration applies to.



7. In the Anti-rootkit Detection area, determine if you want to detect rootkits for only selected virtualmachines.

Tip: If you only have a few virtual machines that you want to monitor, you can select Off for GlobalSettings and then create an exception for the virtual machines that you want to monitor.

8. Click the Add icon.9. Complete the following fields:

Option Description

Description Specify a unique description for the inclusion


Option Description

Operating system Select from the following options:

v Windows XP SP0, SP1, SP2, SP3

v Windows 2003 SP0, SP1, SP2



v Linux RHEL 5 32-bit

Note: Not all operating system and operating systemarchitecture combinations are currently supported. For acomplete list of supported platforms, see “Rootkitdetection” on page 46.

Operating system architecture Select from the following options:

v 32-bit

v 64-bit

Table entry type Select from the following options:

v IDT (Interrupt Descriptor Table)

v SSDT (System Service Descriptor Table)

Table entry number Specify a number from 0-512 that indicates the row inthe selected SSDT or IDT table

SHA256 hash Specify a valid SHA256 hash expressed as hexadecimaldigits (64 hexadecimal digits)

Module name Specify the name of the module. The module name alsocan be represented with fullpath.Example: \Windows\system32\drivers\InvisibleDrvNT.sysNote: The module name for Linux is ″Unknown″.

Module owner Specify the owner of the moduleExample: Microsoft Corporation

10. Click OK.

Excluding virtual machines from rootkit detectionYou can exclude virtual machines from rootkit detection by disabling the anti-rootkit component in theProventia Server for VMware agent, disabling rootkit detection for all virtual objects, or disabling rootkitdetection for a specific virtual object.

Disabling the anti-rootkit componentProcedure1. In the Navigation pane, click the Site Group and open the Policy view for Proventia Server for

VMware.2. Right-click Anti-rootkit, and click Open.3. Clear the Enable check box to disable all anti-rootkit monitoring and protection.

Disabling the anti-rootkit component for all non-global virtual objectsProcedure1. In the Navigation pane, click the Site Group and open the Policy view for Proventia Server for

VMware.2. Right-click Anti-rootkit, and click Open.3. Select the Enable check box to enable global anti-rootkit settings.4. On the Exceptions tab, clear all of the Enable check boxes.


Disabling the anti-rootkit component for a specific virtual object:Procedure

1. In the Navigation pane, click the Site Group and open the Policy view for Proventia Server forVMware.

2. Right-click Anti-rootkit, and click Open.3. Select the Enable check box to enable global anti-rootkit settings.4. On the Exceptions tab, clear the Enable check box for the appropriate virtual object.

Configuring the Network Access Control policyNetwork Access Control (NAC) in the Proventia Server for VMware agent controls which virtual assetshave access to the network. Use NAC to quarantine unknown or untrusted virtual assets until you areready to trust them.

Network Access ControlThe Proventia Server for VMware agent can quarantine unknown or untrusted virtual assets until you areready to trust them.

Note: If you enable Network Access Control, then you must create at least one entry in the trusted assetslist or the access control list.

Tab Description

Trusted Assets Lists the virtual assets that you trust and that you knoware compliant. Trusted assets will never be quarantined.Note: You do not have to add your security virtualmachines to the trusted assets list as these areautomatically trusted.

Access Control for Quarantined Assets Lists the resources that noncompliant assets can access sothat they can be trusted

How NAC works

Network Access Control (NAC) works by quarantining new virtual assets until they comply with yournetwork security policy. For example, when a new virtual machine is created and the IP address of thevirtual machine is not in the trusted assets list, the virtual machine is quarantined. While the virtualmachine is in quarantine, it can only access those resources defined in the access control list. Thequarantined asset can connect to specified resources to download and apply any patches or updatesnecessary to comply with your security policy. After the quarantined asset is compliant, you can removethe asset from quarantine by adding the IP address of the virtual machine to the trusted list.

Ensuring network access

To give network access to a new virtual machine you must add the IP address of the virtual machine tothe trusted asset list. Trusted assets are those assets that you never want to be quarantined. Becausetrusted assets can access any network resource, ensure that any virtual machine you add to the listcomplies with your security policy. If you are not sure that a virtual machine can be trusted, wait untilafter it has been quarantined before you add it to the trusted assets list.

Removing assets from quarantine

Any virtual asset that comes online and is not on the trusted asset list is quarantined and has limitedaccess to the network. To remove an asset from quarantine, you must add it to the trusted asset list.


Note: Trusted assets can access any network resource. Ensure that any virtual machine you add to thetrusted asset list complies with your security policy.

Navigation


Creating a trusted asset listThe Proventia Server for VMware agent uses the trusted asset list to control which virtual assets canaccess the network.

Before you begin

You must define the protection scope in the Asset settings policy before the settings in the NAC policycan take effect.


VMware.2. Right-click Network Access Control, and click Open.3. Select the Enable check box.

4. In the Trusted Assets tab, click the Add icon.5. Select the virtual assets to include from the following options:

Option Description

Virtual object Specifies the virtual assets that this configuration appliestoNote: The topic Appendix A, “Adding or editing avirtual object,” on page 81 describes how to add or edit avirtual object to achieve the policy configuration youneed.

Single IP address Specifies a single IP address


Network address/CIDR format Specifies a block of IP addresses

6. Click OK.

Creating an access control list for quarantined assetsThe Proventia Server for VMware agent uses the access control list to determine which resourcesquarantined assets are allowed to access.

Before you begin

You must define the protection scope in the Asset settings policy before the settings in the NAC policycan take effect.


VMware.2. Right-click Network Access Control, and click Open.3. Select the Enable check box.


4. In the Access Control for Quarantined Assets tab, click the Add icon.5. Ensure the Allow check box is selected.6. Select the resource to include from the following options:

Option Description

Virtual object Specifies the virtual assets that this configuration appliestoNote: The topic Appendix A, “Adding or editing avirtual object,” on page 81 describes how to add or edit avirtual object to achieve the policy configuration youneed.

Domain name Specifies the domain name

Single IP address Specifies a single IP address


Network address/CIDR format Specifies a block of IP addresses

7. Click OK.

Configuring the Agent Settings policyThe Agent Settings policy for the Proventia Server for VMware agent allows you to customize certainagent-level behaviors.

Agent settingsThe Agent Settings policy for the Proventia Server for VMware agent allows you to configure agent alerttypes and customize agent behavior. You can deploy this policy to a single agent or to a group of agents.

Agent alert types

The agent generates alerts to notify you of the following types of activity.

Alert Type Description

Error These alerts indicate that the agent has stoppedfunctioning.

Warning These alerts indicate that a minor problem has occurredwith the agent.

Informational These alerts provide useful information about typicalagent operations.

Important: If you change any of the settings on the Agent Alerts tab, you must restart the agent for thechanges to take effect.

Advanced parameters

Advanced parameters customize the behavior and the performance of your agent to better meet yoursecurity needs.

Refer to the PAM Help (Help → Attack Signatures → Protocol Analysis Module) for information aboutconfiguring PAM parameters.

Refer to “Agent-specific advanced parameters” on page 54 for information about agent-specificparameters.


Important: If you enter conflicting or duplicate advanced parameters, the parameter entered lastoverrides the parameter entered first.

Navigation

Locate the policy you want to edit and then click Agent Settings.

Configuring agent alertsThe Proventia Server for VMware agent generates error, warning, and informational alerts to notify youof the status of the agent.

About this task

Each alert type indicates a different risk to your environment. Set an appropriate priority for each alerttype to help you prioritize your response time to these alerts.

Procedure1. In the Navigation pane, expand the group the agent is assigned to, right-click Agent Settings, and

click Open.2. Click the Agent Alerts tab.3. For each area, change the following options as necessary:

Option Description

Enable Indicates that error, warning, or informational alerts willbe displayed in the consoleNote: All alerts are also logged to the SiteProtectordatabase.

Priority Assigns a priority to error, warning, or informationalalertsTip: Choose a higher priority for more critical alerts

Results

When the agent generates an alert, the alert is displayed in Agent Properties → Health Summary → AgentMessages.

Configuring advanced parametersIn the Proventia Server for VMware agent, you can use advanced parameters to customize the behaviorand the performance of your agent to better meet your security needs.

Before you begin



Refer to “Agent-specific advanced parameters” on page 54 for information about agent-specificparameters.


About this task

Advanced parameters fine-tune the performance of your agent; however, assigning inappropriate settingsto an advanced parameter might have significant negative effects on the behavior of the agent. If youenter conflicting or duplicate parameters, the parameter you entered last overrides the parameter youentered first.


click Open.

2. In the Advanced Parameters tab, click the Add icon to add a new parameter or select an existing

parameter and click the Edit icon to edit the parameter.3. Complete the following options:

Option Description

Name Specifies the name of the parameter

Description Describes the purpose of the parameter

Value Specifies the value of the parameter

v Boolean

v Number

v String

4. Click OK.

Agent-specific advanced parametersThis topic describes the agent-specific advanced parameters you can use to customize the ProventiaServer for VMware agent and the default settings the agent was installed with.

Attention:


Parameter name Default setting Description

sensor.trace.level 3 ProventiaServerV log level

vmevents.trace.level 3 ProventiaServerV log level for vmomodule.

ark.trace.level 3 ProventiaServerV log level for arkmodule.

discovery.trace.level 3 ProventiaServerV log level fordiscovery module.

vmevents.log.size 20000000 ProventiaServerV log size (20MB) forvmo module.

ark.log.size 20000000 ProventiaServerV log size (20MB) forark module.

discovery.log.size 20000000 ProventiaServerV log size (20MB) fordiscovery module.



engine.heartbeatinterval 5 The length of time in secondsbetween heartbeats from theVMware-based network engine to theVMware-based network driver. Therange is 1 to 500 seconds.

engine.numberofheartbeat 2 When this number of consecutiveheartbeats is missed, theVMware-based network driver willassume the VMware-based networkengine is dead or nonresponsive.Default value: 2

engine.packetqlength 150000 Size of the queue in theVMware-based network engine,defined in number of packets

engine.pamlook.enabled True Method for handling congestion bycross-referencing connection tablesand forwarding packets consideredsafe.

engine.pamlook.connections.tcp.size 5000 The number of TCP connections tomaintain in connection tables foreach vNIC

engine.pamlook.connections.udp.size 1000 The number of UDP connections tomaintain in connection tables foreach vNIC

engine.mia.enabled True Enables or disables a setting that willeliminate duplicate inspection for thesame packet as it moves through thenetwork, hereby referred to asMultiple Inspection Avoidance (MIA).

engine.failopen.threshold.high 90 Percentage of queue that is filledbefore a FAIL OPEN condition isreported

engine.failopen.threshold.low 10 Percentage level that the queue mustfall back to in order to recover from aFAIL OPEN condition.

engine.droplog.enabled False Determines whether logging ofdropped packets is enabled.

engine.adapter.low-water.default 1 The minimum number of packets pertraffic sampling interval which areexpected to flow on each adapter.

engine.adapter.high-water.default 5 The number of packets per trafficsampling interval which are expectedto flow on each adapter. Thehigh-water mark is used to preventmultiple low traffic warnings frombeing issued when the traffic ishovering around low-water mark.

engine.nack.enabled False Enables acceleration of data-less TCPACK packets.



engine.pamlook.threshold.busy1 5 Packets outstanding threshold #1 forPCC to cache more; handlescongestion by cross-referencingconnection tables and forwardingpackets that are considered safe.



engine.remote.syslog.enabled False Allows system log entries to beduplicated on a remote server.

engine.remote.syslog.address Specifies the IP address of the remotesyslog server.

pam.traffic.sample True Enables traffic sampling for thepurpose of detecting abnormal levelsof network activity.

pam.traffic.sample.interval 300 The interval, expressed in seconds, atwhich traffic flow should be sampledfor the purpose of detectingabnormal levels of network activity.

fastpath.trace.level 3 Log level for fastpath engine

np.statistics on Determines whether logging of PAMstatistics is enabled.

np.statistics.file.pam /var/iss/pamstats.dat The PAM statistics file name.

np.log.quarantine.added on Log the details of rules that areadded to the quarantine table.

np.log.quarantine.expired on Log the details of rules that haveexpired from quarantine table.

np.firewall.log on Determines whether to log the detailsof packets that match firewall rulesthat are enabled.

np.firewall.log.prefix /var/iss/fw Prefix of firewall log file name.

np.firewall.log.suffix .log Suffix of firewall log file name.

np.firewall.log.size 1400000 Maximum size of a firewall log file inbytes.

np.firewall.log.count 10 Number of firewall log files.

np.log.prefix /var/iss/event Prefix of event log file name.

np.log.suffix .log Suffix of event log file name.

np.log.size 1400000 Maximum size of event log file inbytes.

np.log.count 10 Number of event log files.



np.drop.invalid.checksum True Determines whether to block packetswith checksum errors in inlineprotection mode.

np.drop.invalid.protocol True Determines whether to block packetsthat violate protocol in inlineprotection mode.

np.drop.rogue.tcp.packets False Determines whether to block packetsthat are not part of a known TCPconnection in inline protection mode.

np.drop.resource.error False Determines whether to block packetsif there are insufficient resources toinspect them in inline protectionmode.


Chapter 3. Configuring filters

There are various filters available to the Proventia Server for VMware agent. These filters control howdata is displayed, how data is processed by the agent, and what data is sent to the SiteProtector Console.

Controlling table display informationIn some Proventia Server for VMware policies you can control the display of the data in the table byusing the table toolbar.

The table toolbar only displays the optionsavailable for the table you are currently viewing.

Option Description

Opens the Edit window for the selected row

Opens the Add window

Removes the selected row

Opens the Group By Columns window. Add and remove columns to control howinformation in the table is grouped. You can group the table by values of a particularcolumn.

Removes all data groupings for this table

Opens the Add or Remove Columns window.

Select and clear check boxes to control which columns are displayed on this table.

Restores data ordering and removes data groupings

Moves selected row up

Moves selected row down

Opens the Configure Filters window in which you can select criteria to control whichinformation is displayed on this table

1. In the Regular Expression area, type the regular expression by which you want to filter.Note: To use this feature, you must be familiar with how regular expressions work.This search feature is not case-sensitive.

2. For each category, select the filters you want to apply. The default is Any, which resultsin the agent searching for any result that matches the regular expression you entered.

Copies a selected row from the table

Pastes a copied row into the table

Event filtersEvent filters contain a sequence of matching criteria such as IP address, TCP/UDP port, or ICMPtype/code values that are used to exclude the reporting of certain events. You can define event filters inthe Proventia Server for VMware agent to reduce the number of alerts displayed in the Console withoutcompromising the security of your system.


How event filters work

After traffic passes through the firewall, the agent processes it against the rules configured in the SecurityEvents policy. If the traffic triggers a security event rule, the agent would typically take protective actionand send an alert to the Console to notify you of the potential threat to your system. If, however, youhave configured an event filter, the agent takes the appropriate protective action but sends no alert to theConsole.

Note: Refer to the PAM Help (Help → Attack Signatures → Protocol Analysis Module) for informationabout configuring PAM parameters for filtering events.

Navigation

Locate the policy you want to edit and then click Agent Settings.

Configuring event filtersIn the Proventia Server for VMware agents, you can configure event filters to reduce the number of alertsdisplayed in the Console without compromising the security of your system.

Before you begin


Refer to the PAM Help (Help → Attack Signatures → Protocol Analysis Module) for more informationabout configuring PAM parameters.

About this task

Advanced parameters fine-tune the performance of your agent; however, assigning inappropriate settingsto an advanced parameter might have significant negative effects on the behavior of the agent. If youenter conflicting or duplicate parameters, the parameter you entered last overrides the parameter youentered first.


click Open.

2. In the Advanced Tuning Parameters tab, click the Add icon.3. Select the Enabled check box.4. Type the parameter name, pam.report.filterall.5. Select String.6. In the Value box, type the IP address.7. Click OK.

8. Click the Add icon.9. Select the Enabled check box.

10. Type the parameter name, pam.report.filter.<alogorithm id>.11. Select String.12. In the Value box, type the IP address.13. Click OK.


Chapter 4. Configuring resource management

Resource managementYou can specify the hardware resources used by the Proventia Server for VMware agent for each virtualmachine.

You may need to modify the CPU utilization and the memory allocation that is used by the ProventiaServer for VMware agent by setting an upper limit. However, for optimal performance, consider thefollowing items:v Do not use less memory than the recommended default (1G)v Do not use more than one CPU

Configuring resource management settingsYou can configure hardware resources such as memory and CPU usage for the Proventia Server forVMware agent in the VMware vSphere Client.

Before you begin

Ensure you have saved any settings in the Proventia Server for VMware agent as you will need to turnoff the Proventia Server for VMware agent to configure hardware resources.

Procedure1. Start the vSphere Client and log on to ESX.2. Power off the Proventia Server for VMWare agent.3. Right-click the Proventia Server for VMWare agent and select Edit Settings.4. Select the Hardware tab.5. Change the memory and CPU hardware resources.

Note: Ensure that the CPUs and the Memory Configuration meet the recommended amount.6. Click OK. Verify any changes you made by powering on the secure virtual machine.

Note: Hardware resources cannot be changed when the Proventia Server for VMware is in the poweron state.


Chapter 5. Configuring responses

You can configure responses at the Proventia Server for VMware agent policy level or at the Site Grouplevel. All Site Group responses use response objects so you can centrally configure these responses; youconfigure other responses at the signature level.

ResponsesResponses are the actions taken when an event is detected by the Proventia Server for VMware agent.

You can configure responses at the agent level or at the Site Group level. Responses at the Site Grouplevel use response objects, where you can centrally configure responses. You configure agent levelresponses at the signature level.

Response Configuration

Block At the signature levelNote: The Block response is not available for allsignatures.Tip: To enable the Block response in the policy that youwant to customize, select the Block check box in thesignature or rule.

Display At the signature levelTip: To enable the Display response in the policy thatyou want to customize, select the Display in consolecheck box in the signature or rule.

Log evidence At the signature levelNote: You can configure the maximum number of files,maximum file size, log file prefix, and log file suffix inthe SiteProtector Central Responses policy.

Quarantine At the signature level after you have configured theQuarantine response object in the SiteProtector CentralResponses policy

Email, SNMP, and User-Specified At the Site Group level in the SiteProtector CentralResponses policyTip: See the Central Responses Help (open the CentralResponses policy and press F1) for information abouthow to configure central responses.

Configuring response objectsYou can configure responses for your Proventia Server for VMware agent at the Site Group level in theSiteProtector Central Responses policy.

About this task

When you use response objects, you only need to edit the response object when you want to reconfigurea response. You do not need to change each instance of the response.

Example: You always send an e-mail to John Smith when an agent detects a High severity event. John’se-mail address changes, and you must update your responses to use the new address. You can simplyedit the address in the response object to avoid changing the address in every response.


Procedure1. Select Tools → Central Responses → Response Objects and press the Help button.2. Select the Response Objects Help topic.3. Select the appropriate topic to learn more.


Chapter 6. Administering

This section contains information about additional features that can help you as you manage yourProventia Server for VMware agent to ensure the security of your virtual environment.

Working with log filesThe Proventia Server for VMware agent generates agent log files and uses system log files to protect yourvirtual environment.

Logging packets from intrusion attemptsYou can configure the Proventia Server for VMware agent to log the summary of an event. Evidencelogging copies a packet that triggers an event to a log file so you can determine exactly what an intruderdid or attempted to do.

About this task

The Proventia Server for VMware agent logs packets that trigger events to the /var/iss/ directory.


VMware.2. Expand the Default Repository node, expand the Shared Objects node, and click Response Objects.3. Select the Log Evidence tab.4. Complete or change settings as indicated in the following table:

Settings Description

Maximum Files Specifies the maximum number of files that the log canstoreNote: When the log reaches the maximum file number, itbegins again with zero (0) and overwrites the existingfiles.

Maximum File Size (in KB) Specifies the maximum file size the log can store

Log File Prefix Specifies the log file name prefix

Log File Suffix Specifies the log filename extensionNote: Both .enc format and .cap (NetMon) formats areaccepted.

5. Click Save.

Viewing system log filesSystem log (syslog) files contain important information about actions the agent has taken, either becausea user performed the action (system restart or manual feature configuration), or the agent has performedthe action itself (such as an automatic update).

Procedure1. In SiteProtector, select the group that contains the Proventia Server for VMware agent.2. Select Agent from the View list.3. In the middle pane, right-click the agent name, and then click Launch → Proventia Manager.


4. Click Yes on the Security Alert.5. Log on to Proventia Manager with the user name admin and the Proventia Manager password you

configured during setup.6. Click OK and accept any alerts. The Proventia Manager home page opens.7. Click System and then click Log Files.8. Select the log file you want to view and click Download.9. Save the file to your local directory.

Forwarding remote log files to SiteProtectorYou can configure a remote syslog server and specify which modules send their messages to that server.

Before you begin


About this task

Advanced parameters fine-tune the performance of your agent; however, assigning inappropriate settingsto an advanced parameter might have significant negative effects on the behavior of the agent. If youenter conflicting or duplicate parameters, the parameter entered last overrides the parameter entered first.


click Open.

2. In the Advanced Parameters tab, click the Add icon.3. Select the Enabled check box.4. Type the parameter name, engine.remote.syslog.enabled.5. Select Boolean.6. In the Value box, select True.7. Click OK.

8. Click the Add icon.9. Select the Enabled check box.

10. Type the parameter name, engine.remote.syslog.address.11. Select String.12. In the Value box, type the address of the remote syslog server.13. Click OK.

Log files available in Proventia Server for VMware agentsProventia Server for VMware agents create and use several log files as they are protecting your virtualassets. This topic lists the various log files and location.


Log name Log location Description

vmo.log /var/log/iss-vmo/vmo.log VMO log file. The VMO log filecontains information about theauthentication status.Note: When the maximum size ofthe log file is reached, then theexisting file will be rolled back and anew vmo.log file will be created. Bydefault, 10 backup files will bemaintained. You can configure thebackup file size in the/etc/vmo/iss-vmo.conf file.

IM_pid.log /var/log/iss-IM/IM_pid.log Anti-rootkit log file. Every virtualmachine has a corresponding log filethat is identified by the process ID(pid). Each log file contains theinformation about the anti-rootkitpolicy settings, anti-rootkitmonitoring details, and unauthorizedchanges to the SSDT and the IDTdata structures by rootkits.

v discovery.log

v discovery_wd.log

v /var/log/iss-dsc/discovery.log

v /var/log/iss-dsc/discovery_wd.log

Discovery scan log files anddiscovery watchdog log files.

v fw1000.log

v fw0000.log

v /var/iss/fw1000.log

v /var/iss/fw0000.log

Firewall log files. Each net engine hasa firewall log that is identified by thezero-based engine index.Note: Only the first packet of aconnection will be logged for blockedpackets when theengine.pamlook.enabled advancedparameter is enabled in the AgentSettings policy.

v engine0.log

v engine1.log

v /var/iss/engine0.log

v /var/iss/engine1.log

IPS net engine log files.

lum.log /var/log/iss-lum/lum.log Update settings log files.

Working with agent health informationThe Proventia Server for VMware agent can provide information to help you assess the health of thesystem on which it resides. You can use this health information to help you ensure the availability ofyour system.

Health summaryProventia Server for VMware agents can provide certain information to keep you apprised of the statusof the agent. The health summary pane in SiteProtector displays this information so that you can manageyour resources and ensure system availability.

Health status

Agents can monitor certain system resources and conditions by running built-in health checks. You canmonitor the results of these health checks to ensure that your system is running efficiently. You canspecify warning and fail levels for certain health checks and, when a health check enters a warning orfailed state, SiteProtector displays a notification in the Notifications tab. In addition, the health status of

Chapter 6. Administering 67

the agent changes to warning or unhealthy in the Agent view.

Health summary icons

Health summary icons appear next to the name of each health check for the selected agent.

Icon Description

Health check failed

Health check passed

Health check provides information, but does not producea notification in the Console

Health check is in a warning state and about to fail

Health check is in an unknown state. Try updatingagents before calling IBM ISS Customer Support

Agent messages

The Agent Messages tab displays agent-specific alerts generated by the agent when error, warning, andinformational conditions exist on your agent. You define which types of alerts the agent sends toSiteProtector in the Agent Settings policy.

Navigation

Locate the agent you want to view health information for and then click Object → Properties.

Navigating to the Health Summary paneHealth summary details for Proventia Server for VMware agents are displayed in the Properties page forthe agent. This information can help you troubleshoot issues with your agent.

Procedure

1. Click the Notifications icon to open the Notifications tab.

Tip: The icon is animated when you have new notifications.2. Select a notification, and then click Action → Open Notification.

Tip: You can also select an agent, click Object → Properties, and then click the Health Summaryicon to open the health summary pane.

Working with health statusProventia Server for VMware agents perform health checks to monitor the health of the system wherethey are installed. You can use the results of these health checks to manage your resources and ensuresystem availability.

Available health checksThis topic describes the health checks available to the Proventia Server for VMware agent.

Health check Description

Disk Space Indicates the amount of free disk space available to theagent


Health check Description

CPU Usage Indicates how much of the processor capacity is beingused

Memory Usage Indicates how much of the memory capacity is beingused

Configuring health checksYou can specify warning and fail levels for certain health checks associated with a Proventia Server forVMware agent. These warnings can help you address issues in a timely manner and before the integrityof the system is compromised.

Procedure1. Select an agent and click Object → Properties → Health Summary.2. In the Health Status tab, locate the health check and click Enable Health Status.

Note: Health check statuses are always displayed in the Health Status tab; when you enable thehealth check, notifications regarding the warning and fail levels for the health check are displayed inthe Notifications tab.

3. Click Configure.4. Complete the following options:

Option Description

Warning Specifies the level at which a warning notification shouldbe displayed in the Notifications tabNote: When the warning level is reached, the healthstatus of the agent changes to warning.

Fail Specifies the level at which a failure notification shouldbe displayed in the Notifications tabNote: When the fail level is reached, the health status ofthe agent changes to unhealthy.

5. Click OK.

Disabling health check notificationsIf you do not want to receive notifications about the status of health checks for your Proventia Server forVMware agent, you can disable the notification.

Procedure1. Select an agent and click Object → Properties → Health Summary.2. In the Health Status tab, locate the health check and click Ignore Health Status.

Results

The status of the health check will still be available on the Health Status tab; however, you will no longerreceive notifications regarding warning and fail levels.

Health check remediesHealth check remedies provide information about the actions you can take to ensure the health of yourProventia Server for VMware agent.

Disk space remedy:


If the amount of free disk space on the drive that contains the Proventia Server for VMware agent hasfallen below the minimum 1.3GB limit, make as much space available as possible on this drive.

CPU usage remedy:

If the amount of CPU being used by the Proventia Server for VMware agent has risen above acceptablelimits, investigate what is using the system resources.

Memory usage remedy:

If the amount of memory being used by the Proventia Server for VMware agent has risen aboveacceptable limits, investigate what is using the system resources.

Network throughput remedy:

If the amount of traffic being inspected by the Proventia Server for VMware agent has risen aboveacceptable limits, consider moving some virtual machines to another ESX server, so that multiple agentscan monitor the high throughput.

Working with agent messagesThe Proventia Server for VMware agent generates error, warning, and informational alerts to keep youapprised of the status of the agent. You can use the information in these alerts to help ensure theavailability of your agent.

Agent messagesA Proventia Server for VMware agent generates alerts when agent-specific error, warning, andinformational conditions exist. You can view these alert messages in SiteProtector so that you can monitorthe state of your agent.

Attention: You define which types of alerts the agent sends to SiteProtector in the Agent Settings policy.

Agent messages icons

The following icons appear in the Severity column of the Agent messages tab for the selected agent.

Note: You define the severity for each alert type in the Agent Settings policy; assign an appropriateseverity to help you prioritize your response time to each type of alert.

Icon Description

Indicates a situation that might require your immediateattention

Indicates a situation that might require a less immediateresponse

Indicates a situation that might require no response at all

Notifications for agent messages

By default, agent messages are only displayed in the Agent Messages tab.

Navigation

Locate the agent you want to view health information for and then click Object → Properties → HealthSummary → Agent Messages.


Forwarding agent messages to the analysis viewBy default, agent messages (agent alerts) are only displayed in the Agent Messages tab in AgentProperties. You can configure SiteProtector to forward these messages to the Analysis view so that youcan remediate them as you remediate other alerts.

About this task

When you forward agent messages to the Analysis view, you forward all agent messages. That is, youcannot forward only high severity messages. You can however, filter low priority messages from theAnalysis view to reduce the amount of data shown in the view.

Procedure

1. In the Analysis view for the group that contains the agent, click the Filters button.2. In Column Filters, select Observance Type.3. In Observance Type Filter, select Informational Only.4. Click OK. SiteProtector begins to forward all agent messages (agent alerts) to the Analysis view.

Viewing information for agent componentsThe Proventia Server for VMware agent generates status information and version information for thecomponents that comprise the agent. You can use this information to troubleshoot issues and to help youensure the availability of your system.

Module statusThe module status information can help you troubleshoot issues with your Proventia Server for VMwareagent.

Note: The icon indicates that an update is available for a component.

Available information

The agent provides the following status and version information:

Category Information Description

Agent Status Agent Information Displays basic information about theagent

Network Information Displays information about the portsused to connect to the network

VM Information Displays operating systeminformation about the virtualmachine

Received Policy Errors Displays error codes received fromthe policy


Category Information Description

Module Status Network Monitoring Displays information about thenetwork monitoring component ofthe agent

Engine Status Displays information about the statusof the engine used for networkmonitoring

Engine Information Displays information about theProtocol Analysis Module and driverversion of the network monitoringcomponent

Anti-Rootkit Displays information about theanti-rootkit component

Discovery Displays information about thediscovery component

VM Events Displays information about the agentcomponent that interacts withVMware components

Navigation

Locate the agent you want to view health information for and then click Object → Properties.

Navigating to the Module Status paneModule status information and module version information for Proventia Server for VMware agents isdisplayed in the Properties page for the agent.

Procedure1. Select an agent and click Object → Properties.

2. Click the Module Status icon.

Agent status - Agent InformationThis topic describes the information provided in the Agent Information pane for a Proventia Server forVMware agent.

Option Description

Base Version Number Indicates the version number of the agent when it wasinstalledNote: This number does not reflect any X-Press Updatesinstalled after the original installation.

Uptime Indicates how long the agent has been runningNote: This interval does not necessarily reflect the timesince the agent was last restarted. It is possible that theagent was offline temporarily and was able to restoreonline status without being restarted.

Last Restart Indicates when the agent was last restarted

Last Firmware Update Indicates when the last core update was installed

Last Intrusion Prevention Update Indicates when the last security content update wasinstalled

Last System Backup Indicates when the last system backup was performed


Option Description

Backup Description Indicates the purpose of the last backup

Module Status - Network MonitoringThis topic describes the information provided in the Network Monitoring pane for a Proventia Server forVMware agent.

Option Description

Module Name Indicates the name of the component

Version Indicates the version of this component

Last Reported Status Indicates the status of the component

Licensed? Indicates if a valid license exists for this component

Module Status - Engine StatusThis topic describes the information provided in the Engine Status pane for a Proventia Server forVMware agent.

Option Description

Number of engines Indicates the number of network protection enginesbeing used by this agent

Engine #1 Status Indicates the status of the first engine

Engine #2 Status Indicates the status of the second engine

High priority alerts Indicates the number of high priority alerts

Medium priority alerts Indicates the number of medium priority alerts

Low priority alerts Indicates the number of low priority alerts

Events blocked Indicates the number of security events blocked by theintrusion prevention engine

Events not blocked Indicates the number of security events detected but notblocked by the intrusion prevention engineNote: An event might be detected but not blockedbecause the Block response is not configured on thesignature that detects this type of event.

Events whitelisted

Module Status - Engine InformationThis topic describes the information provided in the Engine Information pane for a Proventia Server forVMware agent.

Option Description

PAM version Indicates the version of the Protocol Analysis Modulebeing used by the agent

Driver Manager version Indicates the version of the Driver Manager being usedby the agent


Module Status - Anti-rootkitThis topic describes the information provided in the Anti-Rootkit pane for a Proventia Server for VMwareagent.

Option Description





Module Status - DiscoveryThis topic describes the information provided in the Discovery pane for a Proventia Server for VMwareagent.

Option Description





Module Status - VM EventsThis topic describes the information provided in the VM Events pane for a Proventia Server for VMwareagent.

Option Description





Monitoring agent command jobsThe Proventia Server for VMware agent can provide information about actions the agent is performing oris scheduled to perform. Monitor command jobs when you want to ensure that an action you requested isbeing run.

Command jobsCommand jobs are created for a Proventia Server for VMware agent when you perform or scheduleactions such as installing updates or running a report. Monitor command jobs when you want to ensurethat an action you requested is being run or is scheduled to run.

On the Command Jobs pane you can edit the schedule of a scheduled job or cancel the job.

Navigation

Locate the agent you want to view a command job for and click Object → Properties.


Navigating to the Command Jobs paneCommand job details for a Proventia Server for VMware agent are displayed in the Properties page forthe agent. Monitor command jobs when you want to ensure that an action you requested is being run.

Procedure1. Select an agent. and click Object → Properties.

2. Click the Command Job icon.


Chapter 7. Troubleshooting

This section contains information about issues you might encounter as you use your Proventia Server forVMware agent, and describes how to troubleshoot them.

Seeing alerts for allowed traffic

Problem

You have defined a firewall rule in your Proventia Server for VMware policy that accepts traffic from aspecific IP address or range of addresses, but you are seeing alerts in the SiteProtector Console for trafficassociated with these addresses.

Background

The firewall is the first line of defense provided by the agent. Even if you trust traffic from a particular IPaddress (so that traffic makes it through your firewall), there might be security related issues with thepackets. The alerts you are seeing indicate that the agent is protecting your system beyond the firewall.

Solution

Respond to the alerts as your security policy specifies. These are the indications that harmful traffic isreaching your system.

Agent is showing as offline in SiteProtector

Problem

The Proventia Server for VMware agent is showing as offline in the SiteProtector Console, but the agentis still sending alerts.

Background

The Unresponsive Agent Threshold setting specifies the number of minutes that can elapse between agentheartbeat signals before the agent is considered unresponsive. If the Unresponsive Agent Thresholdsetting is shorter than the heartbeat interval for an agent, the SiteProtector system might show that youragent is offline when in fact it is available but it has not sent a heartbeat within the threshold period.

For example, if your Unresponsive Agent Threshold is set to two hours (the default) and your heartbeatinterval is set to six hours, the agent status in the SiteProtector system changes to offline when two hourshave passed because the agent has not sent a heartbeat within those two hours. The agent will not send aheartbeat for another four hours based on the heartbeat interval setting.

Solution

Do one of the following things:v Change the Unresponsive Agent Threshold setting so that it is longer than the heartbeat intervalv Send a Refresh Agent command to initiate a heartbeat from the agent to the SiteProtector system


Changing the unresponsive agent threshold

About this task

If the Unresponsive agent threshold setting is shorter than the heartbeat interval defined for yourProventia Server for VMware agent, your agent might show as offline in the SiteProtector Console when,in fact, the agent is not offline. Change the Unresponsive agent threshold setting to a period longer thanthe heartbeat interval to prevent this from happening.

Procedure1. In the Navigation pane, expand the group the agent is assigned to, right-click Group Settings, and

click Open.2. In the Agent Communication Settings tab, type the number of minutes SiteProtector should wait

between agent heartbeat signals before the agent is considered unresponsive.

Traffic seems to be bypassing analysisProblem

Traffic seems to be reaching the system without being processed by the Proventia Server for VMwareagent.

Background

The agent supports bypass filters and event filters.v Bypass filters allow packets from certain IP addresses to bypass analysis by the firewall and the

security event rules.v Event filters ensure that traffic associated with useful or helpful addresses is not blocked by security

event rules.

Scenario 1

It is possible that a bypass filter or an event filter is allowing traffic to pass to the system.

Scenario 2

It is possible that you disabled a bypass filter but you still have an event filter that is preventing thesecurity event rules from blocking malicious or suspicious traffic.

Scenario 3

It is possible that you disabled an event filter but you still have a bypass filter that is preventing thefirewall and the security event rules from blocking malicious or suspicious traffic.

Solution

Check for filters that might be preventing the agent from inspecting all the traffic you want inspected.

Troubleshooting issues with OneTrust

Problem

You are experiencing issues with your OneTrust token or entitlements.


Solution

Collect the following information, which is vital to troubleshooting the problem:v Namev MyISS usernamev Companyv Product and versionv Tokens stored in SiteProtectorv Description of the tokens stored in SiteProtector

After you collect this information, contact Technical Support for assistance.

Unable to access the security virtual machine (SVM)

Problem

You are unable to access the security virtual machine because traffic from the source host to the securityvirtual machine is getting blocked in the Proventia Server for VMware agent.

Background

Traffic is blocked because the Quarantine-Intruder response for a signature that applies to the securityvirtual machine in the agent is enabled. Since traffic to and from the security virtual machine is subject toIPS inspection, all traffic from the source host to the security virtual machine is blocked. Therefore, thesource host cannot access the local management interface or get any other TCP connections to thesecurity virtual machine.

If you enable the Quarantine-Intruder response for audit or attack signatures, any source IP address thattriggers such an event will have all of its traffic to the destination host blocked. For example, if youenable the Quarantine-Intruder response for the HTTP_GET signature because you want the source of theHTTP_GET signature to be quarantined, all traffic will be blocked. Because the security virtual machine’sIPS functionality inspects traffic over its own management interface, any attempt to access the securityvirtual machine’s local management interface will be blocked, along with any subsequent traffic from theoriginating host.

Solution

Do one of the following things:v Do not enable the Quarantine-Intruder response for signatures that may result in necessary traffic to

the security virtual machine being blockedv Do not include the security virtual machine in the IP address range to which these signatures apply

Note: If you must enable the Quarantine-Intruder response, you can exclude the security virtual machinefrom network monitoring by creating an exception in the Asset Settings policy.

Chapter 7. Troubleshooting 79

Appendix A. Adding or editing a virtual object

As you configure certain policies in the Proventia Server for VMware agent, you might need to add oredit a virtual object to get the policy configuration you need.

Before you begin

As you make changes to a virtual object, keep in mind that any changes you make are applied to all ofthe policies that use that virtual object. This might result in less than desirable results.

Procedure1. Configure the Virtual object based on the following options:

Option Task

Add a virtual object 1. Click the Add icon.

2. Select Enable.

3. Type a name for the virtual object.

4. Type a description for the virtual object.

5. Type the IP addresses of the virtual assets containedin this virtual object.

Edit a virtual object 1. Select the virtual object from the dropdown list, thenclick the Edit icon.

2. Edit any of the following options:

v Enable

v Name

v Description

v IP addresses of the virtual assets contained in thisvirtual object

Enable a virtual object 1. Select the virtual object from the dropdown list, thenclick the Configure icon.

2. In the Enabled column, select the check box for thevirtual object you want to enable.

2. Click OK.

Results

The agent applies the changes you made to the virtual object to all policies that use that virtual object.


Appendix B. Informational links from the product interface

Proventia Server for VMware provides links to more information to help you as you configure policies.The topics listed here provide the information included in those links.

Can I edit this VM event?VM events are events generated by the VMware components associated with your virtual environment.Proventia Server for VMware agents only forward these events to the SiteProtector managementcomponent, so the only available configuration option is whether to display these events in SiteProtector.

Can I disable the global virtual object?Each virtual asset is protected by the policy settings that are defined for the IP address that is assigned tothe asset. The global virtual object contains the IP addresses of all of the virtual assets in your virtualenvironment, which ensures that every asset is always protected. You cannot disable the global virtualobject, because then your virtual assets would not be protected.

Help me understand how to define a trusted assetWhen you define a trusted asset you are ensuring that the asset is never quarantined and that is hasaccess to your network. Trust only those assets that you know comply with your security policy.

Help me understand how to define access control

When you define an access control list you are ensuring that any virtual asset that is quarantined canreach the resources it needs to become compliant with your security policy. For example, the list mayinclude Web sites that have the latest service packs or Web servers that have antivirus applications.

Help me understand how to define my protection scope

When you define your protection scope, you should include all IP addresses that are currently used orcan potentially be used in your virtual environment.

Consider reserving a block of IP addresses for use by virtual machines and then entering that block ofaddresses into the Protection scope field. Reserving a block of addresses can ensure that your virtualenvironment is accurately and completely defined, and that all virtual machines are protected. To ensurethat authorized virtual machines are protected in the future, you can enter the IP addresses as trustedassets in the Network Access Control (NAC) policy so that they will not be quarantined.

If you are unable to reserve a block of IP addresses and create a predefined scope, you should develop aprocess for when a new virtual machine is created. After the IP address is dynamically assigned to thenew virtual machine, remember to enter the IP address in the protection scope list and to the trustedassets list in the Network Access Control (NAC) policy to prevent the virtual machine from beingquarantined.


How does the Any firewall protocol work?If you select the Any option, these criteria are applied.

If you... Then...

set an ICMP code an ICMP clause is added to the rule.

set a source or destination port both a UDP and a TCP clause are added to the rule.

set a Protocol number greater than zero (0) a protocol number clause is added to the rule.

do not specify any protocol settings an IP clause is added to the rule. The source anddestination IP addresses will also be added if you havespecified them.

Note: If you set a Protocol value other than Any, the firewall rule is set to that protocol only.

How do I ensure the agent can authenticate?To have access to the VM events generated in your virtual environment, the Proventia Server for VMwareagent must be able to authenticate with its host ESX server.

To authenticate, the agent needs the IP address of the host ESX server and the user name and passwordfor the administrator account on the server. This information was provided when the agent was installed;however, if the account credentials change or expire, the agent will not be able to authenticate until youupdate the information.

If the agent is ever unable to authenticate, a system event is generated and displayed in the AgentProperties in SiteProtector. This event notifies you that the authentication credentials need to be updated.

See the Updating authentication credentials topic for information about how to update the authenticationcredentials. Open the VM Events policy, press F1, scroll to the bottom of the topic, and then clickUpdating authentication credentials.

How do I use virtual objects?Attention: This topic provides a high-level overview of how to use virtual objects. For a more thoroughexplanation of how to manage policies for your virtual environment, see the Proventia Server forVMware Help system.

Proventia Server for VMware agents provide protection based on the IP address of the virtual asset thatis being protected. This approach ensures that, even if a virtual asset moves from one server to another(VMotions), the appropriate protection is still enforced.

Virtual objects define collections of IP addresses that are assigned to the virtual assets in your virtualenvironment, and they are a convenient way of ensuring that the appropriate policy settings are used toprotect each asset.

The two kinds of virtual objects are:v Global virtual object: Contains the IP addresses of all of the virtual assets in your virtual environment.

this object is defined by the system so that you do not need to maintain this object.v User-defined virtual object: Contains a subset of IP addresses for specific virtual assets in your virtual

environment


When you configure policy settings in asset-based policies, you are configuring the settings for the globalvirtual object, which means you are configuring policy settings for every virtual asset in your virtualenvironment.

If the settings you configure for the global virtual object are not appropriate for all of your virtual assets,you can create a user-defined virtual object. You can then use this virtual object to create an exceptionthat defines customized protection for the subset of virtual assets.

Agents enforce the policy settings specified in exceptions before they enforce the policy settings definedfor the global virtual object. This approach ensures that any customized protection for a virtual asset isenforced, but it also ensures that every asset in your virtual environment is always protected because theglobal virtual object functions as a clean-up virtual object.

Tip: Try to define policy settings that are appropriate across your entire virtual environment and useexceptions only when you must have a customized policy for a collection of virtual assets.

How frequently should I scan the virtual machines?Periodically scanning your virtual machines ensures that the data reported to SiteProtector is current.How frequent you schedule the scan depends on the type of VMotion policy you have established andthe level of activity in your virtual environment.v If you have a liberal VMotion policy or if you frequently create and remove virtual machines, then you

should schedule the scan to run more often.v If you have a restrictive VMotion policy or if you do not frequently create and remove virtual

machines, then you can schedule the scan to run less often or perform a scan manually.

Note: The minimum frequency you can scan a virtual machine is one hour.

Tell me more about logging packets that match firewall rulesWhen you select the Log details of packets check box, logging is enabled for the rule. If enabled, packetdetails that match the rule are sent to the firewall log in the /var/iss/ directory.

You may want to log packets that match firewall rules when troubleshooting unexpected firewallbehavior.

Note: Logging packets that match firewall rules may impact system performance.

Tell me more about network monitoringThe Network Monitoring option controls the Security Events policy settings and the Firewall policysettings.

Tell me more about the intrusion responseThe Intrusion Response option specifies whether the Security Events policy functions in intrusiondetection mode or intrusion prevention mode.

Tell me more about the pass-through modeThe Pass-through Mode option specifies how the agent handles traffic during overload conditions.

Appendix B. Informational links from the product interface 85

Tell me more about the IBM ISS X-Force blocking recommendationsWhen you use the IBM ISS X-Force blocking recommendations in the agent, the block response is enabledautomatically for events (or signatures) as the X-Force recommends.

You can specify whether you want to use blocking responses provided by X-Force. You may need todisable the X-Force blocking occasionally so that you can determine whether current suspicious activityon your network is valid, or so that you can protect against explicit threats to your network.

Note: If you change the X-Force blocking recommendations setting, you must save and then reopen theSecurity Events policy to see the changes.

What do these trust levels mean?The trust level determines how authentication between the agent and the license or update server ismanaged.

Trust level Description

trust-all Specifies that the agent should trust the server and notuse certificates for authentication

first-time-trust Specifies that the agent should trust the server for thefirst communication and then use the server’s certificatefor all future authentications

explicit-trust Specifies that the agent should use the local certificate toauthenticate the server

What is a valid parameter name?Tuning parameters are a powerful way to configure your Proventia Server for VMware agents. Thedefault parameter configuration should meet most of your security and performance needs. If you mustconfigure tuning parameters, use the following resources to find valid parameter names:v Refer to the PAM Help (Help → Attack Signatures → Protocol Analysis Module) for information about

configuring PAM parameters.v See the Agent-specific advanced parameters topic for information about agent-specific parameters.

Open the Agent Settings policy, press F1, scroll to the bottom of the topic, and then clickAgent-specific advanced parameters.

What is a valid parameter name for an update settings parameter?Tuning parameters provide a powerful way to configure your agents. The default parameterconfiguration should meet most of your security and performance needs.

If you must configure tuning parameters, see the Automatic updates advanced parameters topic forinformation about update settings parameters. Open the Update Settings policy, press F1, scroll to thebottom of the topic, and then click Automatic updates advanced parameters.

What is event throttling?Event throttling reduces the number of events received and prevents the console from being flooded withunimportant events. The value is specified in seconds. At most, one event that matches the attack isreported during the specified interval.

Event throttling sets a time window (in seconds) during which multiple events are reported only once.


Tip: Use this feature to prevent your console from being overrun with duplicate events that couldpotentially mask a more dangerous event.

Note: The default value is 0 (zero), which disables event throttling.

What regular expressions are supported in user-defined signatures?Regular expressions (strings) are a combination of static text and variables that the agent uses to detectpatterns in the contexts (network packets) you specify for user-defined event signatures. Use regularexpressions when you create user-defined event signatures if you need the agent to detect more than asingle static text string.

Type the text string in the packet (context) that determines whether an event matches this signature. Youcan use wildcards and other expressions in strings. You must follow standard POSIX regular expressionsyntax. For example, a period is a wildcard character that matches any character, and any periods in aDNS name search must be escaped.

You can use these regular expression syntax in a user-defined event signature.


(r) r

x x

xr x followed by r

\s either a space or a tab (not a hard break or newline)

\d a decimal digit

\″ a double quote

\’ a single quote

\\ a backslash

\n a newline (ASCII NL or LF)

\r a carriage return (ASCII CR)

\t a horizontal tab (ASCII HT)

\v a vertical tab (ASCII VT)

\f a formfeed (ASCII FF)

\b a backspace (ASCII BS)

\a a bell (ASCII BS)

\ooo a specified octal character code

\xhhh a specified hexadecimal character code

. any character except newline

\@ nothing (represents an accepting position)

″″ nothing

[xy-z] x, or anything between y and z, inclusive (characterclass)

[^xy-z] anything but x, or anything between y and z, inclusive

″text″ literal text, without regard for meta-characters

r? r or nothing

r* zero (0) or more occurrences (kleene closure)

r+ one or more occurrences of r (positive kleene closure)



r{m,n} r at least m times, at most n times (repeat operator)

r|l either r or l (alternation operator)

r/l r only if followed by l (lookahead operator)

^r r only at the beginning of a line (bol anchor)

r$ r only at the end of a line (eol anchor)

r, l any arbitrary regular expression

m, n an integer

x,y,z any printable or escaped ASCII character

text a sequence of printable or escaped ASCII characters

ooo a sequence of up to three (3) octal digits

hhh a sequence of hex digits

Where can I see the information gathered by discovery scans?You can view the information gathered by the scans in the SiteProtector console in the Asset view.

Why does the Asset-Specific tab have different VM events than theSystem tab?There is no overlap in the information on the System tab and the Asset-Specific tab.

The VM events listed on the System tab are events that do not occur on one asset or one host; they areevents that happen across the system.

The VM events listed on the Asset-Specific tab are events that can be directly associated with a specific IPaddress and are, therefore, specific to a particular virtual asset or a particular ESX server.

Why is the order of exceptions important?The order in which exceptions are listed is important when you have virtual assets that belong to morethan one virtual object.

Proventia Server for VMware agents process exceptions in the order in which they are listed in the policy.To ensure that a virtual asset that belongs to more than one virtual object is protected, the virtual objectwith the most appropriate policy configuration for that virtual asset must be listed higher in theexceptions table.

Why should I limit the number of exceptions I configure?Agents apply exceptions in the order they are listed in a policy. To ensure that a virtual asset that existsin more than one virtual object is protected by the correct policy settings (the policy settings configuredfor the exception that is listed highest in the Exceptions table), the agent must keep track of which virtualassets have been processed as a part of each exception.

Agent performance is reduced due to this processing and tracking, so you should consider limiting thenumber of exceptions you define. Configure global settings that are appropriate for all of the virtualassets in your virtual environment whenever possible.


Why should I schedule the installation of updates?By scheduling the installation of updates, you can control when core updates are installed. This ensuresthat updates are deployed at a convenient time and after they have been tested in a non-productionenvironment.

Sometimes you do not want to install updates on production systems until you have tested them innon-production environments. This is particularly true for core updates, which must frequently beapplied only during change control windows. You can control the update process in the following ways:v For core updates, you can use the Update Settings tab to control the version of the updates that are

downloaded and installedv For core updates and security content updates, you can use the Scheduled Installation tab to schedule

when updates should be installedv For core updates and security content updates, you can manually manage your update process

Note: This approach is not recommended as keeping current with updates (particularly securitycontent updates) is the best way to protect your assets.


Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries.Consult your local IBM representative for information on the products and services currently available inyour area. Any reference to an IBM product, program, or service is not intended to state or imply thatonly that IBM product, program, or service may be used. Any functionally equivalent product, program,or service that does not infringe any IBM intellectual property right may be used instead. However, it isthe user’s responsibility to evaluate and verify the operation of any non-IBM product, program, orservice.

IBM may have patents or pending patent applications covering subject matter described in thisdocument. The furnishing of this document does not grant you any license to these patents. You can sendlicense inquiries, in writing, to:

IBM Director of LicensingIBM CorporationNorth Castle DriveArmonk, NY 10504-1785U.S.A.

For license inquiries regarding double-byte (DBCS) information, contact the IBM Intellectual PropertyDepartment in your country or send inquiries, in writing, to:

Intellectual Property LicensingLegal and Intellectual Property LawIBM Japan Ltd.1623-14, Shimotsuruma, Yamato-shiKanagawa 242-8502 Japan

The following paragraph does not apply to the United Kingdom or any other country where suchprovisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATIONPROVIDES THIS PUBLICATION ″AS IS″ WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS ORIMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OFNON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Somestates do not allow disclaimer of express or implied warranties in certain transactions, therefore, thisstatement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodicallymade to the information herein; these changes will be incorporated in new editions of the publication.IBM may make improvements and/or changes in the product(s) and/or the program(s) described in thispublication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not inany manner serve as an endorsement of those Web sites. The materials at those Web sites are not part ofthe materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate withoutincurring any obligation to you.


Licensees of this program who wish to have information about it for the purpose of enabling: (i) theexchange of information between independently created programs and other programs (including thisone) and (ii) the mutual use of the information which has been exchanged, should contact:

IBM CorporationProject ManagementC55A/74KB6303 Barfield Rd.,Atlanta, GA 30328U.S.A

Such information may be available, subject to appropriate terms and conditions, including in some cases,payment of a fee.

The licensed program described in this document and all licensed material available for it are providedby IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement orany equivalent agreement between us.

All statements regarding IBM’s future direction or intent are subject to change or withdrawal withoutnotice, and represent goals and objectives only.

TrademarksIBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International BusinessMachines Corp., registered in many jurisdictions worldwide. Other product and service names might betrademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at“Copyright and trademark information” at Copyright and trademark information at www.ibm.com/legal/copytrade.shtml.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

UNIX® is a registered trademark of The Open Group in the United States and other countries.

Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, orboth.

Other company, product, or service names may be trademarks or service marks of others.


http://www.ibm.com/legal/copytrade.shtml

Index

Aadvanced parameters 52, 53

agent-specific 54automatic updates 15, 16

agent alerts 53, 70agent information pane 72agent messages 70

forwarding 71agent properties

agent alerts 70agent messages 70command jobs 74health summary 67locating agent messages 68locating command jobs 75locating health status 68locating health summary 68locating module status 72module status 71

Agent Settings policy 52agent status

agent information pane 72agent-based policies 1agent-specific

advanced parameters 54alert types 52alerts

agent 53error 53informational 53warning 53

anti-rootkit log file 67Anti-rootkit policy 46Asset Settings policy 6, 8, 9, 10, 11asset-based policies 1authentication 42authentication credentials

ESX 45authentication system event 42automatic updates

advanced parameters 15, 16

BBASEL II 8bypass filters 34, 38

Ccolumns

grouping 59command jobs 74

navigating to 75CPU usage health 68

Ddetailed scanning 41discovery log files 67

Discovery policy 41, 42disk space health 68documentation feedback iii

Eengine information 73engine log files 67ESX authentication credentials 45event filters 60evidence logging 65

Ffail closed 8fail open 6, 8feedback iiifilter 59filters

bypass 34, 38clearing had no effect 78event 60interaction among 78response 23

firewall log files 67Firewall policy 33, 34, 37, 38, 39firewall rule actions 39firewall rules 34, 37, 39

deleting 37disabling 37examples, rule syntax 39examples, syntax 39processing order 33rule syntax 39rule syntax examples 39syntax 39syntax examples 39syntax, rules 39

Ggetting started

first tasks 3policy management 1

global virtual object 5

Hhardware resources 61health check

disable notification 69set failed level 69set warning level 69

health checksCPU usage 68disk space 68memory usage 68network throughput 68

health summary 67

health summary (continued)navigating to 68

HIPAA 8

IIBM Internet Security Systems

technical support iiiWeb site iii

intrusion response 6, 8, 17, 20

Llimitation on number of rules 34log evidence 65log files 67

remote 66syslog 65

Mmemory usage health 68module status 71

navigating to 72module version

navigating to 72monitoring

rootkits 46

NNAC 50

protection scope 51quarantined assets 51trusted assets 51

Network Access Control policy 50, 51network monitoring 6, 8, 10network throughput health 68notification

disable 69

OOneTrust

troubleshooting 78open signature rules

ordering 33open signatures 32

Pparameters 54pass-through mode 8

BASEL II 8consideration 8HIPAA 8SB1386 8SOX 8


pausing virtual machine 9policy management

agent-based policies 1asset-based policies 1fundamentals 1global virtual object 1, 5shared object policies 1user-defined virtual object 5

pre-defined signaturessecurity event properties 22

protection scope 6, 8, 10

Qquarantined assets 50, 51

Rregular expressions 30related publications iiiremedy

cpu usage 70disk space 70memory usage 70network throughput 70

remote log files 66resource management 61response filters 23, 25, 27

event throttling 23ICMP code 23ICMP type 23ordering 26with raw 23without raw 23

responses 25, 63email 63response objects 63site group level 63SNMP 63user specified 63

rootkit detection 46

SSB1386 8scanning ports 42schedule discovery scans 41scheduling

updates 15security events 17

customize signatures 17editing multiple 21quarantine 19responses 19signature properties 22signatures 17with raw 17without raw 17

Security Events policy 17, 19, 20, 21, 22,23, 25, 26, 27, 28, 32, 33

signatures, user-defined 28, 30, 31user-defined 28, 30, 31

shared object policies 1SOX 8suspend and resume virtual machine 41syslog files 65

system log files 65

Ttable toolbar 59technical support, IBM Internet Security

Systems iiitraffic

bypassing analysis 78troubleshooting

access to security virtual machine 79alerts for allowed traffic 77filter interaction 78offline agent status 77OneTrust 78

trusted assets 50, 51tuning parameters 52, 53, 54

Uunresponsive agent threshold setting 77,

78update settings 13update settings log file 67Update Settings policy 11, 13, 15, 16

license server 13update server 13

updatescore 11frequency to check for 13scheduling 15security 11uninstalling 16

user-defined 30

Vvirtual object

adding 6custom policy settings 5global 5user-defined 5

vm events 74asset-specific 44categories 42default settings 42severity 42system 44

vm events policy 42authentication credentials 45

vmo log file 67VMotion 1VMware VMotion technology 1VMware vSphere 44VMX file 6, 9, 11vSphere 44

WWeb site, IBM Internet Security

Systems iii

XX-Force blocking recommendations 21


��

Printed in USA

virtual server security for vmware: administrator guide

Technology

virtual server security

agent settings policy

update settings policy

technical documentation

network settings

vm settings

agent health information

asset settings policy