Copyright © <2020> by <Trend Micro Inc.>. All Rights Reserved.

TREND MICRO™Deep Security

AMEA Partner Case Submission Handbook

TREND MICRO™ Deep Security AMEA Partner Case Submission HandbookDocument Version 1.5Prepared by: Michael MortizContributor: Glen Ronidel

2 / 40

Table of contents

Introduction ...................................................................................................................... 4Deep Security Environment ................................................................................................. 5

System Requirement and Sizing Guide .............................................................................. 6Deep Security Version 10 ............................................................................................ 6Deep Security Version 11 ............................................................................................ 6Deep Security Version 12 ............................................................................................ 7

Port numbers, URLs, and IP addresses ............................................................................. 8Deep Security Agent supported platforms ......................................................................... 8

Deep Security Version 10 ............................................................................................ 9Deep Security Version 11 .......................................................................................... 10Deep Security Version 12 .......................................................................................... 10

Deep Security Agent dependencies ................................................................................. 11Windows ................................................................................................................. 11Linux ...................................................................................................................... 11Aix ......................................................................................................................... 11Solaris .................................................................................................................... 12Debian/Ubuntu ........................................................................................................ 13

Deep Security Agent Kernel Support ............................................................................... 15Agent/Manager Upgrade Matrix ...................................................................................... 16

Enabling Debug Logs ........................................................................................................ 19Manager ...................................................................................................................... 19

Enable Advance Logging ........................................................................................... 19Debug Options ......................................................................................................... 20Increase File Size and Count ...................................................................................... 23Generate Diagnostic Package ..................................................................................... 23

Agent ......................................................................................................................... 25Enable Advance Logging ........................................................................................... 25Increase File Size and Count ...................................................................................... 25Generate Diagnostic Package ..................................................................................... 26Enable Anti-malware Debug ...................................................................................... 27

Common Issues ............................................................................................................... 29Deep Security Agent Installation .................................................................................... 30

Troubleshooting ....................................................................................................... 30Logs to Collect ......................................................................................................... 30

Anti-malware Engine Offline .......................................................................................... 32Troubleshooting ....................................................................................................... 32Logs to Collect ......................................................................................................... 33

Security Update Failed .................................................................................................. 34Troubleshooting ....................................................................................................... 34Logs to Collect ......................................................................................................... 34

Agent Offline ............................................................................................................... 34Troubleshooting ....................................................................................................... 34Logs to Collect ......................................................................................................... 36

Crash Issue (kernel panic / bsod) ................................................................................... 37Troubleshooting ....................................................................................................... 37Logs to Collect ......................................................................................................... 37

Performance issue (High CPU, High Memory) .................................................................. 38

3 / 40

Troubleshooting ....................................................................................................... 38Logs to Collect ......................................................................................................... 38

Feedback ......................................................................................................................... 40

4 / 40

Deep Security Partner Handbook

This document serves as a manual for troubleshooting common issues. It provides in-depthtroubleshooting guidelines about configuration, components, and functionality of Deep Security OnPremise.

By following this document, we can ensure that submitted cases are already isolated and verified fromthe given troubleshooting guidelines.

5 / 40

Deep Security Environment

Verify if your environment meets Deep Security requirements.

· System Requirement and Sizing Guide· Port numbers, URLs, and IP addresses· Deep Security Agent supported platforms· Deep Security Agent dependencies· Deep Security Agent Kernel Support· Agent/Manager Upgrade Matrix

6 / 40

System Requirement and Sizing Guide

Requirements vary by version. For previous versions of Deep Security Manager, agents, Relays, orvirtual appliances, see those versions' documentation.

Deep Security Version 10 System Requirement and Sizing Guide

Here are the system requirements for each of the Deep Security components.

· Deep Security Manager requirements

· Deep Security Agent requirements

· Deep Security Virtual Appliance requirements

· Deep Security Notifier requirements

Sizing Guide

· Deep Security Manager and Database Sizing Guide

· Deep Security Relay Sizing Guide

· Sizing for Azure Marketplace

Deep Security Version 11 System Requirement and Sizing Guide

Here are the system requirements for each of the Deep Security components.

· Deep Security Manager requirements

· Deep Security Agent 11.0 requirements

· Deep Security Virtual Appliance requirements

· Deep Security Notifier requirements

Sizing Guide

· Deep Security Version 11 Sizing Guide

7 / 40

Deep Security Version 12 System Requirement and Sizing Guide

Here are the system requirements for each of the Deep Security components.

· Deep Security Manager requirements

· Deep Security Agent requirements

· Deep Security Virtual Appliance requirements

Sizing Guide

· Deep Security Version 12 Sizing Guide

8 / 40

Port numbers, URLs and IP address used by Deep Security

Deep Security default port numbers, URLs, IP addresses, and protocols are listed in the sectionsbelow. If a port, URL or IP address is configurable, a link is provided to the relevant configuration page.

· Deep Security port numbers

· Deep Security URLs

Note: If your network uses a proxy or load balancer, you can configure Deep Security to useit instead of the default ports and URLs listed on this page. For details, see Proxysettings and Load Balancers.

Note: In addition to the ports on this page, Deep Security uses ephemeral ports whenopening a socket (source port). Under rare circumstances these may be blocked, causingconnectivity issues. For details, see Activation Failed - Blocked port.

Deep Security port numbers

The following diagram shows the default ports in a Deep Security system. For details, see the tablebelow the diagram.

Deep Security Agent supported platforms

9 / 40

This guide will show supported agent version and platform per Deep Security Manager version.

Deep Security Agent 10 supported platforms

Deep Security Manager 10.0 supports Deep Security Agent on the operating systems shown in thetable below. If platform support was added in an update release, the minimum update version is notednext to the check mark in the table.

Deep Security Manager supports the use of older agent versions, but we do encourage customers toupgrade agents regularly. New agent releases provide additional security features and protection,higher quality, performance improvements, and updates to stay in sync with releases from eachplatform vendor. Each agent has an end-of-life date. For details, see Deep Security LTS life cycledates.

· Agent platform support table and Docker support

10 / 40

Deep Security Agent 11 supported platforms

Deep Security Manager 11.0 supports the Deep Security Agents on the operating systems shown inthe table below. If platform support was added in an update release, the minimum update version isnoted next to the check mark in the table.

Deep Security Manager supports the use of older agent versions, but we do encourage customers toupgrade agents regularly. New agent releases provide additional security features and protection,higher quality, performance improvements, and updates to stay in sync with releases from eachplatform vendor. Each agent has an end-of-life date. For details, see Deep Security LTS life cycledates and Deep Security FR life cycle dates.

· Agent platform support table

· Docker support

· Systemd support

See also Agent platform support policy.

Deep Security Agent 12 supported platforms

Deep Security Manager 12.0 supports the Deep Security Agents on the operating systems shown inthe table below. If platform support was added in an update release, the minimum update version isnoted next to the check mark in the table.

Deep Security Manager supports the use of older agent versions, but we do encourage customers toupgrade agents regularly. New agent releases provide additional security features and protection,higher quality, performance improvements, and updates to stay in sync with releases from eachplatform vendor. Each agent has an end-of-life date. For details, see Deep Security LTS life cycledates and Deep Security FR life cycle dates.

· Agent platform support table

· Docker support

· Systemd support

See also Agent platform support policy.

11 / 40

Pre-checking the dependencies of Deep Security Agent before installation

This list dependencies needed by the agent for installation.

· Windows· Linux· AIX· Solaris· Debian/Ubuntu

Windows

An external tool, such as depends.exe, can check if there is any DLL file missing.

Linux

Below are the dependencies for Linux:

· linux-vdso.so.1 (0x00007ffc86953000)· /opt/ds_agent/lib/libwx_baseu-2.9.so.4 (0x00007f584ac58000)· /opt/ds_agent/lib/dsa_core.so (0x00007f584a7d7000)· /opt/ds_agent/lib/libslb.so (0x00007f584a5cb000)· /opt/ds_agent/lib/liblua.so (0x00007f584a399000)· /lib64/libdl.so.2 (0x0000003c87200000)· /opt/ds_agent/lib/libcrypto.so.1.0.0 (0x00007f5849f50000)· /opt/ds_agent/lib/libssl.so.1.0.0 (0x00007f5849ce0000)· /usr/lib64/libstdc++.so.6 (0x0000003c92a00000)· /lib64/libm.so.6 (0x0000003c88200000)· /lib64/libgcc_s.so.1 (0x0000003c92600000)· /lib64/libpthread.so.0 (0x0000003c87a00000)· /lib64/libc.so.6 (0x0000003c87600000)· /lib64/libz.so.1 (0x0000003c88600000)· /lib64/ld-linux-x86-64.so.2 (0x0000003c86e00000)· /lib64/libacl.so.1 (0x0000003c93600000)· /opt/ds_agent/lib/libwxsqlite.so (0x00007f5849aaf000)· /opt/ds_agent/lib/libsqlite.so (0x00007f5849825000)· /opt/ds_agent/lib/libexpat.so.1 (0x00007f58495fb000)· /lib64/libattr.so.1 (0x0000003c97200000)

AIX

Below are the dependencies for AIX:

12 / 40

· /opt/ds_agent/lib/librpc.so· /opt/ds_agent/lib/dsa_core.so· /opt/ds_agent/lib/libfingerprint.so· /opt/ds_agent/lib/libwx_base-2.8.a· /opt/ds_agent/lib/libsqlite.so· /opt/ds_agent/lib/libssl.so· /opt/ds_agent/lib/libcrypto.so· /usr/lib/libpthread.a(shr_xpg5_64.o)· /opt/ds_agent/lib/libz.so· /opt/ds_agent/lib/liblua.so· /opt/ds_agent/lib/libstdc++.a(libstdc++.so.6)· /opt/ds_agent/lib/libgcc_s.a(shr.o)· /usr/lib/libc.a(shr_64.o)· /unix· /opt/ds_agent/lib/libexpat.a(libexpat.so.0)· /opt/ds_agent/lib/libslb.so· /usr/lib/libiconv.a(shr4_64.o)· /usr/lib/libpthreads.a(shr_xpg5_64.o)· /usr/lib/libcrypt.a(shr_64.o)

13 / 40

Solaris

Solaris 11 will perform some dependency check based on the publisher before the programinstallation.

To disable the publisher, run any of the following commands:

pkg unset-publisher solarispkg set-publisher --disable solaris

Note that Solaris 11 requires gcc-45-runtime. If IPS function is required, OS also needs the kshpackage as it provides the ksh93 package, which provides the /usr/bin/sh shell.

Debian/Ubuntu

Below are the dependencies for Debian and Ubuntu:

· linux-vdso.so.1 (0x00007fff301ff000)· /opt/ds_agent/./lib/libwx_baseu_net-2.9.so.4 (0x00007f24cd439000)· /opt/ds_agent/./lib/libwx_baseu-2.9.so.4 (0x00007f24ccf81000)· /opt/ds_agent/./lib/dsa_core.so (0x00007f24ccb1e000)· /opt/ds_agent/./lib/libslb.so (0x00007f24cc911000)· /usr/lib/libstdc++.so.6 (0x00007f24cc5f3000)· /lib/libm.so.6 (0x00007f24cc370000)· /lib/libgcc_s.so.1 (0x00007f24cc15a000)· /lib/libpthread.so.0 (0x00007f24cbf3e000)· /lib/libc.so.6 (0x00007f24cbbdb000)· /opt/ds_agent/./lib/libssl.so.0.9.8 (0x00007f24cb985000)· /opt/ds_agent/./lib/libcrypto.so.0.9.8 (0x00007f24cb5f3000)· /opt/ds_agent/./lib/liblua.so (0x00007f24cb3c2000)· /usr/lib/libz.so.1 (0x00007f24cb1ab000)· /lib/libdl.so.2 (0x00007f24cafa7000)· /lib64/ld-linux-x86-64.so.2 (0x00007f24cd63d000)· /lib/libacl.so.1 (0x00007f24cad9f000)· /usr/lib/libapt-pkg.so.4.10 (0x00007f24caa99000)· /opt/ds_agent/./lib/libwxsqlite.so (0x00007f24ca869000)· /opt/ds_agent/./lib/libsqlite.so (0x00007f24ca5e0000)· /opt/ds_agent/./lib/libexpat.so.0 (0x00007f24ca3b8000)· /lib/libattr.so.1 (0x00007f24ca1b3000)· /lib/libutil.so.1 (0x00007f24c9fb0000)· /opt/ds_agent/./lib/libwx_baseu-2.9.so.4 (0x00007f4b94e44000)· /opt/ds_agent/./lib/libsqlite.so (0x00007f4b94bbc000)· /opt/ds_agent/./lib/dsa_core.so (0x00007f4b94759000)· /opt/ds_agent/./lib/libdsam.so (0x00007f4b9452e000)· /opt/ds_agent/./lib/libssl.so.0.9.8 (0x00007f4b942d8000)· /opt/ds_agent/./lib/libscancache.so (0x00007f4b93d3a000)· /opt/ds_agent/./lib/libvmpdcommon.so (0x00007f4b93b31000)· /opt/ds_agent/./lib/libglib-2.0.so.0 (0x00007f4b9381d000)

14 / 40

· /opt/ds_agent/./lib/libgthread-2.0.so.0 (0x00007f4b933f3000)· /lib/librt.so.1 (0x00007f4b91319000)

15 / 40

Deep Security Agent Kernel Support

Deep Security Agent Linux kernel support· Deep Security Agent 12.0 Linux kernel support

· Deep Security Agent 11.3 Linux kernel support

· Deep Security Agent 11.2 Linux kernel support

· Deep Security Agent 11.1 Linux kernel support

· Deep Security Agent 11.0 Linux kernel support

· Deep Security Agent 10.3 Linux kernel support

· Deep Security Agent 10.2 Linux kernel support

· Deep Security Agent 10.1 Linux kernel support

· Deep Security Agent 10.0 Linux kernel support

· Deep Security Agent 9.6 SP1 Linux kernel support

· Deep Security Agent 9.5 SP1 Linux kernel support

You can also use a JSON version of the complete list of the supported Linux kernels for Deep SecurityAgent 10.0 and higher with scripts and automated workflows.

16 / 40

Agent/Manager Upgrade Matrix

Manager Version 10 11 1211.3 FR X X Y11.2 FR X X Y11.1 FR X X Y11.0 LTS Update 20 11.0.415 X X Y11.0 LTS Update 19 11.0.408 X X Y11.0 LTS Update 18 11.0.399 X X Y11.0 LTS Update 17 11.0.389 X X Y11.0 LTS Update 15 11.0.381 X X Y11.0 LTS Update 14 11.0.374 X X Y11.0 LTS Update 13 11.0.360 X X Y11.0 LTS Update 12 11.0.349 X X Y11.0 LTS Update 11 11.0.346 X X Y11.0 LTS Update 10 11.0.340 X X Y11.0 LTS Update 9 11.0.336 X X Y11.0 LTS Update 8 11.0.328 X X Y11.0 LTS Update 7 11.0.319 X X Y11.0 LTS Update 6 11.0.308 X X N11.0 LTS Update 5 11.0.298 X X N11.0 LTS Update 4 11.0.292 X X N11.0 LTS Update 3 11.0.270 X X N11.0 LTS Update 2 11.0.249 X X N11.0 LTS Update 1 11.0.240 X X N11.0 GA 11.0.221 X X N10.3 FR X Y Y10.2 FR X Y Y10.1 FR X Y Y10.0 LTS Update 25 10.0.3466 X Y Y10.0 LTS Update 24 10.0.3461 X Y Y10.0 LTS Update 23 10.0.3458 X Y Y10.0 LTS Update 21 10.0.3456 X Y Y10.0 LTS Update 20 10.0.3445 X Y Y10.0 LTS Update 19 10.0.3437 X Y Y10.0 LTS Update 18 10.0.3432 X Y Y10.0 LTS Update 17 10.0.3428 X Y Y10.0 LTS Update 16 10.0.3419 X Y N10.0 LTS Update 15 10.0.3410 X Y N10.0 LTS Update 14 10.0.3402 X Y N10.0 LTS Update 13 10.0.3392 X Y N10.0 LTS Update 12 10.0.3382 X Y N10.0 LTS Update 11 10.0.3376 X Y N10.0 LTS Update 10 10.0.3374 X Y N10.0 LTS Update 9 10.0.3370 X Y N10.0 LTS Update 8 10.0.3367 X Y N

17 / 40

10.0 LTS Update 7 10.0.3359 X N N10.0 LTS Update 6 10.0.3346 X N N10.0 LTS Update 5 10.0.3325 X N N10.0 LTS Update 4 10.0.3315 X N N10.0 LTS Update 3 10.0.3305 X N N10.0 LTS Update 2 10.0.3297 X N N10.0 LTS Update 1 10.0.3271 X N N10 GA 10.0.3259 X N N9.6SP1_P1_U26 9.6.4218 Y Y N9.6SP1_P1_U25 9.6.4214 Y Y N9.6_SP1_P1_U24 9.6.4212 Y Y N9.6SP1_P1_U23 9.6.4208 Y Y N9.6SP1_P1_U22 9.6.4204 Y Y N9.6SP1_P1_U21 9.6.4199 Y Y N9.6SP1_P1_U20 9.6.4193 Y Y N9.6SP1_P1_U19 9.6.4191 Y Y N9.6SP1_P1_U18 9.6.4184 Y Y N9.6SP1_P1_U17 9.6.4179 Y Y N9.6SP1_P1_U16 9.6.4178 Y Y N9.6SP1_P1_U15 9.6.4174 Y Y N9.6SP1_P1_U14 9.6.4168 Y Y N9.6SP1_P1_U13 9.6.4159 Y Y N9.6SP1_P1_U12 9.6.4152 Y Y N9.6SP1_P1_U11 9.6.4145 Y Y N9.6SP1_P1_U10 9.6.4143 Y Y N9.6SP1_P1_U9 9.6.4133 Y Y N9.6SP1_P1_U8 9.6.4125 Y Y N9.6SP1_P1_U7 9.6.4111 Y Y N9.6_SP1_P1_U6 9.6.4093 Y Y N9.6_SP1_P1_U5 9.6.4085 Y Y N9.6_SP1_P1_U4 9.6.4072 Y Y N9.6_SP1_P1_U3 9.6.4064 Y Y N9.6_SP1_P1_U1 9.6.4014 Y Y N9.6_SP1_P1_CP1 9.6.4000 Y Y N9.6_SP1_P1 9.6.3400 Y Y N9.6_SP1 9.6.3177 Y N N9.6 GA 9.6.1589 N N N9.5SP1_P3_U8 9.5.7235 Y N N9.5SP1_P3_U7 9.5.7232 Y N N9.5SP1_P3_U6 9.5.7230 Y N N9.5_SP1_P3_U5 9.5.7228 Y N N9.5_SP1_Patch3_U4 9.5.7226 Y N N9.5_SP1_Patch3_U3 9.5.7222 Y N N9.5_SP1_P3_CP1 9.5.7200 Y N N9.5_SP1_P3 9.5.7008 Y N N9.5_SP1_P2 9.5.6511 N N N9.5_SP1_P1 9.5.6008 N N N9.5_SP1 9.5.5600 N N N

18 / 40

9.5_Patch1 9.5.4112 N N N9.5_CP1 9.5.2459 N N N9.5 GA 9.5.2456 N N N

19 / 40

Enabling Debug Logs

Enabling debug logs gathers more detailed information for your Deep Security Environment and canhelp support identify issue easily.

· Manager· Agent

Deep Security Manager

Enabling debug logs gathers more detailed information for your Deep Security Environment and canhelp support identify issue easily.

· Enable Advance Logging

· Debug Options

· Increase File Size and Count

· Generate Diagnostic Package

Enable advance logging (Debug)

Follow steps below to enable DSM debug.

Windows LinuxEnable debug using the following steps: Enable debug using the following steps:1. Stop the Deep Security Manager service. 1. Stop the Deep Security Manager

service.2. Open the logging.properties file under: 2. Open the logging.properties file under:For Windows: ..\Program Files\TrendMicro\Deep Security Manager\jre\lib\

For Linux: /opt/dsm/jre/lib

3. Add one or more of the debug optionsenumerated below, depending on the issueyou encountered. We recommend adding thelines to the last part of the file for easymonitoring and maintenance.Debug Options

Ex. If you have AD Synchronization IssuesJust addcom.thirdbrigade.manager.core.util.UserUtilities.level=ALL on the last line

3. Add one or more of the debug optionsenumerated below, depending on the issueyou encountered. We recommend addingthe lines to the last part of the file for easymonitoring and maintenance.DebugOptions

Ex. If you have AD Synchronization IssuesJust addcom.thirdbrigade.manager.core.util.UserUtilities.level=ALL on the last line

If you are unsure on what to use just addbelow to enable all logging.com.thirdbrigade.level = ALL

20 / 40

If you are unsure on what to use just add belowto enable all logging.com.thirdbrigade.level = ALL4. Save the changes and close the file. 4. Save the changes and close the file.5. Start the DSM service. 5. Start the DSM service.

(# /opt/dsm/dsm_s start)

Note: Can Enable Debugging via DSM as well. (DSM > Administration > SystemInformation > Diagnostic Logging

Debug Options

Here are the debugging options:

Option 1: UI Related Issues

· com.thirdbrigade.manager.webclient.screens.level=ALL

Option 2: Configuration and Protocol Issues

· com.thirdbrigade.manager.webclient.screens.level=ALL

· com.thirdbrigade.manager.core.protocol.session.CommandProtocolSession.level=ALL

· com.thirdbrigade.manager.core.scheduler.jobschedulers.jobs.HostUpdaterJob.level=ALL

Option 3: Scan Management Issues

· com.thirdbrigade.manager.core.scheduler.jobschedulers.jobs.HostUpdaterCommand.level=ALL

· com.thirdbrigade.manager.core.scheduler.jobschedulers.jobs.HostUpdaterCommandGetStatusEvents.level=ALL

· com.thirdbrigade.manager.core.db.AgentEventPeer.level=ALL

Option 4: Anti-Malware Scan Issues

· com.trendmicro.ds.antimalware.jobs.HostUpdaterCommandInvokeAntiMalwareScanAction.level=FINE

21 / 40

· com.thirdbrigade.manager.core.scheduler.jobschedulers.jobs.HostUpdaterCommandVirtualAgentSync.level=FINE

· com.thirdbrigade.manager.core.db.AgentEventPeer.level=FINE

Option 5: All screens, including Wizard-related Issues

· com.thirdbrigade.manager.webclient.screens.level = ALL

Option 6:  vCenter-related Issues

· com.thirdbrigade.manager.core.virtual.level=ALL

· com.thirdbrigade.manager.core.virtualization.vmware.level = ALL

Option 7:  Database-related Issues

· com.thirdbrigade.persistence1.level = ALL

Option 8: Startup Information Logging

· com.thirdbrigade.manager.webclient.initialization.level = ALL

· com.thirdbrigade.manager.core.Core = ALL

· com.thirdbrigade.manager.core.security.ClientSecurityManager.level=ALL

Option 9: Host Updater Job (including agent security configuration XML) Debugging

· com.thirdbrigade.manager.core.scheduler.jobschedulers.jobs.HostUpdaterJob.level=ALL

· com.thirdbrigade.manager.core.scheduler.jobschedulers.jobs.HostUpdaterCommand.level=ALL

Option 10: Agent Communication Protocol Logging

· com.thirdbrigade.manager.core.protocol.level = ALL

Option 11: Detection Engine (ie Recommendation Scans) Logging

· com.thirdbrigade.manager.core.detectionengine.level=ALL

Option 12: Manager Job-related Issues

· com.thirdbrigade.manager.core.scheduler.jobschedulers.HostJobScheduler.level=ALL

· com.thirdbrigade.manager.core.scheduler.JobQueuingThread.level=ALL

· com.thirdbrigade.manager.core.scheduler.JobCreationThread.level=ALL

· com.thirdbrigade.manager.core.scheduler.ManagerJobs.level=ALL

Option 13: AD Synchronization Issues

· com.thirdbrigade.manager.core.util.UserUtilities.level=ALL

Option 14: Dashboard Bean Performance Issues

· com.thirdbrigade.manager.webclient.screens.DashboardBean.level=ALL

22 / 40

· com.thirdbrigade.manager.webclient.ScreenServlet.level=ALL (to replace the preceding bullet)

Option 15: Active Update Issues

· com.thirdbrigade.manager.core.au.level=ALL

· com.thirdbrigade.manager.webclient.ActiveUpdateServlet.level=ALL

· com.trendmicro.ds.vulnerabilityprotection.au

Option 16: Maintenance Job and Entity Purge-related Issues

· com.thirdbrigade.manager.core.scheduler.jobschedulers.jobs.MaintenanceJob.level=ALL

· com.trendmicro.ds.integrity.db.EntityPeer.level=ALL

Option 17: Enable ALL Logging on the manager

· com.thirdbrigade.level = ALL

Option 18: Job Load and Performance Profile related

· com.thirdbrigade.manager.core.scheduler.JobQueuingThread.level=ALL

· com.thirdbrigade.manager.core.scheduler.JobLoad.level=ALL

Option 19: NSX syncing related logging

· com.thirdbrigade.manager.core.virtual.NSXSync.level=ALL

Option 20: Rehoming

· com.thirdbrigade.manager.core.scheduler.jobschedulers.jobs.HostUpdaterSession

· com.trendmicro.manager.core.cloud.CloudSupportingServices

Option 21: AMI Baking Support

· com.thirdbrigade.manager.core.scheduler.jobschedulers.jobs.HostUpdaterSession

· com.trendmicro.manager.core.cloud.CloudSupportingServices

Option 22: CTD jobs

· Com.thirdbrigade.manager.core.scheduler.jobschedulers.SuspiciousFileSubmission.Job.level=ALL

· Com.thirdbrigade.manager.core.scheduler.jobschedulers.DDAnReportQueryJob.level=ALL

Option 23: DDAn API

· Com.trendmicro.manager.core.ddan.level=ALL

Option 24: CTD AM

23 / 40

· Com.trendmicro.ds.antimalware.ctd.level=ALL

· Com.trendmicro.ds.antimalware.models.AntiMalwareQuarantinedFilesWizardDean.level=ALL

Option 25: Enable ALL Logging on the manager

· com.thirdbrigade.level = ALL

Increase File Size and File Count

This will increase default size of log files and the maximum number of logs files that can be generated.We recommended to increase this when replication might take hours or days so we can capture asmuch log as we can during the replication. Once completed with the replication revert to defaultsettings.

Windows LinuxOpen the logging.properties file. Change thevalues for the following below

Open the logging.properties file. Change thevalues for the following below

java.util.logging.FileHandler.limit =10000000(Default)

java.util.logging.FileHandler.limit =10000000(Default)

java.util.logging.FileHandler.count = 5(Default) java.util.logging.FileHandler.count = 5(Default)

Generate Diagnostic Package

To diagnose an issue, your support provider may ask you to send a diagnostic package containingdebug information for either or both:

Deep Security Manager diagnostics

Create a diagnostic package for Deep Security Manager

1. Go to Administration > System Information.

2. Click Create Diagnostic Package.

The package will take several minutes to create. After the package has been generated, a summarywill be displayed and your browser will download a ZIP file containing the diagnostic package.

Enable debug logs for Deep Security Manager

24 / 40

In addition to a diagnostic package, your support provider may ask you to enable diagnostic logging.

Don't enable diagnostic logging unless recommended by your supportprovider. Diagnostic logging can consume large amounts of disk space and increaseCPU usage.

1. Go to Administration > System Information.2. Click Diagnostic Logging.

3. In the wizard that appears, select the options requested by your support provider.

If you have a multi-tenant Deep Security Manager, and the issue that you want to diagnose only occurswith a specific tenant, select that tenant's name in the option that appears. This will focus the debuglogs, and minimize performance impacts while debug logging is enabled.

Some features need more time and disk space to collect enough debug logs. For example, you mightneed to increase Maximum log file size to 25 MB and the time period to 24 hours for Database-related Issues and Cloud Account Synchronization - AWS.

If you decrease Maximum number of log files, Deep Security Manager does notautomatically delete existing log files that now exceed the maximum. For example, if youreduce from 10 to 5 log files, server5.log to server9.log would all still exist. To reclaimdisk space, manually delete those files from the file system.

While diagnostic logging is running, Deep Security Manager will display themessage Diagnostic Logging enabled on the status bar. If you changed the default options, thestatus bar will display the message Non default logging enabled upon diagnostic loggingcompletion.

4. To find diagnostic logging files, go to the root directory of the Deep Security Manager, and lookfor file names with the pattern server#.log, such as server0.log.

25 / 40

Deep Security Agent

Enabling debug logs gathers more detailed information for your Deep Security Environment and canhelp support identify issue easily.

· Enable Advance Logging

· Debug Options

· Increase File Size and Count

· Generate Diagnostic Package

Enable advance logging (Debug)

Follow steps below to enable DSA debug.

Windows LinuxTo enable detailed logging:1. Create a file named ds_agent.ini under the %SystemRoot% directory (example: C:\Windows\ds_agent.ini).

1. Modify the /etc/syslog.conf(or /etc/rsyslog.conf) file by adding any of thefollowing lines:

2. Put the either line inside the file: local0.info     /var/log/messagesTrace=Appl Beat Cmd Cfg Conn HTTP Log LstnSrvc SSL

local0.*         /var/log/messages

Trace=* 2. Create a file named ds_agent.conf underthe /etc directory.

Alternatively you can add additional switches 3. Add the following line inside theds_agent.conf file:

Trace.file_name=dsa_debug_Computer1 Trace=Appl Beat Cmd Cfg Conn HTTP LogLstn Srvc SSL

Trace.file_count=10 This will enable extra tracing for the varioussub-components of the Deep Security Agent.If you do not want output from a certaincomponent, just exclude that component fromthe line.

Trace.file_size=1048576 4. Restart the Trend Micro Deep SecurityAgent Service using this command:

Restart dsa service # service ds_agent restartDelete the ds_agent.ini once done with replicationand restart agent.

The output goes to syslog using "local0", sothe location depends on your /etc/syslog.confsettings.Delete the ds_agent.ini once done withreplication and restart agent.

Increase File Size and File Count

26 / 40

This will increase default size of log files and the maximum number of logs files that can be generated.We recommended to increase this when replication might take hours or days so we can capture asmuch log as we can during the replication.

Windows LinuxOpen the ds_agent.ini file. Change the valuesfor the following below

Open the ds_agent.conf file. Change thevalues for the following below

dsa.log.maxSize dsa.log.maxSizedsa.log.maxFiles dsa.log.maxFiles

Generate Diagnostic Package

To diagnose an issue, your support provider may ask you to send a diagnostic package containingdebug information for either or both:

Deep Security Agent diagnostics

For an agent, you can create a diagnostic package either:

· via the Deep Security Manager

· using the CLI on a protected computer (if the Deep Security Manager cannot reach the agentremotely)

Create an agent diagnostic package via Deep Security Manager

 Deep Security Manager must be able to connect to an agent remotely to createa diagnostic package for it. If the Deep Security Manager cannot reach the agent remotely, or if theagent is using agent-initiated activation, you must create the diagnostic package directly from theagent.

1. Go to Computers.2. Double-click the name of the computer you want to generate the diagnostic package for.3. Select the Actions tab.4. Under Support, click Create Diagnostics Package.

5. Click Next.

The package will take several minutes to create. After the package has been generated, a summarywill be displayed and your browser will download a ZIP file containing the diagnostic package.

When the System Information checkbox is selected, it might create a huge diagnostic package thatcould have a negative impact on performance. The checkbox is greyed out if you are not a primarytenant or do not have the proper viewing rights.

Create an agent diagnostic package via CLI on a protected computer

Linux, AIX, or Solaris

1. Connect to the server that you want to generate the diagnostic package for.

27 / 40

2. Enter the command:

sudo /opt/ds_agent/dsa_control -d

The output shows the name and location of the diagnostic package: /var/opt/ds_agent/diag

Windows

1. Connect to the computer that you want to generate the diagnostic package for.

2. Open a command prompt as an administrator, and enter the command.

In PowerShell:

& "\Program Files\Trend Micro\Deep Security Agent\dsa_control" -d

In cmd.exe:

cd C:\Program Files\Trend Micro\Deep Security Agent

dsa_control.cmd -d

The output shows the name and location of the diagnostic package: C:\ProgramData\TrendMicro\Deep Security Agent\diag

Collect debug logs with DebugView

On Windows computers, you can collect debug logs using DebugView software.

Only collect debug logs if your support provider asks for them. During debug logging, CPU usage willincrease, which will make high CPU usage issues worse.

1. Download the DebugView utility.2. If self-protection is enabled, disable it.3. Stop the Trend Micro Deep Security Agent service.4. In the C:\Windows directory, create a plain text file named ds_agent.ini.

5. In the ds_agent.ini file, add this line:

trace=*

6. Launch DebugView.exe.7. Go to Menu > Capture.

8. Enable these settings:

· Capture Win32

· Capture Kernel

· Capture Events

9. Start the Trend Micro Deep Security Agent service.10.Export the information in DebugView to a CSV file.11.Re-enable self-protection if you disabled it at the beginning of this procedure.

Enable advance logging (Debug)

Follow steps below to enable AM debug.

28 / 40

Windows Linux1. Disable the self-protection and stop the AMSPservice.

Create file"/var/opt/ds_agent/am/ds_am.ini"with below content:

2. Go to the AMSP installation folder. By default, itis located under C:\Program Files\TrendMicro\AMSP.

/var/opt/ds_agent/am/ds_am.inimain=debug_level=7,vmpd_log_file_count=[2~1000],vmpd_log_file_MB=[1~100]3. Open the AmspConfig.ini file with an

administrative permission.4. Set the following parameters and save thechanges:DebugLogAMSPServiceStart=1DebugLogMode=0Where the values of DebugLogMode are asfollow:0 - Local mode1 - Remote pipe mode

vmpd_log_file_count andvmpd_log_file_MB are supportedafter: DSA 9.6_SP1_P1_U12_CP (9.6.2-8198)DSA 10 Update 4 (DSSEG-1305,merged into 10.0.0-2470)For example, log level is 6and vmpd_log_file_count=10,vmpd_log_file_MB=10main=debug_level=6,vmpd_log_file_count=10,vmpd_log_file_MB=10

5. Start the AMSP service.6. Open the AMSP installation folder\debug\ folderand make sure the Amsp_LocalDebugLog.log fileexists.

29 / 40

Deep Security Common Issues

· Deep Security Agent Installation· Anti-malware Engine Offline· Security Update Failed· Agent Offline· Crash Issue (kernel panic / bsod)· Performance issue (High CPU, High Memory)

30 / 40

Deep Security Agent Installation

Deep Security Agent Installation Issue

Issues related to installing Deep Security Agent core component only.

Troubleshooting Agent Installation

Procedure

☐

Check if the agent installer is imported in the DSM console.

To install Deep Security Agent, you must download the agent installer and loadpackages for the Agent's protection modules into Deep Security Manager. To view alist of software that has been imported into Deep Security Manager, go toAdministration > Updates > Software > Local.

Deep Security is modular. Initially, Deep Security Agent only has core functionality.When you enable a protection module, then the agent downloads that plug-in andinstalls it. So before you activate any agents, first download the agent softwarepackages into Deep Security Manager's database ("import" them) so that they willbe available to the agents and relays.

☐Make sure all dependencies are installed in the system.Pre-checking the dependencies of Deep Security Agent before installation

☐Confirm if platform is supported by your agent version.Agent platform support table

☐For non-windows systems check if the kernel version is supported.Run command uname -r Deep Security Agent Linux kernel support

☐

If using deployment script:The deployment scripts generated by Deep Security Manager for Windows agentdeployments require Windows PowerShell version 4.0 or later. You must runPowerShell as an Administrator and you may have to run the following command tobe able to run scripts: Set-ExcecutionPolicy RemoteSignedIf you want to deploy an agent to an early version of Windows or Linux that doesn'tinclude PowerShell 4.0 or curl 7.34.0 at a minimum, remove the --tls1.2 tag (Linux)or[Net.ServicePointManager]::SecurityProtocol=[Net.SecurityProtocolType]::Tls12;line (Windows) so that early TLS (version 1.0) is used to communicate with themanager. Also make sure that early TLS is allowed on the manager and relays.See Determine whether TLS 1.2 is enforced and Enable early TLS (1.0) for details.

Logs to collect for Agent installation Issues

LogsPlatform Logs Detail Location

☐ Windows msinfo32.exe System Information .n.a

31 / 40

☐ Windows setupapi.log driver install log.(OS log file)

%SystemRoot%\

☐ Windows setupapi.dev.log driver install log.(Device install file)

%SystemRoot%\inf\

☐ Windows setupapi.app.log driver install log.(Device install file)

%SystemRoot%\inf\

☐ Windows ds_agent.log Agent install log %programdata%\TrendMicro\Deep Security Agent\diag

☐ Windowssc query tbimdsasc query ds_agentscreenshot

AMSP install log%programdata%\TrendMicro\Deep Security Agent\am

☐ Linux uname -a machineinformation

.n.a

☐ Linux cat /proc/driver/dsa/info

Driver Information .n.a

☐ Linux rpm -qa ds_agent PackageInformation

.n.a

☐ Linux (syslog local0facility)

DSA main log. (Depend on syslogconfiguration)

☐ Linux ds_agent.log Agent install log /var/opt/ds_agent/diag/

☐ Linux

lsmod | grep -idsa_filtercat /proc/driver/dsa/infoscreenshot

Network DriverStatus screenshot

32 / 40

Anti-malware Engine Offline

Anti-malware Engine offline issue.

Troubleshooting Anti-Malware engine offline

Agent-based protection

Procedure

☐Check if there are other AV product/s (e.g. Officescan or other 3rd party AV)causing conflict with Deep Security.

☐Make sure “Trend Micro Solutions Platform” (Windows) service or ds_am (Linux)process is running.

☐

Check in DSM Local Software repository if the agent package of the version you areusing is already imported

· Go to DSM > Administration > Updates > Software > Local

If you're using a Linux server, your kernel might not be supported. For moreinformation, see Error: Module installation failed (Linux).

☐If using DNS in the environment, check if the hostname resolution is working fine

· nslookup Relay-Hostname· nslookup DSM-Hostname

☐Confirm if the agent can connect to the relay server on port 4122

· telnet [Relay-Hostname] 4122 or curl –v telnet://[Relay-Hostname]:4122· telnet [Relay-IP] 4122 or curl –v telnet://[Relay-IP]:4122

☐

For Windows:Check if the following drivers are installed. (Note: This step is applicable to Windowsmachines only)

· sc query AMSP· sc query tmactmon· sc query tmcomm· sc query tmvetmgr

For Linuxps -ef | grep dsroot     32501     1  0 17:23 ?        00:00:00 /opt/ds_agent/ds_am -g ../diag -v 6 -d /var/opt/ds_agent/am -m /opt/ds_agent/lib/libvmpd_full_scan.so -m /opt/ds_agent/lib/libvmpd_scanctrl.so -m /opt/ds_agent/lib/libvmpd_dsa_rtscan.so

☐If agent was upgraded specially on Windows platform it needs a server reboot to complete the agent upgrade.

☐

Most of the time, agent reinstallation fixes this type of issue. If possible, perform anagent reinstallation.

· Deactivate the agent from DSM console or use command “dsa_control –r”locally to reset the agent

· Uninstall the agent· Restart the machine (for windows only)

33 / 40

· Reinstall the agent

Agentless protection

Procedure

☐

In the Deep Security Manager, verify synchronization to vcenter and nsx. Underthe Computers section, right click on your Vcenter and go to Properties. Click TestConnection. Then click on the NSX tab and test the connection. Click Add/UpdateCertificate in case the certificate has changed.

☐ Log into the NSX manager and verify that it is synching to vCenter properly.

☐Log into your vSphere client and go to Network & Security > Installation > ServiceDeployments. Check for errors with Trend Micro Deep Security and GuestIntrospection, and resolve any that are found.

☐In vSphere client, go to Network & Security > Service Composer. Verify that thesecurity policy is assigned to the appropriate security group.

☐Verify that your VMware tools are compatible with Deep Security. For moreinformation, see VMware Tools 10.x Interoperability Issues with Deep Security.

☐Verify that the File Introspection Driver (vsepflt) is installed and running on the targetVM. As an admin, run sc query vsepflt at the command prompt.

☐

All instances and virtual machines deployed from a catalog or vApp template fromvCloud Director are given the same BIOS UUID. Deep Security distinguishesdifferent VMs by there BIOS UUID, so a duplicate value in the vCenter causes anAnti-Malware Engine Offline error. To resolve the issue, see VM BIOS UUIDs are notunique when virtual machines are deployed from vApp templates (2002506).

Logs to collect for AM Engine Offline Issues

LogsPlatform Logs Detail Location

☐ WindowsAM debugDiagnosticPackage

DiagnosticPackage with AMdebug enabled

AM debugDiagnostic Package

☐ LinuxAM debugDiagnosticPackage

DiagnosticPackage with AMdebug enabled

AM debugDiagnostic Package

34 / 40

Security Update Failed

· Troubleshooting· Logs to Collect

Troubleshooting Security Update Failed

Procedure

☐Check if the Deep Security Manager and Deep Security Relay are using higher buildversion than the agents. Check the update number

☐Confirm if the deep security relay can download updates without issues and hasgreen status in the console.

☐ Make sure Relay Group being used has an active working relay.

☐If using proxy server with ssl inspection, kindly add the Trend Micro URLs(specifically the Active Update) in the bypass/exception list in the web proxy server. Port numbers, URLs, and IP addressePort numbers, URLs, and IP addressess

☐Check connection from agent to relay server:Telnet Relay_server 4122.Ping test between DSA and DSR.

Logs to collect for Security Update Issues

LogsPlatform Logs Detail Location

☐ WindowsDiagnosticPackage

Diagnostic Package AgentDiagnosticPackage

☐ LinuxDiagnosticPackage

Diagnostic Package AgentDiagnosticPackage

☐ WindowsDiagnosticPackage

Diagnostic Package RelayDiagnosticPackage

☐ LinuxDiagnosticPackage

Diagnostic Package RelayDiagnosticPackage

☐ Window/LinuxResult of telnetand ping test

Result of telnet and ping test screenshot

☐ Window/Linux Packet Capture Wireshark or tcpdump pcap file

Agent Offline

· Troubleshooting· Logs to Collect

35 / 40

Troubleshooting Agent Offline Issues

A computer status of "Offline" or "Managed (Offline)" means that the Deep Security Manager hasn'tcommunicated with the Deep Security Agent's instance for some time and has exceeded the missedheartbeat threshold. (See Configure the heartbeat.) The status change can also appear in alerts andevents.

Procedure

☐

On the computer with the agent, verify that the Trend Micro Deep Security Agentservice is running. Method varies by operating system.· On Windows, open the Microsoft Windows Services Console (services.msc)

or Task Manager. Look for the service named ds_agent.· On Linux, open a terminal and enter the command for a process listing. Look

for the service named ds_agent or ds-agent, such as:sudo ps -aux | grep ds_agentsudo service ds_agent status

· On Solaris, open a terminal and enter the command for a process listing. Lookfor the service named ds_agent, such as:sudo ps -ef | grep ds_agentsudo svcs -l svc:/application/ds_agent:default

☐

Check connection from Agent to Manager:

From DSATelnet DSM 4120.Ping test between DSA and DSM.

From DSMTelnet DSA 4118.Ping test between DSA and DSM.

If telnet fails, trace the route to discover which point on the network is interruptingconnectivity.

On Linux, enter the command:

traceroute [agent IP]

On Windows, enter the command:

tracert [agent IP]

☐Check the agent's or manager's system time is incorrect (required by SSL/TLSconnections)

☐

Check if Computer has left the context of the private networkThis can occur if roaming endpoints (such as a laptop) cannot connect to themanager at their current location. Guest Wi-Fi, for example, often restricts openports, and has NAT when traffic goes across the Internet.

☐

Verify if communication direction is configure properly. Bi-directional communicationis enabled, but only one direction is allowed or reliable (see Configurecommunication directionality).

36 / 40

Logs to collect for Agent Offline Issues

LogsPlatform Logs Detail Location

☐ WindowsDiagnosticPackage

Diagnostic Package AgentDiagnosticPackage

☐ LinuxDiagnosticPackage

Diagnostic Package AgentDiagnosticPackage

☐ WindowsDiagnosticPackage

Diagnostic Package RelayDiagnosticPackage

☐ LinuxDiagnosticPackage

Diagnostic Package RelayDiagnosticPackage

☐ Window/LinuxResult of telnetand ping test

Result of telnet and pingtest

screenshot

☐ Window/Linux Packet Capture Wireshark or tcpdump pcap file

☐ Window/LinuxNetworkDiagram

Netowrk diagram ofaffected server to DSM

screenshot

37 / 40

Crash Issue (kernel panic / bsod)

· Troubleshooting· Logs to Collect

Troubleshooting Crash Issues

Procedure

☐Work with OS vendor (e.g Microsoft, Redhat etc.) to identify the cause of kernel panicor BSOD.

☐Check if platform is supported and agent security requirements are met.System Requirement and Sizing Guide

Logs to collect for Agent Offline Issues

LogsPlatform Logs Detail Location

☐ WindowsDiagnosticPackage

Diagnostic Package AgentDiagnosticPackage

☐ LinuxDiagnosticPackage

Diagnostic Package AgentDiagnosticPackage

☐ WindowsWindows FullDump

Windows Full DumpWindows FullDump

☐ WindowsWindowsEvents

Windows System,Application, Security Events

Event Viewer

☐ Linux kdump (vmcore) kdump (vmcore) kdump (vmcore)

☐ Linux messages logs messages logs /var/log/messages

☐ Linux dmesg dmesg dmesg

☐ Window/Linux

Full RCA reportfrom OS vendor

Full RCA report from OSvendor

Full RCA reportfrom OS vendor

38 / 40

Performance issue (High CPU, High Memory)

· Troubleshooting· Logs to Collect

Performance issue (High CPU, High Memory, Network)

For performance we need to quantify the performance issue being encountered compared to normaloperation. Ex. Download is taking too low which usually finish in 2 min now taking 10 minutes.

Procedure

☐Identify which process is consuming high CPU or high memory by disabling eachmodule being used one by one until the issue disappear.

☐

If Anti-malware is found causing the issue:

· Ensure proper scan exclusion list is added. o Review this recommended scan exclusion list and add whichever is

necessary ~ https://success.trendmicro.com/solution/1059770o If you have third party software installed that is not listed on the article,

reach out to software vendor for the AV Exclusion listso For other references in configuring Anti-Malware, please refer to the

articles below;1. Enable and configure anti-malware2. Configure malware scans 3. Create anti-malware exceptions4. Performance tips for anti-malware

☐If issue is caused by Intrusion Prevention, please ensure you remove allunnecessary IPS rules and run a recommendation scan to get trend microrecommended rules.

☐If issue is caused by Integrity Monitoring and/or Log Inspection, review all therules you have and only assign the rules you need.

☐For Network performance issue on cluster environment make sure cluster dedicatedinterface is bypassed in filter scanning.

Performance issue (High CPU, High Memory)

LogsPlatform Logs Detail Location

☐ WindowsDiagnosticPackage

Diagnostic Package AgentDiagnosticPackage

☐ LinuxDiagnosticPackage

Diagnostic Package AgentDiagnosticPackage

☐ WindowsTask Managerscreenshot

Task Manager screenshot oftop process

Task Manager

☐ Linux top - look for Top Results top - look for PID

39 / 40

PID oftop_processtop -Hp [PID]gstack [PID]

of top_processtop -Hp [PID]gstack [PID]

40 / 40

Feedback

For comments and suggestions you can answer a quick survey below.

· Comments and Suggestions