how to create a data source plugin

Upload: sanoun-kilanis

Post on 13-Oct-2015

149 views

Category:

Documents


9 download

TRANSCRIPT

  • Copyright 2014 AlienVault. All rights reserved.

    AlienVault Unified Security Management Solution

    Complete. Simple. Affordable

    How to create a data source plugin

  • AlienVault, AlienVault Unified Security Management, AlienVault USM, AlienVault Open Threat Exchange, AlienVault OTX, Open Threat Exchange, AlienVault OTX Reputation

    Monitor, AlienVault OTX Reputation Monitor Alert, AlienVault OSSIM and OSSIM are trademarks or service marks of AlienVaul t.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 3 of 38

    CONTENTS

    1. INTRODUCTION ..................................................................................................... 4 2. TYPES OF DATA SOURCE PLUGINS .................................................................. 4

    2.1. DETECTOR PLUGINS ............................................................................................. 5 2.2. MONITOR PLUGINS.............................................................................................. 23

    3. HOW TO CREATE A CUSTOM DATA SOURCE PLUGIN .................................. 23 3.1. Exchange Web SMTP server logs .......................................................................... 24 3.2. Creation of the plugin configuration file exchangews.cfg ........................................ 24 3.3. Create the database file exchangews.sql ............................................................... 26 3.4. Activate data source plugins ................................................................................... 26 3.5. Files .local .............................................................................................................. 33

    4. HOW TO USE CUSTOM FUNCTION IN DATA SOURCE PLUGINS .................. 33 APPENDIX A - RECOMMENDATIONS BEFORE CREATING A NEW PLUGIN ......... 35 APPENDIX B - LIST OF DATA SOURCE PLUGINS ................................................... 36

    B.1. DATABASE PLUGINS ........................................................................................... 36 B.2. LOG PLUGINS ....................................................................................................... 36 B.3. MONITOR PLUGINS.............................................................................................. 37 B.4. REMOTE PLUGINS ............................................................................................... 38 B.5. SDEE PLUGINS ..................................................................................................... 38 B.6. WMI PLUGINS ....................................................................................................... 38

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 4 of 38

    1. INTRODUCTION The objective of this document is to explain how to create plugins supported by AlienVault USM.

    A plugin is a software component that adds a specific feature to AlienVault USM. Plugins are used to improve the collection capabilities of the AlienVault Sensors and to indicate to the system, how to understand and to collect events generated by each application and device.

    Sensors receive events from remote hosts using the Syslog, WMI or any other protocols. The sensors use the Collection Plugins (also called Data Source connectors) in order to support the maximum possible number of applications and devices.

    For any system that consumes logs, it is needed a parser to read those logs and extract information from them into standard information fields (username, IP addresses, etc.).

    AlienVault does this via Agent plugin that defines how to collect events from the application or device as well as how events should be normalized before sending them to the AlienVault USM central Server. Log Normalization is essentially breaking down a log message into common fields.

    It is necessary to enable a plugin in order to indicate to the system that must collect events generated by an application or device. Plugins may be pre-configured by AlienVault or defined by users.

    AlienVault plugins are text configuration files and have the extension *.cfg. These files are located in /etc/ossim/agent/plugins in the Sensors file system.

    2. TYPES OF DATA SOURCE PLUGINS There are 2 types: monitor and detector:

    x Detector. These plugins receive logs, information and extract events from them. They process text log information from log files created by RSyslog collection system; and from log data retrieved from remote systems via one of the remote collection protocols such as SDEE and SFTP. These plugins can be:

    Database. They monitor a file in external databases. Logs. They monitor a file, usually receiving data through syslog. Remote Logs. They monitor a file in a remote appliance. SDEE (Security Device Event Exchange). CISCO device logs. WMI (Windows Management Instrumentation). They collect remotely Microsoft

    Windows events and data in an agent-less way.

    x Monitor. These plugins request information from systems, checking the status of the things they monitored at the time of the request. They generate text logs that are fed into the

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 5 of 38

    syslog like normal logs and they are often used to correlate log events into alarms by matching events against the current status of systems.

    2.1. DETECTOR PLUGINS

    2.1.1. DATABASE PLUGINS It is easier to understand how this type of plugin works by means of an example:

    ;; PCI Trace [DEFAULT] plugin_id=1698 [config] type=detector enable=yes source=database source_type=mssql source_ip= source_port=3306 user= password= db= sleep=60 process= start=no stop=no [start_query]

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 6 of 38

    query="select TOP 1 pci.RowNumber from pcitrace as pci ORDER BY pci.RowNumber desc" regexp= userdata1={$3} log= Virus {$3} detected on {$2}, path: {$4} {$5} [query] query="select pci.RowNumber, pci.EventClass, pci.TextData, pci.ApplicationName, pci.NTUserName, pci.LoginName, pci.CPU, pci.Reads from pcitrace as pci ORDER BY pci.RowNumber" regexp= ref=0 plugin_sid=1 username={$5} userdata1={$2} userdata2={$3} userdata3={$4} userdata4={$6} userdata5={$7} userdata6={$8} log={$1},{$2},{$3},{$4},{$5},{$6},{$7},{$8}

    The fields related to database fields are an example for mssql. In case of having mysql, it must be indicated.

    Indicate the point to start to capture. It must be a query to obtain the last event identified by a sequence number. In this case will be:

    select TOP 1 pci.RowNumber from pcitrace as pci ORDER BY pci.RowNumber desc

    The last RowNumer is obtained from a table. A query for getting all values is needed. The same field used in the start query must be selected as first element.

    query="select pci.RowNumber, pci.EventClass, pci.TextData, pci.ApplicationName, pci.NTUserName, pci.LoginName, pci.CPU, pci.Reads

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 7 of 38

    from pcitrace as pci ORDER BY pci.RowNumber"

    Regexp field must be empty:

    regexp= ref=0 plugin_sid=1

    $2 is the second element in the query. In this example is the value of pci.EventClass

    username={$5} userdata1={$2} userdata2={$3} userdata3={$4} userdata4={$6} userdata5={$7} userdata6={$8} log={$1},{$2},{$3},{$4},{$5},{$6},{$7},{$8}

    2.1.2. LOGS This is an example of a log plugin:

    [DEFAULT] plugin_id=1563 [config] enable=yes type=detector source=log location=/var/log/optenet.log

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 8 of 38

    create_file=false process= start=no ; launch plugin process when agent starts stop=no ; shutdown plugin process when agent stops startup= shutdown= [optenet - spam detected] regexp="^(?P\S+)\t(?P\IPV4)\t*\[\S+\]*\t\[(?P\S+)\]\t\[(?P\S+)\].*" event_type=event plugin_sid=1 device={resolv($SRC_IP)} src_ip={resolv($SRC_IP)} userdata1={$domain} userdata2={$src_mail} userdata3={$dst_mail}

    Plugins extract events (SIDs) from logs by matching each line in the log according to a regular expression, and then normalizing out data fields from the text. So when the following log message arrives:

    Feb 8 10:09:06 golgotha sshd[24472]: Failed password for dgil from 192.168.6.69 port 33992 ssh2

    It matches the following SID from the SSH plugin.

    [01 - Failed password]

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 9 of 38

    The information in a log entry to be normalized into field is specified in the regular expression1:

    regexp="(?P\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P\S+).*ssh.*Failed (?Ppublickey|password|none) for\s+(?Pinvalid user)?\s*(?P\S+)\s.*from\s+(?P\S+)\s.*port\s+(?P\d{1,5})"

    And these values are normalized out of it:

    Date = Feb 8 10:09:06 src_ip =192.168.6.69 Username = dgil

    The level of information that can be extracted from a log source is dependent on the level of detail in the plugin. The more SIDs defined, the greater the ability to extract meaning from processed logs.

    2.1.3. REMOTE LOGS This is an example of a remote log plugin:

    # Alienvault plugin # Author: Alienvault Team at [email protected] # Plugin ssh-remote id:4003 version: 0.0.1 # Last modification: 2013-06-05 11:43 # # Accepted products: # openbsd - openssh 5.4 # openbsd - openssh 5.5 # openbsd - openssh 5.6

    1 The bolded fields in the regexp indicate that the matching text will be mapped to information fields during normalization.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 10 of 38

    # openbsd - openssh 5.7 # openbsd - openssh 5.8 # openbsd - openssh 5.8p2 # openbsd - openssh 5.9 # Description: # # Ssh (Secure Shell) is a program for logging into a remote machine # and for executing commands on a remote machine. # URL: http://www.openssh.com # # $Id: ssh.cfg,v 1.12 2010/03/23 16:42:18 juanmals Exp $ # # [DEFAULT] plugin_id=4003 dst_ip=\_CFG(plugin-defaults,sensor) dst_port=22 [config] type=detector enable=yes source=remote-log location=/var/log/auth.log create_file=false process=sshd

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 11 of 38

    start=no stop=no startup=/etc/init.d/ssh start shutdown=/etc/init.d/ssh stop host= user=root passwd= readAll=false [ssh - Failed password] event_type=event regexp="(\SYSLOG_DATE)\s+(?P[^\s]*).*?ssh.*?Failed password for (?P\S+)\s+from\s+.*?(?P\IPV4).*?port\s+(?P\PORT)" plugin_sid=1 device={resolv($sensor)} date={normalize_date($1)} src_ip={$src} dst_ip={resolv($sensor)} src_port={$sport} username={$user}

    Entries marked in bold must always appear because they are used for connecting to remote host:

    source=remote-log host= user=root passwd= readAll=false

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 12 of 38

    2.1.4. SDEE (SECURITY DEVICE EVENT EXCHANGE) SDEE 2 is a standard that specifies the format of messages and protocol used to communicate events generated by security devices. This protocol is used in the Cisco Systems IPS Sensor 5.0. AlienVault support this type of logs collection. AlienVault USM captures events from:

    x Cisco Network Prevention Systems (IPS)

    x Cisco Network Detection Systems (IPS)

    x Cisco Switch IDS

    x Cisco IOS routers with Inline Intrusion Prevention System (IPS) functions

    x Cisco IDS modules for routers

    x Cisco PIX Firewalls

    x Cisco Catalyst 6500 Series firewall services modules (FWSMs)

    x Cisco Management Center for Cisco security agents

    x CiscoWorks Monitoring Center for Security servers

    If you have your own update package from your vendor, you can populate the AlienVault database with the new signatures.

    Go to /usr/share/ossim/scripts/ to update plugin sid information:

    python createCiscoIPSSidmap.py IOS-S416-CLI.pkg.xml DELETE FROM plugin WHERE id = "1597"; DELETE FROM plugin_sid where plugin_id = "1597"; INSERT INTO plugin (id, type, name, description) VALUES (1597, 1, 'Cisco-IPS', 'Cisco Intrusion Prevention System'); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (1597, 5986, NULL, NULL, 'Cisco-IPS: Microsoft GDI GIF Parsing Vulnerability', 3, 4);

    2 This protocol is used in the Cisco Systems IPS Sensor 5.0 to replace Remote Data Exchange Protocol (RDEP).

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 13 of 38

    INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (1597, 5984, NULL, NULL, 'Cisco-IPS: IE COM Object Code Execution', 3, 4); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (1597, 5985, NULL, NULL, 'Cisco-IPS: Quicktime RTSP Content-Type Excessive Length', 3, 4); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (1597, 19159, NULL, NULL, 'Cisco-IPS: Green Dam Youth Escort Software Update Check', 1, 4); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (1597, 19401, NULL, NULL, 'Cisco-IPS: Microsoft Publisher File Parsing Vulnerability', 3, 4);

    This is script generates the needed SQL information to update AlienVault database. Write the following to insert information:

    python createCiscoIPSSidmap.py IOS-S416-CLI.pkg.xml > sdee.sql ossim-db < sdee.sql

    If you want to update cross-correlation information:

    python ciscoIPSOsMap.py IOS-S416-CLI.pkg.xml replace into plugin_reference values (1597, 1109, 3001, 3); replace into plugin_reference values (1597, 1109, 3001, 3); replace into plugin_reference values (1597, 1109, 3001, 3); replace into plugin_reference values (1597, 1109, 3001, 3); replace into plugin_reference values (1597, 2156, 3001, 1); replace into plugin_reference values (1597, 2157, 3001, 3); replace into plugin_reference values (1597, 2157, 3001, 3); replace into plugin_reference values (1597, 2157, 3001, 3); ... ... python ciscoIPSOsMap.py IOS-S416-CLI.pkg.xml > sdee-os.sql ossim-db < sdee-os.sql

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 14 of 38

    Do not forget to restart ossim-server in order to update AlienVault Server cache.

    Follow the instructions below to configure AlienVault Agent and collect events from SDEE capable device:

    1. Add SDEE reference to this file: /etc/ossim/agent/config.cfg

    2. Edit this file: /etc/ossim/agent/plugins/cisco-ips.cfg

    [DEFAULT] plugin_id=1597 [config] type=detector enable=yes source=sdee source_ip= user= password= sleep=5 process= start=no stop=no

    3. Insert the credentials: your source_ip, user and password data.

    4. Restart AlienVault Agent for receiving data from SDEE device.

    Keep in mind the following points:

    x Each time a new session begins with a SDEE device, a Subscription ID will be provided. If the device closes the connection or the connectivity is lost, you have to close the session

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 15 of 38

    in order to continue collecting from the device. The AlienVault Agent closes the session automatically, but if not, you should do it manually.

    x The latest Subscription ID can be found here: /etc/ossim/agent/sdee_sid.data

    x Execute the following:

    python /usr/share/ossim/scripts/closeSDEEsession.py SubscriptionID

    x This closes the last session. If you still have problems, execute the following:

    grep subs /var/log/ossim/agent.log

    x The agent debugging can also turn on, stopping the current agent and starting it manually on verbose mode:

    ossim-agent -v

    x You should get something like this:

    2012-05-07 05:15:40,925 Agent [DEBUG]: 373397c2f80a792a4029fbcc0cd027e50

    2.1.5. WMI (WINDOWS MANAGEMENT INSTRUMENTATION) They collect remotely Microsoft Windows events and data in an agent-less way.

    This is an example of a WMI plugin:

    [DEFAULT] plugin_id=1518 [config] type=detector

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 16 of 38

    enable=yes source=wmi credentials_file=/etc/ossim/agent/wmi_credentials.csv sleep=10 process= start=no stop=no [start_cmd] cmd=wmic -U OSS_WMI_USER%OSS_WMI_PASS //OSS_WMI_HOST "Select LogFile,RecordNumber from Win32_NTLogEvent Where Logfile = 'Application'" | head -n 3 | tail -n 1 | cut -f 2 -d \| regexp= [cmd] cmd = wmic -U OSS_WMI_USER%OSS_WMI_PASS //OSS_WMI_HOST "Select ComputerName,EventCode,Logfile,Message,RecordNumber,SourceName,TimeWritten,User from Win32_NTLogEvent Where Logfile = 'Application' and RecordNumber > OSS_COUNTER" | cat start_regexp=^([^\|]+)\|(\d+)\|([^\|]+)\| regexp="^(?P[^\|]+)\|(?P\d+)\|(?P[^\|]+)\|(?P[^\|]+)\|(?P[^\|]+)\|(?P[^\|]+)\|(?P[^\|]+)\|(?P.*)$" src_ip={resolv($0)} plugin_sid={$1} userdata2={$2} userdata3={$3} userdata4={$4} userdata5={$5} userdata6={$6} username={$7}

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 17 of 38

    2.1.5.1. PREPARING WINDOWS 1. Create a new limited user for not using an administrator account for remote

    connections and make the installation much more secure.

    2. For this example, the user wmiuser and password wmi have been created.

    3. Configure DCOM to allow the user access to the computer remotely.

    4. Grant remote launch to DCOM and activation permissions for our user:

    a) Run Dcomcnfg by selecting Run on the Start menu and typing in Dcomcnfg. Then click OK.

    b) Open Administrative Tools 3 and expand Component Services. Click the secondary button of the mouse over My Computer and select Properties.

    3 This option is not always in the same place. It depends on the Windows version.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 18 of 38

    c) Click on COM Security tab. Then click on Edit Limits under Access Permissions:

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 19 of 38

    d) Click on Anonymous Logon and then on Remote Access:

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 20 of 38

    e) Click OK.

    f) Click Apply.

    g) Click OK.

    5. Run Dcomcnfg by selecting Run on the Start menu and typing in Dcomcnfg. Then click OK.

    6. Open Administrative Tools4 and expand Component Services. Click the secondary button of the mouse over My Computer and select Properties.

    7. Click on COM Security tab and then click on Edit Limits under Launch and Activation Permissions.

    4 This option is not always in the same place. It depends on the Windows version.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 21 of 38

    8. Click Add button.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 22 of 38

    9. Enter the user name and click OK.

    10. In the Launch and Activation Permisson screen, click the options: Remote Launch, Local Activation and Remote Activation. Then click OK.

    11. Click OK.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 23 of 38

    12. Click Apply.

    13. Click OK.

    2.1.5.2. CONFIGURING ALIENVAULT USM Before the activation of wmi plug-ins, it is necessary to create a file having Windows IPs and credentials.

    1. Create a wmi_credentials.csv file.

    vim /etc/ossim/agent/wmi_credentials.csv

    2. Add ips, users and password with following formats:

    127.0.0.1,user,pass 127.0.0.2,domain/user, pass 127.0.0.3,domain/user ,pass

    2.2. MONITOR PLUGINS These plugins are used to execute actions in sensors in correlation time through directives. For instance, the 2005 plugin monitor is used in these 2 files:

    ntop-monitor.cfg session-monitor.cfg

    This is used from monitor directives like:

    /etc/ossim/server/alienvault-attacks.xml:

    3. HOW TO CREATE A CUSTOM DATA SOURCE PLUGIN This section explains how to create a custom plugin to process Exchange Web Server logs through the SIEM engine.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 24 of 38

    3.1. EXCHANGE WEB SMTP SERVER LOGS The log file used for the following handson exercise can be downloaded from here: exchangews.log Once the file has been downloaded, open it to see the logs we are going to parse. Here are some sample lines:

    2011-10-09 05:00:19 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 QUIT - 36A42160 240 6219 68 4 0 SMTP - - - - 1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:40 -0600] "HELO -?+1.1.1.9 SMTP" 250 46

    3.2. CREATION OF THE PLUGIN CONFIGURATION FILE EXCHANGEWS.CFG 1. Global plugin configuration settings:

    x Copy the file ssh.cfg and name this new one as exchangews.cfg.

    x Change the pluginBid field (use 9001 as it is part of the user range that goes up to 10000).

    x Change location to point to the log file /var/log/exchangews.log

    x Delete startup and shutdown fields. These fields are not going to be used (there is no application associated with this plugin).

    x Create a new translation table5, as it is shown below. This step is optional:

    HELO=1 MAIL=2 RCPT=3 DATA=4 QUIT=5

    5 A translation table is used for translating a string to a number in order to use it as plugin_sid. It is necessary to include the function {translate($field_to_translate)} for using a translation table.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 25 of 38

    xxxx=6 DEFAULT_=9999

    2. Create new rules, filling up the fields below. Create two regular expressions to parse the data, because there are two different formats in the log file.

    [exchangews - Generic rule] #2011-10-09 05:00:15 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 HELO - +36A42160 250 0 48 13 0 SMTP - - - - #2011-10-09 05:00:16 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 MAIL - +FROM:+ 250 0 57 45 0 SMTP - - - - event_type=event regexp="(?P\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})\s(?P\IPV4)\s(?P\S+)\s(?P\S+)\s(?P\S+)\s(?P\IPV4)\s\d\s(?P\w+)" date={normalize_date($date)} plugin_sid={translate($type)} dst_ip={resolv($dst_ip)} src_ip={resolv($src_ip)} hostname={$hostname} userdata2={$userdata2} userdata3={$userdata3}

    [exchangews = Generic rule 2 NCSA Format] #1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:40 -0600] "HELO -?+1.1.1.9 SMTP" 250 46 #1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:41 -0600] "MAIL -?+FROM:+ SMTP" 250 46 event_type=event regexp="(?P\IPV4)\s-\s(?P\S+)\s\[(?P\d\d\/\w{3}\/\d{4}:\d\d:\d\d:\d\d)\s-\d{4}\]\s\"(?P\w+)" date={normalize_date($date)} plugin_sid={translate($type)} dst_ip={resolv($dst_ip)}

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 26 of 38

    src_ip={resolv($src_ip)}

    3. Check regular expressions with logs inside the file /var/log/exchangews.log. There are several utilities on the Internet to test regular expressions written in Python. It is recommended to use one of these utilities to check that the created regular expressions match the logs.

    3.3. CREATE THE DATABASE FILE EXCHANGEWS.SQL 1. Create a file using the following examples:

    INSERT INTO plugin (id, type, name, description) VALUES (9001, 1, 'exchangews', 'Exchange E-mail Web server'); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 1, NULL, NULL, 'exchangews: HELO' ,3, 2);

    INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 9999, NULL, NULL, 'exchangews: Generic exchange event' ,3, 2);

    2. Insert file values into the database at the server box.

    cat exchangews.sql | ossim-db

    3. Apply changes in SIEM.

    /etc/init.d/ossim-server restart

    3.4. ACTIVATE DATA SOURCE PLUGINS Choose one of the following options to activate plugins:

    x Through console

    x Through web

    3.4.1. ACTIVATE PLUGINS THROUGH COMMAND LINE CONSOLE To activate plugins by a console, the instructions below should be followed:

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 27 of 38

    1. Open a console terminal application and connect to the AlienVault System by running the following command:

    ssh root@IP_address

    IP_address refers to the default IP of your appliance.

    2. Next, it appears a screen which includes the main menu:

    Figure 1. Activate plugins by console: AlienVault Setup Screen (configure sensor)

    3. By using the arrow keys on the keyboard, select the option 2: Configure Sensor. Accept the selection () by pressing Enter key.

    Use the Tab key on the keyboard to move from to or vice versa:

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 28 of 38

    Figure 2. Activate plugins by console: Configure Sensor (Select Data Sources)

    4. Select the option 4: Select Data Sources. Accept the selection () by pressing Enter key.

    Figure 3. Activate plugins by console: Select Data Sources

    5. Select the plugins to activate. To move between them use the arrow keys on the keyboard and select/deselect it by pressing the Space Bar on the keyboard. Accept the selection () by pressing Enter key. It is possible to select several plugins. Accept the selection () by pressing Enter key.

    6. The Figure 2 appears. Move from to by using the Tab key on the keyboard. Accept the selection () by pressing Enter key.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 29 of 38

    Figure 4. Activate plugins by console: AlienVault Setup Screen (Apply changes)

    7. Select the option 7: Apply changes. Accept the selection () by pressing Enter key.

    Figure 5. Activate plugins by console: apply changes

    8. Apply changes () by pressing Enter key.

    9. The process can take several minutes depending on the number of plugins to activate:

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 30 of 38

    Figure 6. Activate plugins by console: activating data sources

    10. Once the process finishes, the following screen appears:

    Figure 7. Activate plugins by console: changes applied

    11. Press Enter key. The AlienVault Setup screen appears (see Figure 1).

    12. Move from to by using the Tab key on the keyboard. Press Enter key.

    3.4.2. ACTIVATE PLUGINS BY WEB To activate plugins by the web, the instructions below should be followed:

    1. Use a web browser access to your AlienVault console at https://your_ip/

    2. Write a valid IP address in the navigation bar of a web browser.

    3. Enter a valid user name and password and click on Login.

    4. Navigate to Configuration > Deployment:

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 31 of 38

    Figure 8. Activate plugins by web: Configuration > Deployment option

    5. The following window appears:

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 32 of 38

    Figure 9. Activate plugins by web: AlienVault Center

    6. Click on one of the Node Name then, on Sensor Configuration link, and finally on Collection link. A table appears:

    Figure 10. Activate plugins by web: Collection Link

    This table displays 2 columns. The left column shows plugins that are enabled and the right column shows plugins that are available to be enabled.

    To pass an item from one side to the other, drag and drop the item or use the links [+] or [-] which are next to each item.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 33 of 38

    7. To make all changes take effect, click the APPLY CHANGES button.

    3.5. FILES .LOCAL Whenever a plugin file is going to be changed, it is recommended to copy the filename.cfg into another file named filename.cfg.local. Make all necessary changes in the .local file and keep the .cfg file without any change. By copying the file, you preserve the original plugin version over the updated version. The new updates will never overwrite your plugin customization.

    Keep in mind that the original plugins can be modified by AlienVault when an update process has been done.

    4. HOW TO USE CUSTOM FUNCTION IN DATA SOURCE PLUGINS The instructions below should be followed:

    1. Insert a custom function field in the plugin .cfg file:

    [config] custom_functions_file=/etc/ossim/agent/plugin/ssh_custom_functions.cfg

    2. Create a function file having in mind that a function must start with Start Function and must end with End function:

    Start Function log_hello def log_hello(self): return "Hello log!" End Function Start Function log_hello_data def log_hello_data(self,data): return "Hello log: %s" % data End Function

    3. Edit the plugin rules to use the function by using two points:

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 34 of 38

    [ssh - Failed password] # Feb 8 10:09:06 golgotha sshd[24472]: Failed password for dgil from 192.168.6.69 port 33992 ssh2 event_type=event regexp="(\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P\S*).*ssh.*Failed password for (?P\S+)\s+from\s+.*?(?P\IPV4).*port\s+(?P\d{1,5})" plugin_sid=1 sensor={resolv($sensor)} date={normalize_date($1)} src_ip={$src} dst_ip={resolv($sensor)} src_port={$sport} username={$user} userdata1={:log_hello()} userdata2={:log_hello_data($user)}

    It is not possible to apply a builtin function to a custom function (for instance, translate(:log_hello()) ), as the last function which are executed are the custom ones. So translate will receive :log_hello() as a simple chain of chars.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 35 of 38

    APPENDIX A - RECOMMENDATIONS BEFORE CREATING A NEW PLUGIN

    Follow the following recommendations before creating a new plugin:

    x Copy a log as big as possible.

    x Extract events from the log using consecutive grep -v, until the command does not return anything.

    x Use grep to check individually every event. Try to find different values that a same event may take.

    x Discard repeated events.

    x Look for event patterns in order to group them using some identifier such as the same field distribution, for instance.

    x Take into account that your target will be identifying individual events using a plugin_sid, you may need to think what translates you will be using.

    x For every event, find out the number of times it is repeated within the log, using the following command to count the lines.

    wc -l

    x Think if it's worth using a single regex for an event or if several can be grouped together without making the regex very complex.

    x Only capture the fields that are going to be used in correlation later on.

    x Create a generic regex at the end to capture any possible event.

    x Choose the right pre-check, keeping in mind that it applies a first filter to the events.

    x Make sure the regex are alphabetically ordered, starting with AAA and finishing with ZZZ, creating BAA, CAA... groups, leaving room for future expressions.

    x The SQL does not need the sids to be correlative. Gaps can be left in order to make it more maintainable.

    x Lets say from 1000 to 1999 for A event types, from 2000 to 2999 for B event types, etc.

    x Be careful if you add a custom function into a plugin; or if you access to a proprietary database. This may deteriorate the performance if it is not well designed.

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 36 of 38

    APPENDIX B - LIST OF DATA SOURCE PLUGINS

    B.1. DATABASE PLUGINS

    drupal-wiki.cfg eljefe.cfg

    forensics-db-1.cfg mcafee-epo.cfg

    moodle.cfg motion.cfg

    oracle-sql.cfg panda-se.cfg

    post_correlation.cfg vmware-vcenter-sql.cfg

    B.2. LOG PLUGINS

    airlock.cfg aix-audit.cfg aladdin.cfg allot.cfg

    alteonos.cfg amun-honeypot.cfg apache.cfg apache-syslog.cfg

    arpalert.cfg arpwatch.cfg artemisa.cfg aruba.cfg

    aruba-6.cfg ascenlink.cfg avast.cfg axigen-mail.cfg

    bind.cfg bit9.cfg bluecoat.cfg bro-ids.cfg

    cisco-3030.cfg cisco-ace.cfg cisco-acs.cfg cisco-acs-idm.cfg

    cisco-asa.cfg cisco-asr.cfg cisco-fw.cfg cisco-ids.cfg

    cisco-ips-syslog.cfg cisco-nexus-nx-os.cfg cisco-pix.cfg cisco-router.cfg

    cisco-vpn.cfg cisco-wlc.cfg citrix-netscaler.cfg clamav.cfg

    clurgmgr.cfg courier.cfg cyberguard.cfg dhcp.cfg

    dionaea.cfg dovecot.cfg dragon.cfg enterasys-rmatrix.cfg

    exchange.cfg extreme-switch.cfg extreme-wireless.cfg f5.cfg

    f5-firepass.cfg fidelis.cfg fortigate.cfg fortiguard.cfg

    fortimail.cfg fw1-alt.cfg fw1ngr60.cfg gfi.cfg

    glastopng.cfg heartbeat.cfg honeyd.cfg hp-eva.cfg

    iis.cfg imperva-securesphere.cfg

    intrushield.cfg ipfw.cfg

    iphone.cfg iptables.cfg ironport.cfg isa.cfg

    juniper-srx.cfg juniper-vpn.cfg kismet.cfg linuxdhcp.cfg

    lucent-brick.cfg m0n0wall.cfg mcafee.cfg mcafee-antispam.cfg

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 37 of 38

    modsecurity.cfg monit.cfg motorola-firewall.cfg mwcollect.cfg

    nagios.cfg nepenthes.cfg nessus.cfg nessus-detector.cfg

    netgear.cfg netkeeper-fw.cfg netkeeper-nids.cfg netscreen-firewall.cfg

    netscreen-igs.cfg netscreen-manager.cfg netscreen-nsm.cfg nfs.cfg

    nortel-switch.cfg ntsyslog.cfg openldap.cfg optenet.cfg

    oracle-syslog.cfg osiris.cfg ossec.cfg ossec-idm.cfg

    ossec-idm-single-line.cfg ossec-single-line.cfg ossim-agent.cfg p0f.cfg

    pads.cfg paloalto.cfg pam_unix.cfg panda-as.cfg

    pf.cfg postfix.cfg prads.cfg prads_eth0.cfg

    proxim-orinoco.cfg pureftpd.cfg radiator.cfg radware-ips.cfg

    raslogd.cfg realsecure.cfg rrd.cfg rsa-secureid.cfg

    sap.cfg sendmail.cfg serviceguard.cfg shrubbery-tacacs.cfg

    sidewinder.cfg siteprotector.cfg siteprotector-snmp.cfg sitescope.cfg

    smbd.cfg snare.cfg snare-idm.cfg snare-mssql.cfg

    snare-msssis.cfg snort_syslog.cfg sonicwall.cfg sophos.cfg

    spamassassin.cfg squid.cfg squidGuard.cfg ssh.cfg

    stonegate.cfg stonegate_ips.cfg storewize-V7000.cfg sudo.cfg

    suhosin.cfg suricata-http.cfg symantec-ams.cfg symantec-epm.cfg

    syslog.cfg tacacs-plus.cfg tarantella.cfg tippingpoint.cfg

    token-rsa.cfg trendmicro.cfg usbudev.cfg vandyke-vshell.cfg

    vmware-esxi.cfg vmware-vcenter.cfg vmware-workstation.cfg vplus.cfg

    vsftpd.cfg vyatta.cfg W2003DNS.cfg watchguard.cfg

    webmin.cfg websense.cfg wuftp.cfg

    B.3. MONITOR PLUGINS The following plugins are monitor plugins:

  • AlienVault Unified Security Management Solution

    How to create a data source plugin

    DC-00138 Edition 01 Copyright 2014 AlienVault. All rights reserved. Page 38 of 38

    nessus-monitor.cfg nmap-monitor.cfg6

    ntop-monitor.cfg ocs-monitor.cfg

    opennms-monitor.cfg ossim-monitor.cfg

    ping-monitor.cfg session-monitor.cfg

    tcptrack-monitor.cfg whois-monitor.cfg

    wmi-monitor.cfg

    B.4. REMOTE PLUGINS ssh-remote.cfg

    B.5. SDEE PLUGINS cisco-ips.cfg

    B.6. WMI PLUGINS

    wmi-application-logger.cfg wmi-security-logger.cfg

    wmi-security-logger-srv2008.cfg wmi-system-logger.cfg

    6 This plugin is used by certain directives to ascertain whether a DOS or DDoS attack was successful. See directive 34031 for use-case.