arcsight variable and operatorshowto · specifically for arcsight the evaluate_velocity_template...
TRANSCRIPT
Securing Your World
© 2013 Accumuli Security
1
Document Version 1.0
Document Classification Public
Descriptions and examples of some of the ArcSight variables and operators.
ArcSight Variable and Operators HOWTO
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
2
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Copyright & Confidentiality Statements This document is Copyright © Accumuli Security Networks Ltd All rights reserved. The contents of this document may not be copied or duplicated in any form, in whole or in part, without the prior written permission of Accumuli Security Networks Ltd. The information in this document is subject to change without notice. Accumuli Security Networks Ltd shall not be liable for any damages resulting from technical errors or omissions which may be present in this document, or from use of this document. This document is an unpublished work protected by the United Kingdom copyright laws and is proprietary to Accumuli Security Networks Ltd. Disclosure, copying, reproduction, merger, translation, modification, enhancement, or use of this document by anyone other than authorised employees, authorised users, or licensees of Accumuli Security Networks Ltd without the prior written consent of Accumuli Security Networks Ltd is prohibited. Document Information Name: ArcSight Variable and Operators HOWTO Version: Document Version 1.0 Publication Date: 23 September 2013 Classification: Document Classification Public Prepared by: Mark Johnston Document Status: Final Document Control Version Date Author Change Summary Document Version 1.0 23/09/2013 Mark Johnston Final
Accumuli Corrective Action Accumuli Corrective Action
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
3
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Contents Copyright & Confidentiality Statements ......................................................................................... 2 Document Information ...................................................................................................................... 2 Document Control ............................................................................................................................. 2 Introduction ....................................................................................................................................... 5 Background ....................................................................................................................................... 5 EvaluateVelocityTemplate ................................................................................................................ 5
General ........................................................................................................................................ 5 Location ....................................................................................................................................... 5 Example – Search for usernames matching the pattern “1 character followed by 5 digits” ......... 5 Example – Compare ActiveList Entry to a regular expression (regex) ........................................ 6 Example – String Substitution ...................................................................................................... 8 Example – String Substitution Expanded (useful for activel:104) ................................................ 9 Example – Sub Strings ................................................................................................................ 9 Example – Conditional String Checks ......................................................................................... 9 Example – Others ........................................................................................................................ 9
JavaMathematicalExpression ........................................................................................................ 10 General ...................................................................................................................................... 10 Location ..................................................................................................................................... 10 Notes ......................................................................................................................................... 10 Example – Add 10 to the value of deviceCustomNumber1 ....................................................... 10 Example – Calculate distance in KM’s given latitude and longitude of two locations ................ 10
Active List Contains ....................................................................................................................... 12 General ...................................................................................................................................... 12 Notes ......................................................................................................................................... 12
Like Logical Operator ..................................................................................................................... 14 General ...................................................................................................................................... 14 Location ..................................................................................................................................... 14 Notes ......................................................................................................................................... 14 Example – Simple Matching ...................................................................................................... 14
Matches Logical Operator .............................................................................................................. 15 General ...................................................................................................................................... 15 Location ..................................................................................................................................... 15
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
4
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Notes ......................................................................................................................................... 15 Example – Search for an IP address in a String ........................................................................ 15 Example – Search for the domain name “@accumuli.com” in a string ...................................... 15 Example – Search for usernames matching the pattern “1 character followed by 5 digits” ....... 15
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
5
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Introduction Background ArcSight provides a number of “variables” and operators that can be used within Content to help massage and search through event data. This document looks at some of the more useful variables and operators for which there is little existing documentation. EvaluateVelocityTemplate
General Velocity is a java based template engine developed by the apache foundation. It provides a language to reference objects defined in java code. Further information on Velocity can be found at http://velocity.apache.org/engine/devel/vtl-reference-guide.html Below is a table of the most common operators used in Velocity.
Specifically for ArcSight the evaluate_velocity_template variable allows interaction via the velocity language with ArcSight event information and other ArcSight variables. The variable is available for most resources. As an example, it could be used in an Active Channel (by implementing FieldSets) to give an Analyst an immediate view on rule fire information in a column.
Location The EvaluateVelocityTemplate function is found under Variables (local or global) String EvaluateVelocityTemplate
Example – Search for usernames matching the pattern “1 character followed by 5 digits”
evaluate_velocity_template(#if($message.matches(‘[A-Za-z]\d{5}’))matches#else notmatches#end)
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
6
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Example – Compare ActiveList Entry to a regular expression (regex)
Create an Active list with sample values
Create global variable with local variables used to extract AL information
Set actual global variable using local variables (calling $userdata which is local to the global)
Set the event field “reason” with the global variable value
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
7
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Insert some test data, and we see the “reason” field being set with appropriate info
Global variable can also be used in conditions
Setting the “reason” field to “This works” to test use of global variable in conditions
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
8
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Channel with results
This can also be done locally within the rule if running an older version of ESM
Matches logical operator can also be used to test for the match
Example – String Substitution
evaluate_velocity_template(#set($tempvar=$message)$tempvar.replaceAll(‘x’,’y’))
evaluate_velocity_template(#set($tempvar=$message)$tempvar.replaceAll(‘[^x]’,’y’))
Assume message string user:mark:stuffhere:otherstuff:12345 and only want to return user value
evaluate_velocity_template(#set($tempvar=$message)$tempvar.replaceAll(‘user:([^:]+).*’,’$1’))
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
9
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Example – String Substitution Expanded (useful for activel:104)
Assume that activelist expiry comes from a list with two columns, rather than using indexOf …
String1 is evaluate_velocity_template($deviceCustomString4.replaceAll(‘([^\|]*)\|([^\|]*)’,’$1’)
String2 is evaluate_velocity_template($deviceCustomString4.replaceAll(‘([^\|]*)\|([^\|]*)’,’$2’)
Example – Sub Strings
Assume message string of xx|yy|zz
evaluate_velocity_template($message.substring(3,5)), which will return “yy”
evaluate_velocity_template($message.substring(0,$message.indexOf(‘|’))), which will return “xx”
evaluate_velocity_template($message.substring($message.lastIndexOf(‘|’),8)) will return “zz”
Example – Conditional String Checks evaluate_velocity_template(#if($message==”test”)Matches#else Not_Matches#end evaluate_velocity_template(#if($message==”test”)ret_test#elseif($message==”other”)ret_other#else No_Match#end
evaluate_velocity_template(#if($message != “test”)No Match#else Match#end
Example – Others Multiple VM expressions in single template
evaluate_velocity_template(#set($tempvar=$message.length())#if($tempvar > 10)Large#else Small#end
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
10
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
JavaMathematicalExpression
General The function allows for Java mathematical expressions to be applied against fields (numerical only) within ArcSight. This function is not available under all resources, but can be used with Rules, Filters, Data Monitors and FieldSets. By virtue of being available in FieldSets, this function can thus be used within Active Channels. See http://www.singularsys.com/jep/doc/html/functions.html for the available mathematical functions.
Location The JavaMathematicalExpression function is found under Variables (local or global) Arithmetic JavaMathematicalExpression
Notes ArcSight fields are referenced WITHOUT the “$” within the expression. For example “deviceCustomNumber1 + 1”
Example – Add 10 to the value of deviceCustomNumber1
evaluate_jep(deviceCustomNumber1 + 10)
Example – Calculate distance in KM’s given latitude and longitude of two locations
The variables
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
11
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
The screenshot showing the results (validated with online calculator)
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
12
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Active List Contains
General Active list “contains” is not currently supported by ArcSight. However during our testing it was possible to extract regex expressions from an Active List and have those compared against respective string values. The screenshots below show the implementation.
Notes It may be possible to achieve the “activelist contains” by using the ESM plugin. The plugin allows for the interaction of custom java code with events before they are written to or retrieved from the database.
Create the activelist, must have a key field in order to be looked up.
Insert all regex into regex expr field with a separator.
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
13
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Extract the regex expr values and then break them down (like activelist:104). Once broken down evaluate each regex expression against the field required.
Screenshot showing the results
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
14
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Like Logical Operator
General The ArcSight LIKE rule operator functions exactly the same as the SQL LIKE condition with the exception of not being able to use escape characters (escape characters are used to match literal “%” or “_”). For the SQL LIKE condition there are two wildcard matches, these being ‘%’ and ‘_’. The ‘%’ allows matching of any string of any length (including zero length). The ‘_’ allows matching on a single character.
Location Like is a logical operator and thus can be found under Event Conditions, in the Common Conditions Editor.
Notes Variables cannot be used for comparison. For example “Message Like $test”
Example – Simple Matching
Given “event.message = hello” the following will work:
Message Like %ell%
Message Like h%
Message Like hell_
Message Like %ell_
© 2013 Accumuli Security
Securing Your World
ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public
15
Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com
Matches Logical Operator
General The ArcSight MATCHES logical operator provides the capability to use regular expressions in rule conditions only for string matching. The regex engine is based upon Perl 5 syntax, and within ArcSight the regular expression must match the entire string to be true (wildcards can be used).
Location The logical operator can only be used within Rules.
Notes Commas cannot be used within the regular expression (ArcSight limitation). For example you cannot stipulate min and max occurrences \d+{1,3}. If shorthand character classes are used (or any other classes that use a “\”) e.g. “.*\d+\.\d+\.\d+\.\d+.*” the rule editor will automatically insert the escaping “\” and encapsulate the expression in quotes when the OK button is clicked.
Example – Search for an IP address in a String
Assume event.message = Check for IP 10.10.10.10 in this field
Message Matches “.*\\d+\\.\\d+\\.\\d+\\.\\d+.*”
Example – Search for the domain name “@accumuli.com” in a string
Assume event.message = From [email protected] to outside
Message Matches .*@accumuli.com.*
Example – Search for usernames matching the pattern “1 character followed by 5 digits”
Assume event.destinationUserName = I12345
Assume event.destinationUserName = x12345
Destination User Name Matches “[A-Za-z]\\d{5}”