BIG-IP Access Policy Manager and Splunk Templates

Summary BIG-IP Access Policy Manager (APM) provides 28 reports to ease the integration of F5

BIG-IP APM logs and the Splunk reporting system. Three are in advanced view report

format, two are in advanced form report format, and nineteen are in saved extended fields

search report format. You can use these reports as-is or as templates to create your own

customized reports. Fifteen of the reports can be displayed in graphical form on the BIG-IP

APM Dashboard. The reports are grouped into three search categories: Geolocation,

Session, and Access.

Prerequisites By default, a Splunk server must be installed and configured to receive syslog entries on

UDP port 514. BIG-IP APM-specific logs are automatically grouped into the sourcetype –

“apm_log.” BIG-IP APM Splunk templates are specifically looking for syslog entries that

contain sourcetype=“apm_log.”

To view Combined Reports in Splunk, you need to enable logging of the session.user.* and

session.client.* session variables in the access policy. Refer to http://support.f5.com/kb/en-

us/solutions/public/11000/200/sol11253.html for details.

Note: To distinguish output from multiple BIG-IP APM syslog sources, you could add a

qualifier to the search command. For example, host=“192.168.1.123”

sourcetype=“apm_log” and so on.

These reports were developed and tested using BIG-IP version 11.5.0.

Customization F5 Networks Access Policy Manager dashboard and saved search reports are placed in

your Splunk installation server’s $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default

directory in XML format. You can add or remove search groups in the

$SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/nav/default.xml file. You

can add or remove graphical reports in the APM_dashboard.xml file located in the

$SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views directory. You can

add or remove saved search reports in the

$SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/savedsearch.conf file. Refer to

http://docs.splunk.com/Documentation/Splunk for detailed customization instructions.

Advanced Search Three advanced view search reports, Geolocation by VIP – Report, Sessions policy steps –

Report, Session variables by session ID – Report, and two advanced form search reports,

Geolocation by state by VIP – Report and Top Users by Access Type – Report, can be

found under the $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views

directory. The files are named geoview.xml, session_policy.xml, session_vars.xml,

geobyvip.xml, and topusersbyaccess.xml. You can find instructions about how to build

advanced form searches on a Splunk server at

http://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/AdvancedIntro.

BIG-IP APM Dashboard The BIG-IP APM Dashboard includes 15 graphical reports and 1 raw event report:

• Geolocation by state – bar chart presentation of users by state in the US.

• Geolocation by country – column chart presentation of users by country.

• Geolocation by region – pie chart presentation of users by region.

• ActiveSync by User – pie chart presentation of the top ActiveSync users.

• ActiveSync by Device – pie chart presentation of top devices used to access

ActiveSync.

• Max Concurrent Sessions – area chart presentation of maximum concurrency over time.

• Session Throughput – area chart presentation of bytes in and out over time.

• Access by User Agent – pie chart presentation of top access user agent count.

• Access Types – pie chart presentation of top access type count.

• Top Users by login – pie chart presentation of top user access by login.

• Top Users by throughput – pie chart presentation of top user access by throughput.

• Client Type over Platform – column chart presentation of client access type by operating system.

• Auth Success vs. Failure – column chart presentation of authentication success

versus failure over time.

• Access by IP – pie chart presentation of top IP addresses that accessed the system.

• Last 5 BIG-IP APM Events – raw syslog entry presentation of the last 5 BIG-IP APM

events that occurred.

• Unique client IP-addresses count – shows the number of unique client IP addresses.

Note: Most widgets on the dashboard have own TimeRangePicker. One of the options it provides is All time. By default, this option searches through all events on the Splunk server. This behavior may be unacceptable if there is a large amount of data on the server. So, you can change the search range by deleting comments in savedsearches.conf on all lines that contain “dispatch.*_time” and setting a more limited time period. Or, you can disable the All time option by creating a file called times.conf that contains the following lines:

[all_time]

disabled = 1

Details on Splunk BIG-IP APM Reports ActiveSync by User – Report This report searches for “User=*” syslog entries and charts the count by user. It then sorts by count. Actual search command:

search= sourcetype="apm_log" User="*" | chart count by User | sort – count

ActiveSync by Device – Report

This report searches for “DeviceType=*” syslog entries and charts the count by device type.

It then sorts by count.

Actual search command: search= sourcetype="apm_log" DeviceType="*" | chart count by DeviceType | sort – count

Geolocation by state – Report This report searches for “New session” syslog entries. It then uses a regular expression to

parse geo_state, geo_country, and geo_region. It removes empty geolocation entries. Then

it charts the number of users by state and country.

Actual search command: search = sourcetype="apm_log" New session | rex field=_raw "\(ST=(?<geo_state>.*)/CC=(?<geo_country>.*)/C=(?<geo_region>.*)\) at VIP (?<vip>\\d+\.\\d+\.\\d+\.\\d+)" | where len(geo_state) > 0 | chart count over geo_state by geo_country | sort – count Note: Further customization can be done by modifying the where clause to create Geolocation by state by VIP – Report, for example, where len(geo_state) >0 AND vip=”10.10.123.111”.

Geolocation by country – Report This report searches for “New session” syslog entries. It then uses a regular expression to

parse geo_state, geo_country, and geo_region. It removes empty geolocation entries. Then

it charts the count by country and region.

Actual search command: search = sourcetype="apm_log" New session | rex field=_raw "\(ST=(?<geo_state>.*)/CC=(?<geo_country>.*)/C=(?<geo_region>.*)\) at VIP (?<vip>\\d+\.\\d+\.\\d+\.\\d+)" | where len(geo_country) > 0 | chart count over geo_country by geo_region | sort – count Note: Further customization can be done by modifying the where clause to create Geolocation by country by VIP – Report, for example, where len(geo_country) > 0 AND vip=”10.10.123.111”.

Geolocation by region – Report This report searches for “New session” syslog entries. It then uses a regular expression to

parse geo_state, geo_country, and geo_region. It removes empty geolocation entries. Then

it charts the count by region.

Actual search command: search = sourcetype="apm_log" New session | rex field=_raw "\(ST=(?<geo_state>.*)/CC=(?<geo_country>.*)/C=(?<geo_region>.*)\) at VIP (?<vip>\\d+\.\\d+\.\\d+\.\\d+)" | where len(geo_region) > 0 | chart count by geo_region | sort – count

Geolocation by state by VIP – Report This is an advanced form search report. Please refer to $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/geobyvip.xml for detailed implementation information. The following search template is used to generate the report:

sourcetype=“apm_log” New session | rex field=_raw “(ST=(?<geo_state>.*) /CC=(?<geo_country>.*) /C=(?<geo_region>.*)\) at VIP (?<vip>\d+.\d+.\d+.\d+)” | where len(geo_state) > 0 AND vip=“$virtualIP$” | chart count over geo_state by geo_country | sort – count

It performs another search for virtual IP addresses ($virtualIP$) and puts them into a dropdown list:

sourcetype=“apm_log” New session | rex field=_raw “(ST=(?<geo_state>.*) /CC=(?<geo_country>.*) /C=(?<geo_region>.*)\) at VIP (?<vip>\d+.\d+.\d+.\d+)” Note: This second search uses its own time interval. It can be set by changing the “earliest” and “latest” parameters of the “populating search” tag in the $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/geobyvip.xml file.

Geolocation by VIP – Report This is an advanced view search report. It uses the Splunk modules: TimeRangePicker,

SearchSelectLister, CovertToIntention, HiddenSearch, SubmitButton,

HiddenChartFormatter, FlashChart, and ViewRedirectorLink. Please refer to

$SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/geoview.xml for

detailed implementation information. Geolocation by VIP – Report contains two subreports:

Geolocation by state and Geolocation by country. You can specify a pre-populated search

of virtual IP address to generate the report.

Allow ACL – Report This report searches for “allow ACL” syslog entries. It then uses a regular expression to

parse out allowed URLs, network protocol, client IP addresses, and server IP addresses.

Then it charts the count by URL and sorts them by count.

Actual search command: search = sourcetype="apm_log" allow ACL | rex field=_raw "packet: (?<url>.\\S+) (?<net_protocol>.\\w+) (?<clntip>.\\d+\.\\d+\.\\d+\.\\d+:\\d+) -> (?<svrip>.\\d+\.\\d+\.\\d+\.\\d+:\\d+)" | chart count by url | sort – count

Note: The search can be further refined to generate other types of reports. For example,

In the Top Access by Backend Server IP Address – Report, you could slightly change the

search command to:

search = sourcetype="apm_log" allow ACL | rex field=_raw "packet: (?<url>.\\S+)

(?<net_protocol>.\\w+) (?<clntip>.\\d+\.\\d+\.\\d+\.\\d+:\\d+) -> (?<svrip>.\\d+\.\\d+\.\\d+\.\\d+:\\d+)" |

chart count by svrip | sort – count

For the Top Access by Client IP Address – Report, you could use “chart count by clntip” to

extract the information you need.

Session Variables – Summary This report searches for “session.*” syslog entries. It then uses a regular expression to

parse the session variable and the session variable value. Then it tabulates the session

variables.

Actual search command: search = sourcetype="apm_log" session.* | rex "(?<sess_var>session\..\\S+) is

(?<sess_var_val>.*\\Z)" | top limit=100 sess_var

Session Variables – by Session Id This report searches for “session.*” syslog entries. It then uses a regular expression to

parse the session variables and the session variable value. Then it charts the count by

session ID and sorts by count.

Actual search command: search = sourcetype="apm_log" session.* | rex "(?<sess_var>session\..\\S+) is

(?<sess_var_val>.*\\Z)" | chart count by session_id | sort – count

Session Duration – Report This report searches for “New session” or “Session deleted” syslog entries. It tracks

transactions on session ID because they start with “New session” and end with “Session

deleted”. It calculates the concurrency weight based on the session duration. It then sorts

by duration, joins session_id by performing a subsearch for username. Then it tabulates the

username and duration.

Actual search command: search = sourcetype="apm_log" ((New session) OR (Session deleted)) | transaction session_id

startswith="New session" endswith="Session deleted" | concurrency duration=duration | sort -

duration | eval str_duration=tostring(duration, "duration") | join session_id [search

sourcetype="apm_log" Username | rex field=_raw "Username '(?<username>.\\S+)'"] | table

username, str_duration

Note: The pie chart only displays in this report if you change “table username, str_duration”

to “table username, duration” in the search command.

Active Session – Report This report searches for “New session” or “Session deleted” syslog entries. It tracks

transactions on session ID to start with “New session,” end with “Session deleted,” and

keep, the evicted entries. It performs a subsequent search for event count equals “1”

transactions. Then it filters out the “Session deleted” transactions leaving only the open and

active “New session” sessions.

Actual search command: search = sourcetype="apm_log" ((New session) OR (Session deleted)) | transaction session_id

startswith="New session" endswith="Session deleted" keepevicted=t | search eventcount="1" NOT

deleted

Max Concurrent Sessions – Report This report searches for “New session” or “Session deleted” syslog entries. It tracks

transactions on session ID to start with “New session,” end with “Session deleted,” and

keeps the evicted entries. It then calculates the concurrency weight based on the session

duration. It charts the maximum concurrent sessions over time.

Actual search command: search = sourcetype="apm_log" ((New session) OR (Session deleted)) | transaction session_id

startswith="New session" endswith="Session deleted" keepevicted=t| concurrency duration=duration |

eval duration=tostring(duration, "duration") | fields session_id, duration, concurrency | chart

max(concurrency) over _time

Session Throughput – Report

This report searches for “statistics” syslog entries and filter out empty statistics. It then uses

a regular expression to parse bytes in and bytes out values. Then it tabulates bytes in and

bytes out over time.

Actual search command: search = sourcetype="apm_log" statistics NOT "0," | rex field=_raw "bytes in: (?<bytes_in>.\\d+),

bytes out: (?<bytes_out>.\\d+)" | table _time, bytes_in, bytes_out | sort + time

Access Type – Report This report searches for “Access policy result:” syslog entries. It then uses a regular

expression to parse the access types. It then charts the count by access type.

Actual search command: search = sourcetype="apm_log" "Access policy result:" | rex field=_raw "Access policy result:

(?<access_type>.*$)" | chart count by access_type

Top Users by login – Report

This report searches for “Username” syslog entries. It then uses a regular expression to

pick up the login name. It performs a statistics count by login name and sorts by count.

Actual search command: search = sourcetype="apm_log" Username | rex field=_raw "Username '(?<login_name>.\\S+)'" | stats

count by login_name | sort – count

Note: You can also combine two search reports into one. For example, to create a Top

Users by login and Access Type – Report, two search commands can be joined using the

same session ID to create the report. Here is how: sourcetype="apm_log" "Access policy result:" | rex field=_raw "Access policy result: (?<access_type>.*$)" | join session_id [search sourcetype="apm_log" Username | rex field=_raw "Username '(?<login_name>.\\S+)'"] | chart count over login_name by access_type | sort – count

This creates the following report.

You can further refine the report so that it shows only BIG-IP Local Traffic Manager and

Access Policy Manager_Mode (also called ActiveSync mode) by replacing “chart count

over login_name by access_type | sort – count” with “where

Access_type=“LTM+APM_Mode” | chart count by login_name | sort – count.”

Top Users by Access Type – Report This is an advanced form search report. Please refer to

$SPLUNK/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/topusersbyaccess.xml

for detailed implementation information. The following search template is used to generate

the report: sourcetype="apm_log" "Access policy result: " | rex field=_raw "Access policy result: (?<access_type>.*$)" | join session_id [search sourcetype="apm_log" Username | rex field=_raw "Username '(?<login_name>.\S+)' "] | where access_type="$accessType$" | chart count by login_name | sort – count

is used to generate the report. It performs another search by access type ($accessType$)

and puts them into a dropdown list. sourcetype="apm_log" "Access policy result:" | rex field=_raw "Access policy result: (?<atype>.*$)" |

stats count by atype

Top Users by Throughput – Report This report searches for “Username” syslog entries and It then uses a regular expression to

extract the login name. It then joins login entries with session ID from the second search

pipeline to obtain bytes in bytes out statistic information. It charts max(throughput) over

login name and sorts by max(throughput).

Actual search command: search = sourcetype="apm_log" Username | rex field=_raw "Username '(?<username>.\\S+)'" | join

session_id [search sourcetype="apm_log" statistics NOT "0," | rex field=_raw "bytes in:

(?<bytes_in>.\\d+), bytes out: (?<bytes_out>.\\d+)" | eval throughput=bytes_in+bytes_out ] | chart

max(throughput) over username | sort - max(throughput)

Auth Success vs. Failure – Report This report searches for “Access policy result:” syslog entries, and It then uses a regular

expression to extract the access types. It then charts the count of the eval “Logon_Deny”

access type and other access types by host.

Actual search command: search = sourcetype="apm_log" "Access policy result:" | rex field=_raw "Access policy result:

(?<access_type>.*$)" | chart count(eval(access_type="Logon_Deny")) AS "Auth Failure",

count(eval(NOT access_type="Logon_Deny")) AS "Auth Success" by host

Access by User Agent – Report This report searches for “session.user.agent” and then uses a regular expression to extract

the session user agent string. It then statistically counts by user agent string and sorts by

count.

Actual search command: search = sourcetype="apm_log" session.user.agent | rex field=_raw "session.user.agent is

(?<user_agent>.*$)" | stats count by user_agent | sort – count

Client Type over Platform – Report This report searches for “session.client.browscap_info” syslog entries. It simply charts the

count over client type by client platform and sorts by count. Splunk searches automatically

to extract the client type and client platform key values.

Actual search command: search = sourcetype="apm_log" session.client.browscap_info | chart count over ctype by cplatform |

sort – count

The following report shows the stacked format of column diagram.

PPP tunnels by session id This report searches for “PPP tunnel started” or “PPP tunnel closed” syslog entries. It

extracts the tunnel ID. It tracks transactions on tunnel ID and session ID to start with

“started,” and end with “closed.” It then calculates the concurrency weight based on the

session duration, sorts by duration, and casts duration as a human-readable string. It also

tabulates time, session_id, tunnel_id, and duration.

Actual search command: search = sourcetype=apm_log ((PPP tunnel started) OR (PPP tunnel closed)) |rex field=_raw "PPP

tunnel (?<tunnel_id>0x[a-f0-9]+) "| transaction session_id, tunnel_id startswith="started"

endswith="closed"| concurrency duration=duration| sort - duration | eval

str_duration=tostring(duration,"duration")| table _time, session_id, tunnel_id, str_duration

PPP tunnels IP addresses per session This report searches for “PPP” and “IPv4” syslog entries. It extracts the tunnel IP address,

and sorts by time. It then tabulates the time, session ID, and tunnel IP address.

Actual search command: search = sourcetype=apm_log PPP IPv4| rex field=_raw "PPP IPv4: (?<tunnel_ip>[0-9]{1,3}\.[0-

9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})"|sort - _time|table _time, session_id, tunnel_ip

Access by IP – Report This report searches for “New session” syslog entries. It then extracts the client IP address.

Then it charts the count by client IP address and displays the top 20.

Actual search command: search = sourcetype=apm_log New session | rex "client IP (?<client_ip>[0-9]{1,3}\.[0-9]{1,3}\.[0-

9]{1,3}\.[0-9]{1,3})"|chart count by client_ip | sort limit=20 – count

Session policy steps This is an advanced view search report. Please refer to

$SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/session_policy.xml for

detailed implementation information. It contains two subreports: User sessions and Events for

session.

Session variables by session id This is an advanced view search report. Please refer to

$SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/session_vars.xml for

detailed implementation information. This report allows you to display all session variables by

chosen session ID.

© 2014 F5 Networks, Inc. All rights reserved. F5, F5 Networks, the F5 logo, BIG-IP, FirePass, iControl, TMOS, and VIPRION are trademarks or registered trademarks of

F5 Networks, Inc. in the U.S. and in certain other countries.