at

Who am I ?

Ops Lead @BlaBlaCar

@jbfavre

http://{blog,www}.jbfavre.org

Agenda

●BlaBlaCar ?

●Zabbix● (very quick) introduction● at BlaBlaCar

●python-protobix

●Jmx-zabbix

BlaBlaCarBlaBlaCar

BlaBlaCarBlaBlaCar

Zabbix

●Server / Proxy / Agent●3 ways to get items

● Agent● Agent (active)● Trappers

●SNMP●JMX

Zabbix

●Many other (very) cool features like

● Low Level Discovery● VmWare monitoring● Web Scenario

Zabbix

25k items

Biggest host>1k items

Zabbix at BlaBlaCar

●~ 200 hosts●~ 6.5k triggers

●75 templates

StandardizationStandardization

Standardization

As few zabbix-agent items as possibleZabbix-agent (active) triggers a script

Standardization

All probes written in PythonEasy to maintainWidely deployed on systems

Or JavaWhen accurate

Use LLD wherever possibleAvoid template duplication

Only 2 zabbix-agent (active) itemsTriggers script to update itemsTriggers script for LLD

python-protobixpython-protobix

python-protobix

At firstA personal project to learn Python

What it is nowUnified way of dealing with trappers

https://github.com/jbfavre/python-protobix

(Soon on pypi.python.org)

python-protobix

Support LLD#!/usr/bin/env python

import protobix

''' create DataContainer, providing data_type, zabbix server and port '''zbx_container = protobix.DataContainer('lld', 'localhost', 10051)

hostname='myhost'item='hardware.power_supply'value=[ { '{#SLOT}': 0, '{#PLUGGED}' : 1 }, { '{#SLOT}': 1, '{#PLUGGED}' : 0 },]zbx_container.add_item( hostname, item, value)

try: zbx_response = zbx_container.send(zbx_container)except protobix.SenderException: print 'Oups...'

python-protobix

As well as item's update#!/usr/bin/env python

import protobix

''' create DataContainer, providing data_type, zabbix server and port '''zbx_container = protobix.DataContainer('items', 'localhost', 10051)

hostname='myhost'item='hardware.power_supply[0,status]'value=1zbx_container.add_item( hostname, item, value)

try: zbx_response = zbx_container.send(zbx_container)except protobix.SenderException: print 'Oups...'

python-protobix : RabbitMQ

LLD for vhosts & queuesblacklist queues (support regex)

LLD also send limitsmessage numberreceive/consume ratio

Can override limits per queue

python-protobix : MariaDB

Low Level Discovery

for « plugins »Galera

for storage engineOnly support InnoDB & MyISAM for nowSpider coming...

for multi-replication

python-protobix

Plugin's list :

hadoopcouchbaseelasticsearchmemcachedmariadbpacemakerphp-fpmrabbitmqredissupervisordvarnish

Jmx-zabbixJmx-zabbix

Jmx-zabbix

●Because python is not enough :)

●Trapper's power brought to Java JMX

https://github.com/n0rad/jmx-zabbix

Jmx-zabbix

●Embedded inside a Java process● Internal Java daemons

●Aside any Java process (separate service)● Cassandra● Elasticsearch● …

Jmx-zabbix

Configure jmx-zabbix  :serverName: <hostname in Zabbix>pushIntervalSecond: 60inMemoryMaxQueueSize: 10

zabbix: host: <Zabbix server hostname or IP> port: 10051

jmx: url: service:jmx:rmi:///jndi/rmi://localhost:7199/jmxrmi username: zabbix password: zabbix timeoutSecond: 30[...]

Jmx-zabbix

Map JMX metrics with Zabbix ones(example with Cassandra)

metrics: cassandra.status.failure: org.apache.cassandra.net:type=FailureDetector cassandra.status.timeouts: org.apache.cassandra.net:type=MessagingService cassandra.db.storage: org.apache.cassandra.db:type=StorageProxy

And specify which values you want valuesCaptured: org.apache.cassandra.gms.FailureDetector: ["DownEndpointCount"] org.apache.cassandra.net.MessagingService: ["RecentTotalTimouts"] org.apache.cassandra.service.StorageProxy: ["RecentRangeLatencyMicros", \ "RecentReadLatencyMicros", "RecentWriteLatencyMicros"]

Questions ?

BlaBlaCar

We hire !https://careers.smartrecruiters.com/BlaBlaCar/