Download - Metrics that talk on cloud using ganglia
Grid Toolkits
Metrics that talk on Cloud using Ganglia
How do we monitor performance on Megam Cloud
We'll Cover
Our Experience Using Ganglia
How does it work in our Platform
Chef cookbooks for metering and setup.
Dashboard integration
What is Ganglia
Scalable distributed monitoring system for high-performance computing systems.
Sends information about your cloud instance.
Can be used as a live cloud monitor.
Can be extended using python plugins
What have we accomplished ?
Oh Yeah -
Dash #2(AngularJs Client)Metrics API
gmon
gmetad
gmetad
gmetad
gmon
gmon
gmon
Would you like to setup one ?
Yes you can
(or) http://www.megam.co
Few facts on gmetad/gmond
gmetad can run standalone or along with gmond.
gmetad can be configured to collect metrics of gmond servers of same cluster or different cluster.
gmetad stores data at
/var/lib/ganglia/rrds/CLUSTER_NAME/GMOND_SERVER_NAME
GMOND_SERVER_NAME can be changed in gmond.conf
gmetad is the metrics collector
gmond is the metrics sender
Ok. Got it.
What is our setup
#1 gmetad : monitor1.megam.co#2 gmetad : montior2.megam.co
Several gmonds(Cloud Apps) pumping data to gmetad
What are Cloud Apps
Any app
for lang := range ProgLanguages {JavaScalaGo . meteor}
&& DB, Queue
Do you need Graphite
No
Why ?
Needs rrds formatted metric files
It copies rrds files from gmetad
Twice storage
Is this the only soln ? Eager to hear feedback.
Let us setup gmetad 3.3.8-1
Ubuntu(raring) : package is gmetad
Ubuntu(saucy) has 3.6.0
sudo apt-get install gmetad
sudo apt-get install ganglia-webfrontend (*optional)
We used Opscode cookbook to setup => Link
Configure gmetad
The above says megcluster collects metrics from .megam.co and .megam.coWhich is like monitoring a Java App in .megam.co
(or) Your favorite App in .megam.co
nano /etc/ganglia/gmetad.confdata_source megcluster .megam.co:8649 .megam.co:8649
gmetad - start/stop.
Start :sudo gmetad
Stop : good old killps -ef | grep gmetadsudo kill -9
Cool gmetad - monitor1.megam.co is running
gmond
Install ganglia-monitor-python package will be installed in a server which is to be monitored.
Package has methods to collect basic metrics(cpu...) using python scripts in /usr/lib/ganglia/
Extended by enabling additional python scripts at /usr/lib/ganglia/python_modules.
For an exhaustive list : https://github.com/ganglia/gmond_python_modules
Let us setup gmond 3.3.8-1
Ubuntu(raring) : package is ganglia-monitor-python
Ubuntu(saucy) has 3.6.0
sudo apt-get install ganglia-monitor-python
We used Opscode cookbook to setup => Link
Configure gmond
The above says megcluster collects metrics from .megam.co and .megam.coWhich is like monitoring a Java App in .megam.co
(or) Your favorite App in .megam.co
nano /etc/ganglia/gmond.conf
globals { override_hostname = .megam.co override_ip = 127.0.0.1}udp_send_channel { host = monitor1.megam.co port = 8649 ttl = 1 }cluster { name = "megcluster" owner = "unspecified" }
What did we configure ?
In the globals we say our monitoring Java App's host name .megam.co
We provide the UDP channel of the gmetad (monitor1.megam.co)
We need to specify the gmetad cluster (megcluster)
cluster attribute groups all gmond to a gmetd in our case megcluster.
gmond - start/stop.
Start :sudo gmond
Stop : good old killps -ef | grep gmondsudo kill -9
Cool gmond - .megam.co is running
&
pumping to monitor1.megam.co
We customized chef - cookbooks
How do we use the chef - cookbooks
Tweak cookbook for gmetad
https://github.com/indykish/chef-repo/tree/master/cookbooks/megam_ganglia
Attributes
default[:ganglia][:cluster_name] = "megcluster"default[:ganglia][:unicast] = truedefault[:ganglia][:hostname] = monitoring1.megam.co
Chef Run : gmetad
Run chef :
runlist 'recipe[megam_ganglia::gmetad]'
Tweak recipes for gmond
For Any AppDefault : installs and configures ganglia-monitor-python. It collects the basic meterings like cpu_usage, memory_usage etc.
Nginx : collects nginx status details.
Rabbit : collects rabbbitmq metrics.
Redis : collects redis metrics.
Riak : collects riak metrics.
Chef Run : gmond
To monitor an appinclude_recipe megam_ganglia
Nginx frontended apps include_recipe megam_ganglia::nginx
Rabbitmq apps include_recipe megam_ganglia::rabbit
Riak apps include_recipe megam_ganglia::riak
Redis apps include_recipe megam_ganglia::redis
Configure gmond
nano /etc/ganglia/gmond.conf
globals { daemonize = yes setuid = yes user = nobody debug_level = 0 max_udp_msg_len = 1472 mute = no deaf = no host_dmax = 86400 /*secs */ cleanup_threshold = 300 /*secs */ gexec = no send_metadata_interval = 30 override_hostname = ".megam.co" override_ip = 127.0.0.1} cluster {
name = "megcluster"
owner = "unspecified"
latlong = "unspecified"
url = "unspecified"
}
/* The host section describes attributes of the host, like the location */
host {
location = "unspecified"
}
/* Feel free to specify as many udp_send_channels as you like. Gmond
used to only support having a single channel */
udp_send_channel {
host = monitoring1.megam.co
port = 8649
ttl = 1
}
Verifying gmond data
Open up your browser http://gmond1.megam.co:8649
It will list the metrics of the gmond instance.
Not recommended for prod.
Sample gmond data (for redis server)
Checking gmeta data
Data files in the below dir increasesGet metrics data at /var/lib/ganglia/rrds/megamcluster/gmond1.megam.co
Sample gmetad data (for thomas.work.local)
Dash Integration in rails
Built on
We'll cover it detail in a separate slideshare.If you are hungry Code is the design :)
For questions on this area:[email protected]
References
Ganglia Wiki
megam chef-repo
Our Organization(Megam Systems)
Beta Launch of Megam Cloud (Polygot PaaS)
Our PaaS design => Link
Register http://www.megam.co for an invite
Twitter : @indykish
Screencast illustrating the Cloud API Servers working live
Thank you
for watching
2012-2013 Megam Systems
2012-2013 Megam Systems