best practices day 2 operations - linux foundation...
TRANSCRIPT
![Page 1: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/1.jpg)
Day 2 Operations Best PracticesJanet Yu, Software Engineer, SignalFx
Ben Lin, APAC Tech Lead, Mesosphere
![Page 2: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/2.jpg)
Agenda
• Overview• Architecture• Metrics API• Demo
![Page 3: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/3.jpg)
Continuously Connected World
Modern Enterprise Architecture
Mobile 4.4B Internet of Things (IoT) 6B
![Page 4: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/4.jpg)
App Transformation
Traditional Enterprise Apps
Monolithic packaged software (in VMs)
Big databases (e.g., Oracle, SQL Server)
App Data
Modern Enterprise Apps
Microservices (in containers)
Cloud native data services(e.g., Spark, Kafka, Cassandra)
![Page 5: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/5.jpg)
Data Intensive
EVENTS
Ubiquitous data streams from connected devices
INGEST STORE ANALYZE ACT
Ingest millions of events per second
Distributed & highly scalable database and file system
Real-time and batch process data
Visualize data and build data driven applications
Sensors
Devices
Clients
![Page 6: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/6.jpg)
Key Challenges
• Scalable Capacity• Dynamic Architecture• Load Balancing
![Page 7: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/7.jpg)
Scalable Capacity
Benefit:Nodes added or removed, based on load
Concern:When does it need to occur
![Page 8: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/8.jpg)
Dynamic Architecture
Benefit:One piece can be easily swapped out with another
Concern:Obtaining meaningful view of application as a whole when pieces can change
![Page 9: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/9.jpg)
Load Balancing
Benefit:Work is fairly shared among resources
Concern:How effective is the algorithm
![Page 10: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/10.jpg)
Mesos ArchitectureFramework A
Scheduler
MESOS MASTER QUORUM
LEADER STANDBY STANDBY
Framework B
Scheduler
Framework A
Executor
Task
Agent 1
Framework B
Executor
Task
Agent N...
ZK
ZKZK
![Page 11: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/11.jpg)
Metric Categories
# of unique users logged in the last hourWeek over week percentage growth in revenue
BUSINESS
• Latency• Availability/SLA
• CPU• Memory• Disk space
APPS - Internal or 3rd party services
INFRASTRUCTURE - Resources which apps rely on
• Logins & Usage
• Region• Profile
USERS
BUSINESS
![Page 12: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/12.jpg)
Metrics
Metric: Anything that is measurable and variableMeasurements captured to determine health and performance of
cluster:• How utilized is the cluster?• Are resources being optimally used?• Is the system performing better or worse over time?• Are there bottlenecks in the system?• What is the response time of applications?
![Page 13: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/13.jpg)
Mesos Metric Sources
● Mesos metrics ○ Resource, frameworks, masters, agents,
tasks, system, events ● Container Metrics
○ CPU, mem, disk, network● Application Metrics
○ QPS, latency, response time, hits, active users, errors
OS
Mesos
Container ContainerContainer
App App App
![Page 14: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/14.jpg)
Master Metrics
● Metrics for the master node are available at the following URL:○ http://<mesos-master-ip>/mesos/master/metrics/snapshot ○ The response is a JSON object that contains metrics names and values as key-value
pairs.
● Metric Groups:○ Resources○ Master○ System○ Slaves○ Frameworks○ Tasks○ Messages○ Event Queue○ Registrar
![Page 15: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/15.jpg)
Master Basic Alerts
Metric Value Inference
master/uptime_secs is low The master has restarted
master/uptime_secs < 60 for sustained periods of time The cluster has a flapping master node
master/tasks_lost is increasing rapidly Tasks in the cluster are disappearing. Possible causes include hardware failures, bugs in one of the frameworks or bugs in Mesos
master/slaves_active is low Slaves are having trouble connecting to the master
master/cpus_percent > 0.9 for sustained periods of time DCOS Cluster CPU utilization is close to capacity
master/mem_percent > 0.9 for sustained periods of time DCOS Cluster Memory utilization is close to capacity
master/disk_used & master/disk_percent DCOS Disk space consumed by Reservations
master/elected is 0 for sustained periods of time No Master is currently elected
![Page 16: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/16.jpg)
Agent Metrics
● Metrics for the agent node are available at the following URL:
http://<mesos-agent-ip>:5051/metrics/snapshot
○ The response is a JSON object that contains metrics names and values as key-value pairs.
● Metric groups:○ Resources○ Slave○ System○ Executors○ Tasks○ Messages
![Page 17: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/17.jpg)
Marathon Metrics
● Metrics for Marathon are available at the following URL:○ http://<marathon-ip>:8080/metrics○ for DC/OS http://<master-ip>:/marathon/metrics
● Redirect metrics to graphite when you start the Marathon process by adding the following flag: --reporter_graphite tcp://<graphite-server>:2003?prefix=marathon-test&interval=10
![Page 18: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/18.jpg)
Container Level Metrics
● Monitoring agent per container?○ Not scalable○ Increased footprint
OS
Container 1 Container 2
![Page 19: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/19.jpg)
Mesos Metrics Module
Simplified config○ Container metrics (automated)○ Application metrics (statsd env vars)
Context injection○ Automated source tagging (container, agents, …)
![Page 20: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/20.jpg)
Metrics API Architecture
![Page 21: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/21.jpg)
Metrics API
Poll for data about cluster, hosts, containers, applications
GET http://<cluster>/system/v1/agent
/<agent_id>/metrics/v0/<resource_path>
Accept: application/json
Authorization: token=<token_string>
![Page 22: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/22.jpg)
Metrics API Response"datapoints": [
{ "name": "processes", "value": 209, "unit": "", "timestamp": "2017-08-31T01:00:19Z" }, …
],
"dimensions": {
"mesos_id": "a29070cd-2583-4c1a-969a-3e07d77ee665-S0", "hostname": "10.0.2.255"
}
![Page 23: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/23.jpg)
Metrics API Tips• Get authentication token
POST http://<cluster>/acs/api/v1/auth/login {“username”: “<user>”, “password”: “<pw>”}
• Datapoint timestamp format may vary2017-09-01T00:25:23.502867353Z, 2017-09-01T06:25Z
• Error check datapoint value type{u'timestamp': u'2017-09-06T21:07:03Z',
u'unit': u'', u'name': u'org.apache.cassandra.metrics.Table.ReadLatency.system.peer_events.mean', u'value': u'NaN'}
![Page 24: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/24.jpg)
Datapoint
Single reported value of a metric from a particular source at a particular time• Metric name• Value• Timestamp• Metric type• Dimensions
![Page 25: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/25.jpg)
Metric Types
Counters
Discrete events that are monotonically increasing.
○ # of failed tasks○ # of agent registrations
Gauges
An instantaneous sample of some magnitude.
○ % of used memory in cluster ○ # of connected slaves
![Page 26: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/26.jpg)
Dimensions
• Key/value pairs• Set of dimensions represents the source
of a datapoint• Correlates related datapoints, patterns• Enables classification, aggregation,
filtering
![Page 27: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/27.jpg)
Metrics vs. Dimensions
![Page 28: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/28.jpg)
Metric + Dimensions = Time Series
![Page 29: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/29.jpg)
Tips for Sending Metrics
• Structure names hierarchically• Use a single, consistent delimiter for
wildcard searches• Separate dimensions from metric names• Don’t use dimensions with high cardinality
– Timestamps, task ids• Don’t send metric type as a dimension
– Gauges average, counters summed
![Page 30: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/30.jpg)
Monitoring
Send data to monitoring app for analysis
POST https://ingest.signalfx.com
Content-Type: application/json
X-SF-TOKEN: <token_string>{ “gauges”: [{
“metric”: “processes”,
“dimensions”: { “host”: “10.0.2.255”, ...},
“value”: 209}, ...}], ...}
![Page 31: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/31.jpg)
DEMO
![Page 32: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/32.jpg)
Key Takeaways
• Scalable Capacity– Collect system and custom metrics, find
outliers that might be bottlenecks• Dynamic Architecture
– Use dimensions common across all related pieces vs. tracking per-instance identifier
• Load Balancing– Compare time series, calculate ratios
![Page 33: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/33.jpg)
Resources
Visit the SignalFx and Mesosphere booths :)
• http://mesos.apache.org/documentation/latest/monitoring/• https://mesosphere.github.io/marathon/docs/metrics.html• https://dcos.io/docs/1.9/metrics/metrics-api/• https://developers.signalfx.com/docs/signalfx-api-overview• https://github.com/signalfx/collectd-mesos
![Page 34: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/34.jpg)
BACKUP SLIDESBACKUP SLIDESBACKUP SLIDESBACKUP SLIDESBACKUP SLIDES
BACKUP SLIDES
![Page 35: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/35.jpg)
Logging
![Page 36: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/36.jpg)
Troubleshooting
![Page 37: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/37.jpg)
Infrastructure Outliers
![Page 38: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/38.jpg)
Service Health
![Page 39: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/39.jpg)
Problem Indicators
![Page 40: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/40.jpg)
Cluster Trends
![Page 41: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/41.jpg)
Filtering by Dimension
![Page 42: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/42.jpg)
Inputs / Outputs
Input: StatsD
● Text records: either one-per-packet or newline separated.
● Optional tagging
memory.usage_mb:5|g
frontend.query.latency_ms:46|g|#shard_id:6,section:frontpage
Pseudocode:
if (env[“STATSD_UDP_HOST”] and env[“STATSD_UDP_PORT”]) {
// 1. Open UDP socket to the endpoint // 2. Send StatsD-formatted metrics}
Output: Apache Avro
![Page 43: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/43.jpg)
Marathon App Performance
$ curl <leader.mesos>/marathon/v2/apps/sleep | jq .
○ Find the appId (sleep),“host”, and “id” (task ID) fields
"tasks": [ { "id": "sleep.cb536c16-c6cf-11e5-a84d-0a43d276f399", "host": "10.0.3.226", "ports": [ 10466 ], "startedAt": "2016-01-29T21:32:28.443Z", "stagedAt": "2016-01-29T21:32:27.644Z", "version": "2016-01-29T21:32:27.599Z", "slaveId": "caa0847c-3751-456f-a2fd-30feb7a1fda5-S1", "appId": "/sleep" } ]
![Page 44: Best Practices Day 2 Operations - Linux Foundation Eventsevents17.linuxfoundation.org/.../slides/MesosCon_NA17_PPT_Day2O… · Master Basic Alerts Metric Value Inference master/uptime_secs](https://reader034.vdocuments.site/reader034/viewer/2022042202/5ea35695cd6e776ae56b41a9/html5/thumbnails/44.jpg)
Marathon App Performance
Curl the Agent host and look for the Marathon Task ID from previous step$ curl http://<agent-internal-IP>:5051/monitor/statistics | jq .
{ "executor_id": "sleep.cb536c16-c6cf-11e5-a84d-0a43d276f399", "executor_name": "Command Executor (Task: sleep.cb536c16-c6cf-11e5-a84d-0a43d276f399) (Command: sh -c 'env && sleep...')", "framework_id": "caa0847c-3751-456f-a2fd-30feb7a1fda5-0000", "source": "sleep.cb536c16-c6cf-11e5-a84d-0a43d276f399", "statistics": { "cpus_limit": 0.2, "cpus_system_time_secs": 0, "cpus_user_time_secs": 0.01, "mem_limit_bytes": 50331648, "mem_rss_bytes": 200704 } }