"containers, devops, microservices and kafka: tools used by our monolith wrecking crew."...
TRANSCRIPT
Containers, DevOps, Microservices and Kafka: Tools used by our Monolith wrecking crew.
Jonathan Owens, Senior Site Reliability Engineer & Jose Fernandez, Lead Software Engineer
This document and the information herein (including any information that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission. Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,”, “expects” or words of similar import. Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings New Relic makes with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov. New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.
Confidential ©2008-15 New Relic, Inc. All rights reserved.
GOOD PROBLEM
Confidential ©2008-15 New Relic, Inc. All rights reserved.
6
Shopping list for replacing a Monolith
New Relic APM + Insights
Kafka
Microservices
DevOps
Containers
CollectCollectCollectCollectCollect
Collect
Collect
Collect
CollectCollect
Collect
Collect
Collect
Collect
Collect
Great job refactoring services this year!“– No Manager Ever
Updating last-seen timestamps for metrics
Metric ID resolution
Computing cluster agent metrics
Writing metric metadata
Writing summary records
Applying black / whitelist rules to accounts
Processing and storing environment values
Legacy alerting
Agent connect() calls
Agent run state
Orchestrating thread profiles
Applying rename rules to incoming metrics
Aggregation of 1-minute and 1-hour timeslices
from raw harvestsWriting 1-minute and 1-hour
timeslices to MySQL
Updating last-seen timestamps for metrics
Metric ID resolution
Computing cluster agent metrics
Writing metric metadata
Writing summary records
Applying black / whitelist rules to accounts
Processing and storing environment values
Agent connect() calls
Agent run state
Orchestrating thread profiles
Applying rename rules to incoming metrics
Legacy alertingAggregation of 1-minute
and 1-hour timeslices from raw harvests
Writing 1-minute and 1-hour timeslices to MySQL
Shopping list for replacing a Monolith
New Relic APM + Insights
Kafka
Microservices
DevOps
Containers
Collect ConsumeProcess
Process
Tap into the monolith and emit data for your use
collector minuteaggregator
houraggregator
hourwriter
houraggregator
collector minuteaggregator
houraggregator
hourwriter
houraggregator
Validate your data
MySQL
query
QueryService
“In the dark”
Insights
Cassandra
query, result, and performance
QueryValidator
MySQL
query
QueryService
“In the dark”
Insights
Cassandra
query, result, and performance
QueryValidator
Shopping lists for replacing a Monolith
New Relic APM + Insights
Kafka
Microservices
DevOps
Containers
collector minuteaggregator
hourwriter
collector minuteaggregator
houraggregator
hourwriter
houraggregator
collector minuteaggregator
houraggregator
hourwriter
houraggregator
minuteaggregator
summaryrecord
generator
Experimentally proven value
Service boundaries match communication boundaries
Photo credit: theimpulsivebuy
cc by Aranami http://www.flickr.com/photos/z0/
Shopping list for replacing a Monolith
New Relic APM + Insights
Kafka
Microservices
DevOps
Containers
Alice GoldfussSite Reliability Engineer New Relic @alicegoldfuss
DevOps means empathy and communication over the wall until there is no more wall.
JonathanOwens
JoseFernandez
OpsDev
… long wait… frustration due
to lack of visibility…
cyclerepeats
“I need 3 Cassandra servers
ASAP!!”(vague request)
“Here they are. Sorry it took so long.
Our queue was backed up.”
<different goals>
“Doh. I needed the
newer version of Cassandra…”
OpsDev
Team Culture
How did we do it? How can you do it?
* insert picture for Upscale event
Upscale
Product Reliability Engineer
Site Reliability Engineer
Shopping list for replacing a Monolith
New Relic APM + Insights
Kafka
Microservices
DevOps
Containers
Docker is opinionated about software architecture in a way that encourages more robustly crafted applications.
“– Karl Matthias & Sean P. Kane
Docker: Up and Running
PUSH and then
PULL door
Push plate?
Policy statement
Push on a pull handle
Visual instruction to use badge
(with light indicator)
PUSH PUSH PULL
This part fits the problem
This part fits the person
Start Small
Shopping list for replacing a Monolith
New Relic APM + Insights
Kafka
Microservices
DevOps
Containers
Q&A
Thank you.
Jonathan Owens, Senior Site Reliability Engineer & Jose Fernandez, Lead Software Engineer