the future of apm and why it requires analytics everywhere!

ApplicationPerformanceManagement


Are the servers ok?


Average CPU Utilization


Average CPU Utilization


Database servers

Web servers

Per Server Load Average


East Coast

West Coast


Dimension ExamplesHost ecomm-ws-1.acmecorp.com

Cluster ecomm-web-server, ecomm-db, etc.

Region East Coast, West Coast, etc.

Application Storefront, Inventory Service, etc.

Instance Type Micro, Small, Large, etc.

Availability Zone east-coast-1, west-coast-2, etc.

Operating System Linux vs. Windows (with version)

Purpose Internal vs. External

Attributes

Is the software ok?


Application Response Times (ms)


Transaction Response Times (ms)


Other

Checkout

Login

Add to Cart

Checkout Transaction Breakdown (ms)


REST

Database Cach

eJava


Dimension ExamplesTransaction Name Checkout, Add To Cart, Login, etc.

Total Response Time (total time spent running the transaction)

Tier Response Time 152ms calling a REST service, 593ms calling the DB, etc.

Response Code 404 Not Found, 500 Server Error, etc.

Error Message NullPointerException

Transaction Type Page View, Ajax, Message Queue, etc.

HTTP Verb POST vs. GET

Browser Chrome, Safari, IE, Mobile or Desktop?

More attributes

Are the customers ok?


Customer Breakdown


REST

Database Cach

eJava

Mary buys 2 green widgets

Carlos buys 3 blue

widgets

Anita buys 1 green widget

Acme Corp. buys 10,000 purple

widgets

Customer Breakdown


REST

Database Cach

eJava


shipping to Boston

Carlos buys 3 blue widgets

shipping to Portland


shipping to London


widgets shipping to Dallas

Customer Breakdown


REST

Database Cach

eJava


shipping to Bostonusing her credit card

Carlos buys 3 blue widgets

shipping to Portlandusing his credit card


shipping to Londonusing a gift card balance


widgets shipping to Dallas

using a corporate P.O.

Even more attributes


Dimension ExamplesCustomer ID “Mary” or email address or 823562

Account Name Acme, Globex, Initech, Hooli, etc.

Customer Type Free, Trial, Paid, Premium, etc.

Item Quantity …

Item Details Color, Size, etc.

Cart Value $100 or S, M, L, XL

Shipping Address City, State, Zip Code, Country, etc.

Payment Mechanism Credit card type, gift card, PO, etc.


Virtually every business

is now asoftware business

Managing Software Requires Context



Relational DataApp Server

Customer Web ServerBrowser

23

Public Cloud

NoSQL data store

Private Cloud

API Micro Services

Private Cloud

Application Data

Business DataCustomer Experience

Data

Mobile / Apps / Browser

Managing Software has become a

Big Data Problem

620,448,401,733,239,439,360,000


Managing software requires LOSSLESS data


OR

Managing software requires DIMENSIONAL data


▪App service/transaction, time ▪Error Code▪Instance Type, Server,

Container▪Script, Error, Page, Location,

Server

• App service/transaction, Time

• Geography, Network• Browser, Action, Page,

Session• Device, Crash, Action,

Session

▪Customer ID, New/Existing, Type▪Product ID, Category Type, Content

Type▪Revenue, Cart Size, Traffic, Ad

Impressions

24 dimensions =620 sextillion

different combinations

BusinessSuccess

ApplicationPerformance

CustomerExperience

Confidential ©2008-15 New Relic, Inc. All rights reserved. FOR INTERNAL USE ONLY 27

Managing software requires data at SCALE

Time

Dat

aLossless business outcomes

Lossless customer experienc

e

Lossless performanc

e

Lossy customer

experience

Lossy app performanc

e


Managing software requires ITERATIVE exploration

Dive into

root of cause

Start here➔

Customers report bad experience

Which companies?

Which individuals?

Revenue impact?

How slow?

Bottlenecked at frontend, network, or backend?

Different by geography?

Different by device?Slow frontend assets?

Network bottlenecks?

Which backend app?

Which portion of backend code?

Database bottlenecks?

Backend error?

Dev


Managing software requires COMMON DATA

Eliminate silos for a collaborative, data-driven organization.

OpsProductSoftware

Data


Managing software requires software analytics

How are customers engaging?

Am I open for business

How is my business doing? Lossless data

With many dimensions

Can be iteratively explored

Common across teamsApplicationPerformanc

e

CustomerExperienc

e

BusinessSuccess

Secure, Elastic, Multi-tenant Cloud Solution


Secure, Elastic, Multi-tenant Cloud Solution

Analytics Everywhere!


Announcing… every Pro & Enterprise customer of APM, Browser or Mobile gets Insights with 8 Days of data retention

36

MAKING VIDEO ADS PERSONAL

Tony ChongVP Engineering

37

Who We are?

- Vungle's mission is to ensure no two users have the same video ad experience

- 6 Countries, 160+ Employees- Android and iOS SDK- Node.js backend- 1000+ EC2 instances in AWS- 2.5B requests per day

38

New Relic

- APM for application health- Initially used for 1 QA instance of our Dashboard tool,

moved to Prod fleet in a few weeks- Goto tool for Production issues

- Insights for traffic intelligence- What type of data do we have?- How many types of requests are we serving per day?

per month?- Is there something wrong?

39

APM

- Response Time- Deployments- Error Rate- Throughput- Transactions

41

Insights

- Custom Dashboards- NRQL- Different perspectives

42

Insights - Custom Dashboards

43

Insights - Custom Dashboards

44

Insights - NRQL

45

Insights - Different Perspectives

Jason DominaSr. Director, Technical Operations

All content copyright © CafePress. All rights reserved.4747

Who are we?• Created ‘Print On Demand’ in 1999• High volume retail site• Average 600,000 sessions daily• But peak at 4-5 million during the holidays

• If you’re interested/passionate about something, we’ve got a t-shirt, mug, etc… for it. Lot’s of them in fact.

• No, seriously, like millions of them.


CORGI-ZILLA


We care a lot about who’s viewing what


How Fast are we going? Where, Who, How?


Whoa, what happened?


Wait, there’s a difference between Mac and PC users?

the future of apm and why it requires analytics everywhere!

Data & Analytics