AWS COST VISUALIZERHDE GIP - MTS JUNE 2014

Presented by Iskandar Setiadi & Saskya Irena

AWS COST VISUALIZEROVERVIEW

1. Problem Description2. Analysis & Implementation3. Result4. Demonstration

ENORMOUS NUMBER OF PRODUCTS

UNDETAILED BILLING REPORT (UI)

COMPLICATED BILLING REPORT

CSV DETAILS

InvoiceID,PayerAccountId,LinkedAccountId,RecordType,RecordId,ProductName,RateId,SubscriptionId,PricingPlanId

"Estimated" ,"LineItem","21610877191184803585998665","AmazonElastic

"Estimated" ,"LineItem","21942771010558985838334611","AmazonElastic

BUT THERE ARE 8.000.000 (八百万)ENTRIES PER MONTH!

HOW MUCH IS THE EXPENSE FOR:

1. Each Customer2. Each S3 Bucket3. Each Infrastructure4. Each Project (Tapirus, Nobita)5. Other Expenses

- DAILY / MONTHLY -

- INSTANCE, ETC -

SOLUTION

CREATE YOUR OWN TOOL!

IMPLEMENTATIONENVIRONMENT

1. Python 2.6.92. Flask Framework & Redis for Database3. Linux/UNIX based Operating System4. Amazon EC2 - t1.micro -> c3.large5. Github (HDE/aws-cost-visualizer)

FIELDS OF DATA (CUSTOMER / CI /PROJECT)

1. Company Name2. EC2 Instance Reserved3. EC2 Instance Non-Reserved4. EC2 EBS Standard5. EC2 EBS PIOPS6. EC2 Data Transfer7. EC2 Others8. S3 Usage Rate *9. S3 **

10. Others

*= CUSTOMER ONLY

** = CI & PROJECT ONLY

FIELDS OF DATA (S3)

1. Bucket Name2. Total Cost

FIELDS OF DATA (NON-CUSTOMER)

1. Product Name (SQS, etc)2. Total Cost

PARSING CONVENTION

EC2 Instance Reserved: BoxUsage, HeavyUsage

EC2 Instance Non-Reserved: BoxUsage, SpotUsage

EC2 EBS Standard: VolumeUsage, VolumeIOUsage,SnapshotUsage

EC2 EBS PIOPS: VPIOPS, VolumeUsage

EC2 Data Transfer: Regional, In-Bytes, Out-Bytes

Others: Idle EIP, LoadBalancerUsage, Monitoring (Alarm,etc), DataProcessing, etc

26 DAYS STATISTICS

7.220.000++ LINES

1147 SECONDS(19 MINUTES 3 SECONDS)

KEY IDEAS

CHECK THE LATEST 2 MONTHS REPORT

RUN IT ONCE / TWICE PER DAY (11 P.M. / 5 A.M.)

USE SCRIPT / CRON TO HANDLE FAILURE [PRODUCTION]

IF AWS MONTHLY REPORT IS GETTING BIGGER (5 TIMES) -> MIGRATE TO BETTER EC2 INSTANCE

NAMING CONVENTIONS / RESERVEDKEYWORDS

THERE'S NO UNDERSCORE '_' ON S3 BUCKET NAME

PLEASE APPEND KEYWORD LIST AS BRIEF AS POSSIBLE

ASSUMPTIONS

All conventions in AWS Billing Report, Usage Report, andCompany List are constant

But the nature of production environment is alwayschanging, so it's needed to maintain the application

If there's no specified date in AWS Billing Report, thisapplication will map it to 1st date

DEMONSTRATION TIME!

ANY QUESTION?THANKS FOR YOUR ATTENTION!

ありがとう ございました