aws databases - devopsschool.com · concepts wilson, joe paperback ... aws managed database...

AWS Databases

1www.scmGalaxy.com

www.scmGalaxy.com

SQL and NoSQL Databases

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

3

SQL NoSQL

Data Storage Rows and Columns Key-Value

Schemas Fixed Dynamic

Querying Using SQL Focused on collection of

documents

Scalability Vertical Horizontal

ISBN Title Author Format

9182932465265 Cloud Computing

Concepts

Wilson,

Joe

Paperback

3142536475869 The Database

Guru

Gomez,

Maria

eBook

SQL NoSQL{

ISBN: 9182932465265,Title: “Cloud Computing Concepts”, Author: “Wilson, Joe”,Format: “Paperback”

}

Data Storage Considerations


4

• No one size fits all.

• Analyze your data requirements by considering:

• Data formats

• Data size

• Query frequency

• Data access speed

• Data retention period

AWS Managed Database Services

Compute Storage

AWS Global Infrastructure

Database

App Services

Deployment and Administration

Networking

Amazon

DynamoDB

Amazon

ElastiCache

AmazonRDS

AmazonRedshift

AWS Database Migration

Service


5

www.scmGalaxy.com

In - MemoryElastic Cache4

Datebase warehouseAmazon Redshift

www.scmGalaxy.com

Amazon Relational Database Service (RDS)

• Cost-efficient and resizable capacity

• Manages time-consuming database

administration tasks

• Access to the full capabilities of Amazon

Aurora, MySQL, MariaDB, Microsoft SQL

Server, Oracle, and PostgreSQL

databases

Amazon

RDS


9

Amazon RDS

• Simple and fast to deploy

• Manages common database administrative

tasks

• Compatible with your applications

• Fast, predictable performance

• Simple and fast to scale

• Secure

• Cost-effective


10

DB Instances

• DB Instances are the basic building blocks of

Amazon RDS.

• They are an isolated database environment in the

cloud.

• They can contain multiple user-created

databases.


11

How Amazon RDS Backups Work

• Restore your database to

a point in time.

• Are enabled by default.

• Let you choose a

retention period up to 35

days.

Automatic Backups: Manual Snapshots:

• Let you build a new

database instance from

a snapshot.

• Are initiated by the user.

• Persist until the user

deletes

them.

• Are stored in Amazon S3.


12

Cross-Region Snapshots

• Are a copy of a

database snapshot

stored in a different AWS

Region.

• Provide a backup for

disaster recovery.

• Can be used as a base

for migration to a

different region.


13

Amazon RDS Security

• Run your DB instance in an Amazon VPC.

• Use IAM policies to grant access to Amazon RDS resources.

• Use security groups.

• Use Secure Socket Layer (SSL) connections with DB instances (Amazon Aurora, Oracle, MySQL, MariaDB, PostgreSQL,Microsoft SQL Server).

• Use Amazon RDS encryption to secure your RDS instancesand snapshots at rest.

• Use network encryption and transparent data encryption (TDE)with Oracle DB and Microsoft SQL Server instances.

• Use the security features of your DB engine to control accessto your DB instance.


14

A Simple ApplicationArchitecture

Amazon RDS database

instance

Amazon EC2

Application Servers

Elastic Load Balancing

load balancer instance

DB snapshots in

Amazon S3


15

Multi-AZ RDS Deployment

• With Multi-AZ operation, your database is

synchronously replicated to another Availability

Zone in the same AWSRegion.

• Failover to the standby automatically occurs in case of

master database failure.

• Planned maintenance is applied first to standby

databases.


16

A Resilient, Durable ApplicationArchitecture

Amazon RDS database instances:

Master and Multi-AZ standby

Application, inAmazon

EC2 instances

Elastic Load Balancing

load balancer instance

DB snapshots in

Amazon S3


17

Amazon RDS Best Practices

• Monitor your memory, CPU, and storage usage.

• Use Multi-AZ deployments to automatically provision and maintain a synchronous standby in a different Availability Zone.

• Enable automatic backups.

• Set the backup window to occur during the daily low in WriteIOPS.

• To increase the I/O capacity of a DB instance:• Migrate to a DB instance class with high I/O capacity.

• Convert from standard storage to provisioned IOPS storage and use a DB instance class optimized for provisioned IOPS.

• Provision additional throughput capacity (if using provisioned IOPS storage).

• Test failover for your DB instance.


18

Amazon DynamoDB

• Allows you to store any amount of data withnolimits.

• Provides fast, predictable performance usingSSDs.

• Allows you to easily provision and change therequest capacity needed for each table.

• Is a fully managed, NoSQL database

service.

Amazon

DynamoDB


19

DynamoDB Data Model

Table:

MusicItems

Attributes (name-value pairs)

Artist


20

Song

Title

Album Year

TitleGenre

Primary Keys

Table: Music

Partition Key: Artist

Sort Key: Song Title

Partition Key

Sort Key

(DynamoDB maintains a sorted index for both keys)

Table:

Music

Artist


21

Song

Title

Album Year

TitleGenre

Provisioned Throughput

You specify how much provisioned throughput capacity

you need for reads and writes.

Amazon DynamoDB allocates the necessary machine

resources to meet your needs.


22

Supported Operations

• Query:

• Query a table using the partition key and an optional sort key filter.

• If the table has a secondary index, query using its key.

• It is the most efficient way to retrieve items from a table or

secondary index.

• Scan:• You can scan a table or secondary index.

• Scan reads every item – slower than querying.

• You can use conditional expressions in both Query and Scan operations.


23

Simple ApplicationArchitecture

Elastic Load

Balancing Amazon EC2

app instances

Clients

Amazon

DynamoDB

Business logic


24

Amazon RDS and Amazon DynamoDBFactors Relational (Amazon RDS) NoSQL (Amazon DynamoDB)

Application

Type

• Existing database apps

• Business process–centric apps

• New web-scale applications

• Large number of small writes and

reads

Application

Characteristics

• Relational data models,

transactions

• Complex queries, joins, and

updates

• Simple data models, transactions

• Range queries, simple updates

ScalingApplication or DBA–architected

(clustering, partitions, sharding)

Seamless, on-demand scaling based

on application requirements

QoS

• Performance–depends on data

model, indexing, query, and

storage optimization

• Reliability and availability

• Durability

• Performance–Automatically

optimized by the system

• Reliability and availability

• Durability


25

Database ConsiderationsIf You Need Consider Using

A relational database

service with minimal

administration

Amazon RDS

• Choice of Amazon Aurora, MySQL, MariaDB, Microsoft

SQL Server, Oracle, or PostgreSQL database engines

• Scale compute and storage

• Multi-AZ availability

A fast, highly scalable

NoSQL database

service

Amazon DynamoDB

• Extremely fast performance

• Seamless scalability and reliability

• Low cost

A database you can

manage on your own

Your choice of AMIs on Amazon EC2

and Amazon EBS that provide scale compute and

storage, complete control over instances, and more.


26

Knowledge Check


27

Q: What are the basic building blocks of Amazon Relational Database Service (RDS)?

DB InstancesTrue or False: Amazon DynamoDB allows you to store any amount ofdata with no limits.

TrueTrue or False: Scan is the most efficient way to retrieve items from a DynamoDB table.

FalseQ: You are creating a resilient, durable application using Amazon RDS. In addition to Amazon RDS’s automatic backups, what feature should you use to ensure that your backups are durable retained?

Manual Snapshots

Understanding AWSRelational Database Services (RDS)

D a ta b a s e eng ine m a n a g e d b y A W S

M y S Q L , Oracle, Microsoft S Q L , P o s t g re S Q L , MariaDB, or A m a zo n Aurora

Mult i -AZ deployment options

O n - d em a n d a n d reserved instance pric ing

Magnetic, G P - S S D, or P I O P S

Oracle a n d Microsoft S Q L licensing:

- Included Licenses

- Bring your own licensing

A u to m ated or manual b a c ku p s

RDSCharacteristics

Continuously tracks c h a n g e s a n d b a c ks u p your D B

V olu m e snapshot of your entire D B instance, not just D B s

O n e d a y of b a cku p s retained b y default but ca n b e conf igured u p to 35 d a ys

B a c k u p retention per iod def ined dur ing configuration

W h e n you delete an R D S instance, all automated snapshots are deleted

- Manual snapshots are preserved

RDSA utomated

Backups

A u tom a ted b a cku p s occu r dai ly dur ing a 3 0 minute conf igurable b a cku p w in d ow

A u to m ate d b a c ku p s are p res e rve d for aconf igurable number of days (retentionperiod)

RDSA utomated

Backups

Y o u cannot restore from a D B snapshot to an ex ist ing D B instance

- A new DB instance is created when you restore

O n ly default D B parameters a n d secur ity g ro u p s are restored

- You must manually associateall other DB parameters and SGs

R D S co m b in e s da i ly b a c ku p s in conjunct ion with transaction lo g s to restore the D B Instance to any point dur ing the retention per iod

U p to the last f ive minutes

RDS Restore

Mult i -AZ R D S deployment d es ig n ed for H A

S yn ch ron ou s replica in s econ d a ry A Z

S t a n d b y replica R D S instance is invisible

D B snapshots a lways taken

against standby instance

A W S automatica l ly adjusts D N S record when n eed ed

Mult i -AZ is different from a R D S read replica

Multi-A Z Failover

A v a ilab il it y Z o n e 1


Sy n c h ron ou s

S tand b y

Master

Master



Sy n c h ron ou s

S tand b y

A uto m atic D N S C h a n g e

Update B a c k u p Maintenance



Sy n c h ron ou s

S tand b y

Master

A s y n c h ron ou s

R e a d Re p l i c a

A v a i lab ilit y Z o n e 3

R e a d repl icas d es ig n ed for w o rk lo a d shar ing / off loading

C rea ted from a snapshot of the master instance

A s yn ch ro n o u s replication / Read-only connect ions

Rea d - o n ly disaster recovery

RDS Read Replicas

Reg io nL icen s e m o d el

D ep loyment typ eD B instance classD B eng ine

RDS Reserved Instances

Move between A z s in the s a m e Reg ion

A re available for Multi-AZ deployments

C a n b e appl ied to R e a d Re p l i ca s p ro v id e d the D B Instance c lass a n d Reg io n are the s a m e

RDS ReservedInstances

Demo

C reat in g an R D S instance

C h o o s e one of the A W S offered directory serv ices

Establ ish a trust relationship

W i n d o w s integrated authenticat ion onlyworks with a d o ma in created using A W Sdirectory serv ice

Alternatively, yo u ca n use S Q L authentication

Windows Integrated

A uthentication

R D S characteristics

Mult i -AZ failover

R D S read replicas

R D S reserved instances

Summary

Understanding AmazonDynamoDB and Redshift

Ful ly m a n a g ed , h ighly ava i lable a n d sca lable N o S Q L

Automatica l ly a n d synchronouslyreplicates data across three Availabil ity Z o n e s

S S D s a n d l imiting indexing on attributes provides h igh throughput a n d low latency

Elast iCache ca n b e u s ed in front of D y n a m o D B

- Offload high amounts of readsfor non- frequently changeddata

Amazon DynamoDB

Ideal for ex ist ing or n ew appl icat ions that need:

- Flexible NoSQL database with low read and write latencies

- Ability to scale storage and throughput up or down as needed without code changes or downtime

Amazon DynamoDB

Mobile A p p s

L i ve Vo t ing

Dig i ta l A D S e r v i ng

IoT

L o g Ing e stion

G a m ing

E -co m m erce Car ts

Web-sess io n Mana g em ent

Metadata S to r age for A m a z o n S 3 o bjects

A c c e s s Co ntro lfor We b - b a s e d Co ntent

L a rg e data with low I / O rateB L O B data

Jo in s and/or com p lex transactions

Prewritten appl icat ion tied to a traditional relational database

Non-ideal DynamoDB Scenarios

Management C o n s o le a n d A P I s

A m a zo n S 3

A m a zo n Data Pipeline

A m a zo n RedshiftA m a zo n Elast ic

Ma p Red u ce

DynamoDB Integration

Stores structured data in tables, in d ex ed b y a pr imary key

Ta b les are a col lect ion of i tems a n d Items are m a d e u p of attributes (columns)

Primary ke y ca n be:

- Single-attribute hashkey

- Composite hash-rangekey

Amazon DynamoDB

S ch em a- lessTr ig gers

C ro ss- reg io n replication

StreamsS econ d a ry indexes

DynamoDB Features

S ca n operationQ u ery operation

Two Ways to Search

O p e n - s o urc e in-memory c a c h in g eng ines

- Memcached

• Widely adopted memoryobject caching system

- Redis

• Popular open-source in-memorykey-value store

• Supports data structures suchas sorted sets and lists

Master / S lave replicat ion a n d Multi-AZ

- Can be used to achieve cross AZ redundancy

ElastiCache

Memcached vs. RedisMemcached Redis

C a c h e to off load D B √ √

Multithreaded performance

√ X

Horizontal scal ing √ X

Mult i -AZ X √

B a c k u p a nd restore X √

P u b /s u b funct ional ity X √

S ort ing a nd ranking X √

A d v a n c e d data t y p e s X √

Pers is tence X √

Performance at S ca le with A m a zo n Elast iCache:

https://d0.awsstat ic.com/whitepapers/performance -at-sca le-with-amazon-elast icache.pdf

Suggested Reading

https://d0.awsstatic.com/whitepapers/performance-at-scale-with-amazon-elasticache.pdf












F a s t a n d fully m a n a g e d petabyte-sca le relational data warehouse serv ice

A n a lyze all your data us ing your ex ist ing bus iness intell igence tools

H D D a n d S S D P latforms

Starts at $0.25/hour

S ca le to $ 1 , 0 0 0 / T B/ Ye a r

Amazon Redshift

L ea d er N o d e

- Simple SQL endpoint

- Stores metadata

- Optimizes query plan

- Coordinates query execution

C o m p u te N o d es

- Local columnar storage

- Parallel/distributed execution of all queries, loads, backups, restores,resizes

Amazon Redshift

A rchitecture

Internal V P C

Customer V P CS Q L C l ient/BI Tools

L e ad er Node

C o m p ute

Nod e

C o m p ute

Nod e

C o m p ute

Nod e

A m a z o n S 3 / D y n a m o D B

J D B C / O D B C

Inte g ration B a c k u p Restore

10 G B

Continuous/ incremental b a c ku p s

- Multiple copies withincluster

- Continuous and incremental backups to S3

- Continuous and incremental backups across regions

- Streaming restore

Backup and Fault Tolerance

Fault tolerance

- Disk failures

- Node failures

- Network failures

- Availability Zone/Region level disasters

Backup and Fault Tolerance

L o a d e n c ry p te d from S 3

S S L to secure data in transitA m a z o n V P C for network

isolation E n cryp t ion to secure

data at rest

A u d i t l o g g i n g a n d A W SCloudTrai l integration

S O C 1/2/3, P C I - D S S , F ed RA MP ,B A A

Security

A m a zo n D y n a m o D B

A m a zo n Redshift

Summary

aws databases - devopsschool.com · concepts wilson, joe paperback ... aws managed database...

Documents