![Page 1: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/1.jpg)
![Page 2: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/2.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Wallet: Increasing performance with DynamoDB
A M Z 3 0 2
Jagan Cheboluraghava
Senior Software Dev Manager
Amazon, Payment Services
Artem Pliasunov
Senior Software Dev Engineer
Amazon, Payment Services
![Page 3: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/3.jpg)
Agenda
Overview of Amazon Wallet
Why did we move to Amazon DynamoDB?
How did we implement migration with zero downtime?
How did Wallet benefit by migrating to Amazon DynamoDB?
Q&A
![Page 4: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/4.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 5: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/5.jpg)
![Page 6: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/6.jpg)
Wallet enables customers to pay for their orders
![Page 7: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/7.jpg)
Wallet’s functionality
![Page 8: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/8.jpg)
Keeping up with exponential growth
• 5 billion requests daily
• 10 billion records
• Exponential growth year over year
![Page 9: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/9.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 10: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/10.jpg)
Wallet
service
Database
![Page 11: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/11.jpg)
Wallet
service
Database
![Page 12: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/12.jpg)
Wallet
service
Shard 2
Shard 1
![Page 13: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/13.jpg)
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
![Page 14: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/14.jpg)
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
![Page 15: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/15.jpg)
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
Hot
Warm
Warm
Hot
!
![Page 16: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/16.jpg)
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
![Page 17: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/17.jpg)
DB scaling
Vertically Horizontally
Shard 1A
Shard 1A
Shard 2A
Shard 1A SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
Shard NA SB 1-NA SB 2-NA
![Page 18: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/18.jpg)
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Region 1
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Region 2
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Region 3
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Region 4
!
!
!
![Page 19: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/19.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 20: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/20.jpg)
Goals
• Reduce the cost of operations
• Simplify and reduce the cost of scaling
![Page 21: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/21.jpg)
Requirements for migration
• Zero downtime
• Keep data integrity
• Improve performance
• Improve availability
• Allow launch of new features
![Page 22: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/22.jpg)
Database decoupling
❑ Database consumers
❑ Join queries
❑ Triggers
❑ Stored procedures
❑ Indexes
❑ Unique constraints
❑ Transactions
❑ Views
![Page 23: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/23.jpg)
Decoupling database consumers
Amazon
Redshift Business
analytics
Wallet
service
Wallet
crawler
Oracle
Amazon
Redshift
Business
analytics
Wallet
service
Wallet
crawler
Amazon
DynamoDB
Oracle
![Page 24: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/24.jpg)
Database decoupling
✓ Database consumers
❑ Join queries
❑ Triggers
❑ Stored procedures
❑ Indexes
❑ Unique constraints
❑ Transactions
❑ Views
Amazon
Redshift
Business
analytics
Wallet
service
Wallet
crawler
Amazon
DynamoDB
Oracle
![Page 25: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/25.jpg)
Breaking join query
Payment instruments
Customer ID
Payment instrument ID
Type
Token
…
External destinations
Customer ID
Payment instrument ID
Address ID
…
SELECT FROM external_destinations
JOIN payment_instruments
External destinations
Customer ID
Payment instrument ID
Address ID
…
Payment instruments
Customer ID
Payment instrument ID
…
QUERY external_destinations
QUERY payment_instruments
![Page 26: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/26.jpg)
Database decoupling
✓ Database consumers
✓ Join queries
❑ Triggers
❑ Stored procedures
❑ Indexes
❑ Unique constraints
❑ Transactions
❑ Views
External destinations
Customer ID
Payment instrument ID
Address ID
…
Payment instruments
Customer ID
Payment instrument ID
…
QUERY external_destinations
QUERY payment_instruments
![Page 27: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/27.jpg)
Database decoupling
✓ Database consumers
✓ Join queries
✓ Triggers
✓ Stored procedures
❑ Indexes
❑ Unique constraints
❑ Transactions
❑ Views
Stored procedures
• Moved to application side
• Set before create/update
![Page 28: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/28.jpg)
Database decoupling
✓ Database consumers
✓ Join queries
✓ Triggers
✓ Stored procedures
✓ Indexes
✓ Unique constraints
❑ Transactions
❑ Views
Payment instruments
IDX U Payment instrument ID
IDX Customer ID
…
Payment instruments
PK Customer ID
SK Payment instrument ID
…
GSI: PIID to customer ID
PK Payment instrument ID
SK Customer ID
…
![Page 29: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/29.jpg)
Database decoupling
✓ Database consumers
✓ Join queries
✓ Triggers
✓ Stored procedures
✓ Indexes
✓ Unique constraints
✓ Transactions
✓ Views
![Page 30: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/30.jpg)
![Page 31: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/31.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 32: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/32.jpg)
3-step migration
Dual read/write
Backfill Switch
![Page 33: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/33.jpg)
Wallet service
Business
Persistence
API
Oracle
![Page 34: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/34.jpg)
Business
Business object
dao: Dao
…
<<interface>>
DAO
Persistence
Oracle DAO
db: Oracle
…
Business
Business object
dao: Dao
…
<<interface>>
DAO
Persistence
Dynamo DAO
db: DynamoDB
…
Amazon DynamoDBOracle
Business
Business object
dao: Dao
…
<<interface>>
DAO
Persistence
Amazon DynamoDB
Multiple DB DAO
oracleDAO: DAO
dynamoDAO: DAO
…
Oracle DAO
db: Oracle
…
DynamoDB DAO
db: DynamoDB
…
Oracle
![Page 35: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/35.jpg)
Dual read/write
![Page 36: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/36.jpg)
Consistent results
Verification
Verification
Transaction consistency
Execution time
All records migrated
![Page 37: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/37.jpg)
Backfill
Records in time
Dual read/write
Backfill Continuous Backfill
![Page 38: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/38.jpg)
Using Wallet service to migrate itself
Data migration service or
data pipelineWallet service
![Page 39: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/39.jpg)
Backfill setup
Wallet
team
Backfill
control
interface
Wallet
service
Wallet service
clients
Hot
replica
Warm
replica
Wallet
backfill
service
![Page 40: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/40.jpg)
Full Backfill
![Page 41: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/41.jpg)
Continuous Backfill
![Page 42: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/42.jpg)
Migration takes time
• Expected: 1 billion records / 1 day requires ~ 11,600 TPS
• Expected: 5,000 TPS → ~2.3 days?
• Actual: TPS ~600 = ~19 days
![Page 43: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/43.jpg)
Getting ready to switch
✓ Dual read/write
✓ Backfill
✓ Switch
1. MATCH
2. DISCREPANT - no data found
3. INCOMPATIBLE - no data found
![Page 44: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/44.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 45: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/45.jpg)
Benefits
• Improved latency by 50%
• Increased request per second per host by 40%
• Reduced operations costs by 90%
![Page 46: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/46.jpg)
Benefits
• Autoscaling (including on-demand)
• Time to live (TTL)
• Point-in-time recovery
• On-demand backup and restore
• Global tables
• Increased number of supported global secondary indexes
![Page 47: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/47.jpg)
From Managing DB to Managed DB
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Wallet
service
Wallet
service
Amazon
DynamoDB
![Page 48: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/48.jpg)
Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 49: Amazon Wallet: Increasing performance with DynamoDB](https://reader030.vdocuments.site/reader030/viewer/2022020910/62008dff6270ed5738549d04/html5/thumbnails/49.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.