Download - Amazon DynamoDB Lessen's Learned by Beginner
![Page 1: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/1.jpg)
Amazon DynamoDBLessen's Learned by Beginner
Correct me if anything wrong
JAWS-UG Aizu Tokuno
![Page 2: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/2.jpg)
About me
● TOKUNO, Hirokazu● IT department in factory● 4th years moved to IT at age of 39
– Studying hard to catch up● Little know about
– VBA, SQL, JavaScript, Java, Perl, Shell...● Like to not write code
– Pentaho ETL, Oracle APEX, Mojolicious
![Page 3: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/3.jpg)
Learning/Challenge in 2013
● Unicage Development Method● Perl, Mojolicious● Jenkins● Automate Test jUnit, Mockito, shUnit2● Twitter Bootstrap, Google Maps API etc.● Fun to attend workshop/seminar!
– efsta!– Open Data Cafe Aizu– JAWS-US Aizu
● Join to CODE for AIZU
![Page 4: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/4.jpg)
Learning/Challenge in 2014
● Enroll to Linked Open Data Challenge 2013● Attend Hack for Town in Aizu● Meteor(Node.js, MongoDB) on Nitrous.IO● GitHub Debut● Amazon DynamoDB Right now!
![Page 5: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/5.jpg)
No experience in Could
![Page 6: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/6.jpg)
Created AWS account just 7 days ago
![Page 7: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/7.jpg)
What's DynamoDB?
● Fully Managed NoSQL Database Service
![Page 8: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/8.jpg)
What's DynamoDB?
● Fully Managed NoSQL Database Service
What's the hell are you talking about?
![Page 9: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/9.jpg)
What's DynamoDB?
● Fully Managed NoSQL Database Service
– NoSQL– Fully Managed Database Service
Split into peaces
![Page 10: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/10.jpg)
Understanding of NoSQL
● Not only SQL● Fast to read from huge amount of data
– Accept inconsistent● Flexible Data Structure (DynamoDB etc.)
– Attributes can be differ for each record● Not suitable for summary and query
![Page 11: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/11.jpg)
Fully Managed DB Service
● Rent database service as server
– Pay for read/write speed, storage, transaction● Managed by Amazon
– Install, Upgrade and Patching– Keep storage space, no limit!– Distribute 3 data centers– Improve performance
![Page 12: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/12.jpg)
Sum up!
● Ask Amazon for boring tasks!● Extreme performance with SSD!● Money solve the issue!● Not to think about storage limitation!● High reliability by distributing at 3 data center!● Be careful! It's not traditional database.
![Page 13: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/13.jpg)
Let's try!
● Service - DynamoDB
![Page 14: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/14.jpg)
Create Table
● Create Table
![Page 15: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/15.jpg)
Configure Table
● Primary Key
– Only Hash or Hash/Range combination
![Page 16: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/16.jpg)
Primary Key
● Hash Key : Column for partitioning
– Pick not converged column like UserID– Do not use picky column like Game ID
● Range Key :
– Column for search● Push into 2 columns to be unique
![Page 17: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/17.jpg)
Example of Primary Keys
● Present forum in 3 tables
Table Primary Key AttributesHash Range
Group JAWS-UG { LastPostBy="Kurihara" LastPostDateTime= "2012-01-03T00:40:57.165Z"}
Thread JAWS-UG Session { Message = "How about unconference?" Tags = [ "JAWS-UG", "Aizu" ] LastPostDateTime = "2012-01-03T00:40:57.165Z" }
Reply JAWS-UG#Session
2011-12-11T00:40:57.165Z
{ Message = "Extreeeeeeme!" PostedBy = "Kurihara" }
![Page 18: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/18.jpg)
Attributes
● Contents other than Hash, Range● Each entry may have own attributes ● Data Type
– String– Number– Binary– String Set (Array)– Number Set (Array)– Binary Set (Array)
Convert Date to string
![Page 19: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/19.jpg)
Create Index
● May skip since optional
![Page 20: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/20.jpg)
Index
● Upto 5 indexes can be attached for each table
– Hash-Range table is created under the table– Cost for read/write and storage
● Able to create at table creation. No edit, add.● Local secondary index
– Create Hash-Range table with shared Hash Key as table, Range Key is not
● Global secondary index
– Create Hash-Range table with non shared Hash and Range
![Page 21: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/21.jpg)
Index
● Sounds working like this...
Hash Range Attribute1 Attribute2
A 1 abc def
B 2 ghi jkl
Hash Attribute2 Attribute1 Range
A def abc 1
B jkl ghi 2
Primary Key
Primary Key
Index
![Page 22: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/22.jpg)
Throughput
● Read Capacity Unit
– Able to read one item per second for 4KB● Write Capacity Unit
– Able to write one item per second for 1KB
![Page 23: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/23.jpg)
Alarm
● Send notification if exceed specified threathold, 75~95%
![Page 24: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/24.jpg)
Final confirmation
● As usual
¥60/month!
![Page 25: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/25.jpg)
Complete!
● 3 tables for forum
![Page 26: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/26.jpg)
Disadvantages
● Unable to modify Index
– Need to re-build if need to modify– Requires cost to migrate data
● (Possibly) No system to sync with doc like, ERD● Suitable application and not suitable
– Need to scan all records to query by othet than Primary Key, Index
– Requires RDB for consistent transaction or use Java Transaction Library
– Complex with RDB as recommended
![Page 27: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/27.jpg)
Access from programs!
● Official SDKs● Unofficial SDK for Perl etc.
Android Browser iOS
Java .NET Node.js
Python PHP Ruby
![Page 28: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/28.jpg)
SDK API
● Table
– CreateTable– DeleteTable– DescribeTable– ListTables– UpdateTable
● Batch processing
– BatchGetItem– BatchWriteItem
● Item– PutItem– GetItem– UpdateItem– DeleteItem
● Search– Query : Primary
or Index– Scan : all record
![Page 29: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/29.jpg)
Let's play with DynamoDB!
![Page 30: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/30.jpg)
NoSQL is for Big Data
![Page 31: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/31.jpg)
Big Data for Twitter
![Page 32: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/32.jpg)
Make clearwhat's interesting!
![Page 33: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/33.jpg)
What is to be investigated...
![Page 34: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/34.jpg)
Voluntary Restraints!!
![Page 35: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/35.jpg)
Okay, okay, let's try...
![Page 36: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/36.jpg)
Forum with WebSock
![Page 37: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/37.jpg)
Too many samples available...Not challenging enough
![Page 38: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/38.jpg)
Create RESTfull API!
![Page 39: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/39.jpg)
Like this
DynamoDBNitrous.IOWeb Client
● All free of charge
![Page 40: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/40.jpg)
RESTfull API
● Implemented very simple function
– No BatchGetItem/BatchWriteItem
Action Function Method Example of URLCreate putItem POST /Group/Hash?p=1&q=A
Read scangetItem
GET /Groups/Groups/Hash
Update updateItem PUT /Group/Hash?p=A&q=1
Delete deleteItem DELETE /Group/Hash
![Page 41: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/41.jpg)
Create by POST
● curl -i -X POST -H "Content-Type: application/json" -d '{"datetime":"2013-03-13T23:27:32.256Z" , "by":"Tokuno"}' http://your_host:8080/Group/JAWS-UG
![Page 42: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/42.jpg)
Read all by GET
● curl -H "Content-Type: application/json" http://your_host:8080/Groups
![Page 43: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/43.jpg)
Read by GET
● curl -H "Content-Type: application/json" http://your_host:8080/Group/JAWS-UG
May need URI encode for Japanese
![Page 44: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/44.jpg)
Update by PUT
● curl -i -X PUT -H "Content-Type: application/json" -d '{"datetime":"2021-03-14T00:04:01.951Z" , "by":"No one tell me..."}' http://your_host:8080/Group/JAWS-UG
![Page 45: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/45.jpg)
Delete by DELETE
● curl -i -X DELETE http://your_host:8080/Group/JAWS-UG
![Page 46: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/46.jpg)
One more thing
![Page 47: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/47.jpg)
![Page 48: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/48.jpg)
Like this
DynamoDBNitrous.IOPentaho ETL
● All free!
![Page 49: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/49.jpg)
Pentaho for data processing
● Limited by DynamoDB throughput● Pay more to boost
Group LastPostDateTime LastPostByあ 2013-03-13T23:27:32.256Z とくのうい 2013-03-13T23:27:32.256Z とくのうう 2013-03-13T23:27:32.256Z とくのうえ 2013-03-13T23:27:32.256Z とくのうお 2013-03-13T23:27:32.256Z とくのうか 2013-03-13T23:27:32.256Z とくのうき 2013-03-13T23:27:32.256Z とくのうく 2013-03-13T23:27:32.256Z とくのうけ 2013-03-13T23:27:32.256Z とくのうこ 2013-03-13T23:27:32.256Z とくのうあ1 2013-03-13T23:27:32.256Z とくのうい1 2013-03-13T23:27:32.256Z とくのうう1 2013-03-13T23:27:32.256Z とくのうえ1 2013-03-13T23:27:32.256Z とくのうお1 2013-03-13T23:27:32.256Z とくのうか1 2013-03-13T23:27:32.256Z とくのうき1 2013-03-13T23:27:32.256Z とくのう
![Page 50: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/50.jpg)
Summary
● Easy to maintaing since managed by Amazon!● NoSQL is not 'Relational' but application links
– Not easy to develop● Unable to add/modify Index● Not flexible to query. Need to query every 1MB● Want method to actual and document
● Only suitable for disposable application and enterprise whom has big data?
![Page 51: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/51.jpg)
Thank you!
![Page 52: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/52.jpg)
Timeline
● 2014.03.08 Create account● 2014.03.11 Attend Hack for Town
2014.03.11 Cross over in last 3 years● 2014.03.12 Presentation for Game Lab● 2014.03.13 Develop RESTful
2014.03.13 Play with Pentaho● 2014.03.14 Presentation, today
![Page 53: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/53.jpg)
Appendix
![Page 54: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/54.jpg)
Region
● Find region usedhttp://docs.aws.amazon.com/general/latest/gr/rande.html
![Page 55: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/55.jpg)
Procedure to find Credentialss
![Page 56: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/56.jpg)
IAM
![Page 57: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/57.jpg)
Create a New Group of Users
![Page 58: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/58.jpg)
Group
![Page 59: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/59.jpg)
Permissions
● Select Power User Access for full access
![Page 60: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/60.jpg)
Verify Policy
● Required to review although nonsense
![Page 61: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/61.jpg)
Create User
![Page 62: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/62.jpg)
User
![Page 63: Amazon DynamoDB Lessen's Learned by Beginner](https://reader033.vdocuments.site/reader033/viewer/2022042813/54b778d64a795938168b4573/html5/thumbnails/63.jpg)
Credentials
● Finally