Rapid Data Modeling and Testing

AARON BENTON

community.couchbase.com

+

Couchbase Server Couchbase Mobile

Use Cases

• Mobile

• Caching

• User Tracking

• Media

• Short Urls

• Products

• Session Management

• How is data represented?

• How are models

communicated?

• How are models

documented?

• How are data types

defined?

First, solve the problem. Then, write the code.“ ”- John Johnson

Data Modeling

Conceptual Data Modeling

Entity Names

Entity Relationships

Users Products

Line Items

Orders

Logical Data Modeling

Entity Names

Entity Attributes

Entity Relationships

UsersUser ID (PK)

Username

Password

First Name

Last Name

Address

City

State

Zip Code

Country

Email

Phone

OrdersOrder ID (PK)

Order Number

User ID (FK)

Order Date

Address

City

State

Zip Code

Country

Phone

Email

Credit Card

Expiration Date

Order Line

ItemsOrder ID (FK)

Product ID (FK)

Quantity

Total

Products

Product ID (PK)

Product Name

Description

Price

Physical Data ModelingEntity Names -> Table NamesEntity Attributes -> Field NamesEntity Relationships -> KeysPrimary / Foreign KeysData Types

NoSql Modeling

• How do you represent a model?

• What tools do you use?

• How do you show relationships?

• Does code define the model?

• What key patterns should be used?

JSON

JSON Schema

YAML

Now What?

START DEVELOPMENT!

FAKE DATA

WASTED TIME

Fake Data Generators• FakerJS

• ChanceJS

• json-generator.com

• objgen.com/json

• mockaroo.com

• jsonschema.net

Relationships are Hard

FakeIT

Fakeit

• CLI Tool inspired by Swagger

• Models defined in YAML

• Generates JSON, CSON, YAML, CSV

• Exports to Couchbase, Sync Gateway, Zip,

Console

Fakeit Data

• Exposes FakerJS and ChanceJS

• Custom Build Functions w/ Pre and Post

• Multiple Models

• Dependencies

• Definitions

*fakeit is for structure, cbworkloadgen is for sizing

npm install fakeit -g

Original YAML Model

Simple Model

28

Data Attribute

Console Output

File Output

Output Formats

CSV Output

Zip Archive

Couchbase

What about Mobile…?

Sync Gateway

Channel Property

Sync Gateway with Auth

How does it Work?

Property Data:

value

• A static value to be used

• Executed at build time

fake

• A FakerJS template string

• Executed at build time

pre_build

• Function body used to initialize

a property

• Property is set to the returned

value

build

• Function body used to set a

property

• Property is set to the returned

value

post_build

• Function body used to

manipulate a property after it

has been set

• Property is set to the returned

value

pre_run

• Function body that is executed

before any models are

generated

post_run

• Function body that is executed

before after all models have

been generated

Flow

Model

Pre_RUNModel

PRE_BUILD

Initialize

Property

Value

(EACH)

Start

Document

Run

Property

PRE_BUILD

+ BUILD

(EACH)

PROPERTY

POST_BUIL

D (EACH)

MODEL

POST_BUIL

D

MODEL

POST_RUN

Flow

Flow

Multiple Models

Multiple Models

Dependencies

Users Products Reviews Orders

Dependencies

Dependencies

Dependencies with Single Output

Definitions

Definitions

continents.json countries.csv

Inputs

continents.json countries.csv

Inputs

Require

Require

Seeds

Seeds

JS API

JS API Options

How we use FakeIt• YAML models store in the repository

• Make commands to generate data

• Docker Compose:

• Couchbase

• Sync Gateway

• Testing / QA

Demos

Fakeit Examples• Contacts

• Ecommerce

• Flat

• Flight Data

• Music

• Simple

• Social

bit.ly/fakeit-examples

Questions?

HOW TO CONTACT MEDon’t hesitate to contact with me.

I will be happy to answer your questions.

TWITTER: @bentonam

github.com/bentonam

EMAIL: aaron@shop.com

Thank You!