api-first: an agile approach to data management · feasible now:an agile approach to design-limit...

4 June 2018© MARKLOGIC CORPORATION

ERIK HENNUMPrincipal Engineer, MarkLogic

@ehennum

API-First: An Agile Approach to Data Management

SLIDE: 2 4 June 2018© MARKLOGIC CORPORATION

Stress on the businessExcess inventory levelsCompetitive pressure on priceFraud levels risingRegulatory compliance challenges

Problems with data integration


“It’s an opportunity.”Now it’s Carla’s problem


Data First Design

2. MODEL 3. SERVICES1. DATA SOURCES 4. APPLICATIONS2. MODEL


A Model for Integration+ Customer in CRM, order, and support systems

+ Product in order and inventory systems

+ Staff in HR and support systems

+ …

= reconcile the semantics, vocabularies, structures, identity, properties, formats, …

Public domain https://commons.wikimedia.org/wiki/File:The_Thinker,_Auguste_Rodin.jpg


The Enterprise-Wide ModelThing

PhysicalThing NonPhysicalThing

Customer

EmployeeInRole

PriceMore workrequired

Agent ???


Questioning Data FirstIntegrate all of the data before any use?

No feedback or course correction?

Waste of effort on the unused data?

Transport data to clients for joins and grouping?

Transfer domain expertise to app developers?

Designing from implementation – not for use

2. MODEL 3. SERVICES1. DATA SOURCES 4. APPLICATIONS

?


API First Design

4. MODEL5. DATA SOURCES 3. DATA SERVICES 2. API SERVICES 1. APPLICATIONS1. APPLICATIONS


Identifying the First Application

value to the business

effort to implement

Replace static pricing with dynamic pricing


Design for Dynamic Pricing4. MODEL5. DATA SOURCES 3. DATA SERVICES 2. API SERVICES 1. APPLICATIONS

Provide areal-time price to a customer for a product

Take customer and product

identifiers and return the price

Look up the price factors

needed by the pricing engine

Integrate the price data

needed by the data service

Load the source data for the price model


Stack for the Price Dynamically Service

API Gateway

API service (Java)

Application

Data Service (SJS)

Partitioned service


Alternative Stacks

API Gateway

Java service

Application

Data serviceData service Data service

API Gateway

Java service

GraphQL

Node.js service

On

prem

ise


“Great progress.”It’s still Carla’s problem


Identifying the First Application

value to the business

effort to implement

Fraud detection^


Developer Roles in Partitioned Services

API service (Java)

Data Service (SJS)

Middle tier generalistJava

Database specialistJavaScript


Design for Check Fraud Service4. MODEL5. DATA SOURCES 3. DATA SERVICES 2. API SERVICES 1. APPLICATIONS

Check for fraudster before

purchase

Take customer identifier and total price and return approval

Match potential fraud signature

and write audit record

Integrate the account data

needed by the data service

Load the source data for

the account model


Iterative Expansion of the Solution

Data Source

Data Source

Data Source

Data Source

1. Identify the next application

2. Design the services &

model

3. Load and integrate the

data


Time to Value

DATA FIRST

API FIRST

firstlaunch

PRICE DYNAMICALLYLAUNCH

CHECK FRAUDLAUNCH

nextlaunches


Evolving the Model and APIs

harmonized customers v2

API Gateway

checkFraud v1

applications

priceDynamically v1

matchSignatures v1

lookupFactors v1.2

independent partitioned services

matchSignaturesOrLookupFactors v1

…


API First Summarized Feasible now: an agile approach to design

- Limit the scope to the minimum requirements for the next application

- Execute the business logic in the middle tier – the data logic close to the persisted data

- Success is iterative expansion of the set of APIs and the integrated model

Coming soon: easier rapid development of partitioned API services

- Declare a database function, generate the Java class, and write the function endpoint

Requested: your feedback

api-first: an agile approach to data management · feasible now:an agile approach to design-limit...

Documents