microservices cloud club 2015-02-26

practical microservices

Powering modern applicationsYour favorite code

Container optimized infrastructure

Your favorite tools

Our data center or yoursJoyent Public Cloud Joyent provides Container Infrastructure as a Service.We run our customer’s mission critical applications across thousands of containers in the Joyent Public Cloud.

Joyent SmartDataCenter Joyent SmartDataCenter is an open source on-premise container run-time environment used by some of the world’s most recognizable companies to run their mission critical applications.

Node.js enterprise support

As the corporate steward of Node.js and one of the largest-scale production users, Joyent is uniquely equipped to deliver the highest level of enterprise support for this dynamic runtime.

Best Practices

PerformanceAnalysis

Core FileAnalysis

Debugging Support

Critical IncidentSupport

⚠

Practicalmicroservices

Merging trends

12 factor app

Two pizza rule

Begrudging acceptance of some SOA patterns

Lightweight, OS-based virtualization

Infrastructure as code

Infrastructure Applications

Microservices: confusing?

from Jeppe Cramon

Microservices: a dirty word?

from @alvaro_sanchez

Joyent’sembraceof microservices

Our data center or yoursJoyent Public Cloud Joyent provides Container Infrastructure as a Service.We run our customer’s mission critical applications across thousands of containers in the Joyent Public Cloud.

Joyent SmartDataCenter Joyent SmartDataCenter is an open source on-premise container run-time environment used by some of the world’s most recognizable companies to run their mission critical applications.

SmartDataCenter 0Human-driven spreadsheets and Perl scripts

SmartDataCenter 0Human-driven spreadsheets and Perl scripts

• Message broker • Scheduler • State • Distributed,

single purpose services(Perl scripts)

SmartDataCenter 6.5• Two monolithic Ruby pieces:

• Machine API • Customer API

• Some edge pieces in Node.js

SmartDataCenter 7

Booter

AMQPbroker

PublicAPI

Customerportal

ZFS-based multi-tenant filesystem

Virtu

al N

IC

Virtu

al N

IC

VirtualSmartOS(OS virt.)

. . .

Virtu

al N

IC

Virtu

al N

ICLinuxGuest

(HW virt.)

. . .

Virtu

al N

IC

Virtu

al N

IC

WindowsGuest

(HW virt.)

. . .

Virtu

al N

IC

Virtu

al N

IC

Virtual OSor Machine

. . .

SmartOS kernel(network booted)

SmartOS kernel(flash booted)

Provisioner

Instrumenter

Heartbeater

DHCP/TFTP

AMQP

AMQP agents

Public HTTP

Head-node

Compute node Tens/hundreds per

head-node

. . .

SDC 7 core services

BinderDNS

Operatorportal

. . .

Firewall

SmartDataCenter 7 core services

Analyticsaggregator

Key/ValueService(Moray)

FirewallAPI

(FWAPI)

VirtualMachine

API(VMAPI)

DirectoryService(UFDS)

DesignationAPI

(DAPI)

WorkflowAPI

NetworkAPI

(NAPI)

Compute-Node API(CNAPI)

ImageAPI

Alerts &Monitoring

(Amon)

PackagingAPI

(PAPI)

ServiceAPI

(SAPI)

DHCP/TFTP

AMQP

DNS

Booter

AMQPbroker

Binder

PublicAPI

Customerportal

Public HTTP

Operatorportal

OperatorServices Manta

Other DCs

Note: Service interdependencies not shown for readability

Head-nodeOther core services

may be provisioned on compute nodes

SDC7 Core Services

25containers

By the numbersThe SmartDataCenter application uses 25 containers for its services. That’s without HA enabled!

Memory Count

128MB 2

256MB 2

512MB 1

768MB 2

1024MB 10

2048MB 3

4096MB 2

8192MB 3

Microservices frustrations1. Messaging 2. Interfacing 3. State 4. Discovery 5. Debugging

DiscoveryMoving from monolithic to microservices architecture means moving from tightly coupled to loosely federated system necessitates service discovery

We built Binder, a node.js-based DNS + ZooKepper system. It works, but this is a complicated space.

Thorny issues without clear solutions: • Upgrades • Rollbacks • Maintenance windows

StateStateless microservices are easy, but stateful services require more thought. Decentralization of services is good, but decentralized state is insane. Answer: statefulness is itself a service made up of a set of microservices: • Moray: key value store

• JSON over TCP messaging using node-fast • Manatee: automated fault monitoring for PostgreSQL

• Automatically spins up new Postgres instances as needed to maintain DB quorum

• PostgreSQL: reliable, performant

Messaging• The principles of AMQP are very attractive… • …but in practice, implementation and operational issues have

made message brokers a single point of failure • Still using AMQP for some broadcast messages • Now mostly using point-to-point HTTP for performance and

reliability

Interfacing• More interfaces = more problems interfacing • JSON’s loose structure multiplies the risk of interfacing

problems • We use JSON Schema to add rigor without sacrificing

agility • Interface versioning is required for sanity

• We use Restify for HTTP routing with built-in versioning

DebuggingSystems break, distributed systems have more breakable surface area. Our diagnostic tools are as critical as any other in the box: • DTrace profiling of node.js • Bunyan node.js logging facility with DTrace support • JS heap analysis from core files

• “core files are the ultimate REPL” Also see @dapsays’ Node 0.12 debugging improvements and Node Summit slides.

SmartDataCenter is openOn Github: https://github.com/joyent/sdc

Test it in VMware on your laptop or install on dedicated hardware. Try the Docker support preview!

Connect:

• Mail list: sdc-discuss@lists.smartdatacenter.org

• IRC: #smartos on Freenode

SDC is easy to installDeploy SmartDataCenter in hours, not days (or minutes, not months) and get to work building value faster

infrastructure for microservices

Infrastructurefor microservices?

Virtualizationvs. microservices?

Microservices memory utilization

CC BY ZeptoBars

Microservices memory utilization

1 GB 1 GB 1 GB 1 GB 1 GB 1 GB 1 GB 1 GB

Microservices memory utilizationG

uest

OS

Gue

st O

S

Gue

st O

S

Gue

st O

S

Gue

st O

S

Gue

st O

S

Gue

st O

S

Gue

st O

S

Serv

ice

1

Serv

ice

2

Serv

ice

3

Serv

ice

4

Serv

ice

5

Serv

ice

6

Serv

ice

7

Serv

ice

8

Hardware virtualization limits memory utilization and is inefficient for microservices architectures

75%efficiencyfor 1GB services

Butmost microservices

are smaller than

1GB

50%efficiencyfor 512MB services

Impracticalfor 256MB services

Impracticalfor 256MB services

Impossible

Microservices memory utilizationSe

rvic

e 1

Serv

ice

2

Serv

ice

3

Serv

ice

4

Serv

ice

5

Serv

ice

6

Serv

ice

7

Serv

ice

8

Serv

ice

9

Serv

ice

10

Serv

ice

11

Serv

ice

12

Serv

ice

13

Serv

ice

14

Serv

ice

15

Serv

ice

16

OS-based virtualization optimizes memory utilization for microservices

CPU schedulingThis is not about core/thread assignment vs. time slicing

CPU schedulingIt’s about competing guest OS kernels and stolen CPU ticks

CPU scheduling1. Under hardware virtualization, the guest OS is a

hard realtime app

2. Hard realtime apps freak out when they don’t get their expected CPU cycles

3. Oversubscribed CPU + hardware virtualization is a recipe for disaster

I/O latencyVirtualized I/O and conflicts among competing schedulers increase I/O latency, sapping performance

20xmore containers

Up to

than HVMs

20x more containers!256GB DRAM / 24 core / 48 thread 960 256MB containers

The Joyent difference• Bare metal container performance

Eliminate the hardware hypervisor tax

• Proven container security Run containers securely on bare metal in multi-tenant environments

• Simplified container networking Each container has its own IP(s) in a user-defined network (SDN)

• Simplified host management One API endpoint (including Docker API) fronts the whole data center

Security: solvedGeneral Challenge Buggy or malicious code can escape Linux containers, execute unexpected processes, and read or write files in harmful ways.

Joyent Solution Joyent’s container runtime was built for security isolation first, and we have nearly ten years of experience without incident. We are now extending this container runtime to work with native Linux binaries and Docker.

Joyent Value Multi-tenant deployments enable cloud-scale economics and density.

Networking: solvedGeneral Challenge: Connecting containers between hosts, or with sophisticated network topologies, is difficult and performs poorly.

Joyent Solution: Joyent’s software defined networking solution brings full network connectivity to each container (including a unique IP and wire-speed access) and VxLANs across containers.

Joyent Value: Lower costs of administration with simplified network management and by leveraging leveraging commodity hardware for networking.

Containers…you mean Docker?Preview access to our next generation Docker container service is launching “soon.”

Joyent’s infrastructure containers form the basis of our Docker service.

Are containers limited to Microservices?Containerization does not dictate application architecture.

Joyent’s container technology scales from single-process Docker containers to infrastructure containers supporting the full suite of services expected from a unix host.

Infrastructure containers can be used as a persistent, durable, and faster alternative to VMs.

…And, we can run KVM inside infrastructure containers to support full hardware virtualized environments necessary to host Windows and other OSs.

Thank you!

Tweet questions to@misterbisson

Remember Joyent for• Container tech that scales from fat apps that expect a full

host down to tiny single-process microservices Better efficiency performance advantages for all application

• Bare metal performance Eliminate the hardware hypervisor tax

• Proven container security Run containers securely on bare metal in multi-tenant environments

• Simplified container networking Each container has its own IP(s) in a user-defined network (SDN)

• Simplified host management One API endpoint (including Docker API) fronts the whole data center