devops with activemq, camel, fabric8, and hawtio

DevOps with ActiveMQ,

Camel, Fabric8 and HawtIO

Christian Posta 6/11/14

RED HAT | ADD NAME 2

Agenda

•  DevOps… What is that?

•  Enterprise Integration

•  Automated Delivery

•  To the Cloud


Your speaker Christian Posta

http://christianposta.com/blog

@christianposta

[email protected]

•  Principal Middleware Specialist

•  Based in Phoenix, AZ

•  Committer on Apache Camel, ActiveMQ, Apollo, PMC on ActiveMQ

•  Author: Essential Camel Components DZone Refcard

Shorten the lifecycle from inception to production so that the business can make money! Developers and Operations must cooperate. Rely on tools/frameworks to automate, automate, and automate.

DevOps


“WTF is DevOps?” •  IT is a core competency •  Set of principles •  There’s more to applications than coding! •  Feedback •  Repetition •  Communication •  People!


Shorten the feedback loop •  Developers •  QA •  Operations •  Business •  Customers!!!

Enterprise Integration


Integration is easy!


•  Off the shelf? Home Grown? Acquisition? •  Platforms •  Protocols / Data Formats •  Data Formats •  Timing •  Organizational mismatch

Why is integration hard?


•  Light-weight integration library •  Domain Specific Language •  Enterprise Integration Patterns •  Components •  Routing and Mediation (like an ESB?) •  Runs in any container (or stand alone)

What is Apache Camel?


•  Message Routing •  Transformation •  Aggregation •  Splitting •  Resequencer •  Routing Slip •  Enricher •  All 65 from the book!

Enterprise Integration Patterns


Components •  ActiveMQ, Websphere, Weblogic (JMS)

•  AMQP

•  ATOM feeds

•  AWS (S3, SQS, SNS, others)

•  Bean

•  Cache (EHCache)

•  CXF (JAX-WS, JAX-RS)

•  EJB

•  Drools

•  File

•  FTP/SFTP

•  Google App Engine

•  GMail

•  HTTP

•  IRC

•  jclouds

•  JDBC

•  Jetty

•  Twitter

•  MQTT

•  MyBatis

•  JPA

•  Spring Integration

•  Spring Web Services

http://camel.apache.org/components.html

To see list of all components!!


Java DSL

public class OrderProcessorRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { from(“activemq:orders”) .choice()

.when(header(“customer-rating”).isEqualTo(“gold”)) .to(“ibmmq:topic:specialCustomer”) .otherwise() .to(“ftp://user@host/orders/regularCustomers”) .end() .log(“received new order ${body.orderId}”) .to(“ibatis:storeOrder?statementType=Insert”); }}


Spring XML DSL

<route id=“processOrders”> <from uri=“activemq:orders”/>

<choice> <when> <simple>${header.customer-rating} == ‘gold’</simple> <to uri=“ibmmq:topic:specialCustomer”> </when>

<otherwise> <to uri=“ftp://user@host/orders/regularCustomers” /> </otherwise>

</choice> <log message=“received new order ${body.orderId}”/> <to uri=“ibatis:storeOrder?statementType=Insert”/> </route>


•  Batch file transfers •  Shared Database •  RPC •  Messaging

Integration Options


•  Asynchronous architectures •  Reliable message passing •  Loose coupling •  Heterogeneous integration •  Fault tolerant •  Scalable •  Real-time data

Why messaging?


•  The most widely used open-source messaging broker

•  Highly configurable •  Friendly license (no license fees!) •  Vibrant community (TLP) •  Backbone of top enterprises in retail, e-retail,

financial services, shipping, many others!

Apache ActiveMQ


•  High performance •  High availability •  Light-weight •  Multi-protocol (AMQP, MQTT, STOMP) •  Multi-transport (TCP,SSL,WS,VM,HTTP) •  JMS compliant •  Supported in production by Red Hat!

ActiveMQ Features


Network of Master/Slave


High-performance, real-time systems

Head Office

STORE

STORE

outlet

shop

24 houra

STORE

shop

Broker Clusters

•  Clustering, Network of Brokers

•  Client-aware failover

•  Master/Slave HA

•  Fabric


Ingestion for BigData Architecture

Broker Cluster

Broker Cluster

Web Servers

Web Servers

Web Servers

Web Servers

Camel HTTP Log

Reader

Camel HTTP Log

Reader

Camel HTTP Log

Reader

Camel HTTP Log

Reader

ESB

ESB

Camel HBase/HDFS

Camel Hbase/HDFS

HDFS


Integration Everywhere – Internet of Things

Connecting Things

• mobile devices • meters •  industrial controls • smart buildings • asset tracking •  traffic control • monitors • sensors • actuators

Broker Clusters

Arrival Airport 1


JBoss Fuse Integrate Everything!


More info on JBoss Fuse…

https://www.redhat.com/products/jbossenterprisemiddleware/fuse/

http://www.jboss.org/products/fuse

Management


Current issues with deployments

•  Installation •  SSH, download, unpack, install, permissions, ENV var

•  Configuration •  Container, individual apps/services, brokers, security

•  Upgrading •  Install, configure, rollback

•  Introspection •  Big picture statistics (JVM/OS memory usage, CPU) •  JMX (jconsole, visualVM)


Clients aware of Topology

•  Brokers •  failover:(tcp://host1:port1,tcp://host2:port2)

•  Camel endpoints •  from(“jetty:http://22.33.44.55:8080/endpoint”) •  to(“http4://11.22.33.44:9000/endpoint”)

•  Web service endpoints •  Access a specific endpoint: http://22.33.44.55:9091/endpoint


Current best practices

•  Use templates for configuration w/ template engine •  Puppet/Chef and/or Capistrano/Ansible •  Store configuration (templates + values) in SCM •  Separate configuration from binary deployments •  Verifiable build and release process


•  Simplifies deployments •  Provides management tools for centralized

configuration •  Visualize your middleware with HawtIO •  Polycontainer •  Blurs the line of PaaS

http://fabric8.io


•  Provides cluster capabilities, coordination •  Service discovery, load balancing, failover •  Deploy to cloud (IaaS, PaaS) •  Supported as JBoss Fuse (managed)

•  fabric8 1.0 is in Fuse 6.1 •  1.1 about to be released

•  RC1 released 6/10

http://fabric8.io


fabric8 1.1 release

•  Java Container •  Tomcat, TomEE, Jetty •  Spring Boot •  fabric:watch * with various containers •  Profile import/export •  fabric8 maven plugin enhancements


Architecture

Clustered Registry

Agent

Agent

Zookeeper

Agent

Zookeeper

Agent

Zookeeper

Version 1.0

Profile CXF Profile Camel

Profile Default

Features Configuration

Registry Content

containers that form an “ensemble”

Pulls profile data

Registers / Listens for changes


Core concepts

•  Fabric registry

•  Holds all configuration data •  Runtime registry for looking up distributed services

•  Profile

•  Describes the container set up •  Features, Bundles, ConfigAdmin PIDs, system properties •  Hierarchical structure •  Container versioning

•  Agent

•  Runs on each container •  Communicates with registry to make sure container provisioned

correctly


How’s fabric8 different than Puppet/Chef?

•  Middleware centric •  Container agnostic •  Deep knowledge about the running processes •  Consistent configuration •  Visualizations •  Versioning built in, upgrades, rollbacks, selective

upgrade, etc •  Use puppet/chef to provision your machines, use

fabric8 to provision and manage your middleware/apps


Fuse Management Console


HawtIO http://hawt.io


FuFuse Management Console

Continuous Delivery


Continuous Delivery •  Builds on continuous integration •  Establish a concrete pipeline to production •  Build/Test/Release often! •  Bottlenecks? •  Involves Dev and Ops to be successful •  Every build is a “release candidate”


Automate everything! •  Developers

•  Unit tests •  Integration tests •  Builds •  Deployments in dev

•  Operations •  VMs •  Provisioning software •  Deployments in QA/UAT/PROD


Tools for a CD pipeline •  Puppet/Chef to provision VMs •  Git for SCM •  Gerrit/Gitlab for code reviews •  Maven •  Jenkins + plugins •  and of course… Fabric8! •  What’s this Docker thingy?


Fabric8 maven plugin •  fabric8:deploy •  fabric8:zip •  fabric8:aggregate-zip •  fabric8:branch •  fabric8:script (for karaf only) •  http://fabric8.io/gitbook/mavenPlugin.html


Sample flow •  Check your code in •  Gerrit for code reviews •  Jenkins for build + CD pipeline •  Use fabric8:zip to deploy profiles to Maven repo •  Use fabric8:branch to automate deploying multiple

profiles to QA/UAT/PROD •  Can use profile-import to manually import zips •  Build the binary once!

Containers


Docker •  http://docker.io •  Open source •  Lightweight VEs •  Linux containers (LXC) •  Portable packaging •  Versioned •  Repositories


How different than VMs?


Fabric8 Docker support

•  Docker images •  Build docker containers for your java apps •  Flat classpath •  Microservices

OpenShift PaaS


What is OpenShift?

Red Hat’s free platform as a service for applications in the cloud.


What’s supported?


Flavors

origin

Public Cloud Service

On-premise or Private Cloud Software

Open Source Project


Terminology l  Broker – Management host, orchestration of Nodes

l  Node – Compute host containing Gears

l  Gear – Allocation of fixed memory, compute, and storage resources for running applications

l  Cartridge – A technology/framework (PHP, Perl, Java/JEE, Ruby, Python, MySQL, etc.) to build applications

l  Application – Instantiation of a Cartridge

l  Client Tools – CLI, Eclipse Plugin, Web Console, Java API, REST API


xPaaS/iPaaS

l  xPaaS

-  Cartridges for Entire JBoss & Fuse Portfolio

-  EAP and EWS Enterprise Cartridges Today

-  Several Community Cartridges Available Today

l  iPaaS

-  Integration PaaS

-  Fuse/Fabric Cartridge


More Info

http://fabric8.io

http://hawt.io

http://docker.io

http://jboss.org/products/fuse

http://activemq.apache.org

http://camel.apache.org


Questions? http://christianposta.com/blog

@christianposta

[email protected]