enterprise messaging foundations

EnterpriseMessagingFoundations

JeremyDeanehttp://jeremydeane.net

Agenda

v EnvironmentSetup

v AsynchronousHelloWorld!

vMessagingFoundations

v Exercises

vMessagingNetworks

v AdvancedMessaging

v ExtensibleMessaging

v Exercises

v AdditionalEnvironmentSetup

v HelloBunny!

v AMQPFundamentals

v Exercises

v RabbitMQ Architecture

v RabbitMQ Security&Operations

v AMQPIntegrations

v Exercises

EnvironmentSetup§ JDK1.8+Pre-Requisite($JAVA_HOMEinpath)§Maven3.3.3+Pre-Requisite($MAVEN_HOMEinpath)

§ DownloadandunzipActiveMQ – Version5.14.0+

§ Downloadhawtio – Version:1.4.56+§ renamehawtio-app.jar

§ Download(Clone)CodeFromGithub§ https://github.com/jtdeane/magic-supplies§ https://github.com/jtdeane/message-client§https://github.com/jtdeane/camel-standalone-router

§WindowsInstallActiveMQ asService:./$ACTIVEMQ_HOME/win65/InstallService.bat

AsynchronousHelloWorld!1. StartActiveMQ

2. BuildMagicSuppliesProject

3. Optional importintoIDE

4. ExecuteMessageProducer– RunasJUnit Test

5. ViewActiveMQ Console

6. ExecuteMessageConsumer– RunasJUnit Test

cd $ACTIVEMQ_HOME/bin./activemq start OR ./activemq.bat

cd magic-suppliesmvn clean install

Open http://localhost:8161/admin/ {admin/admin}Open http://localhost:8161/admin/queues.jsp

mvn test -Dtest=JMSProducerFunctionalTest

mvn test -Dtest=JMSConsumerFunctionalTest

ActiveMQWebConsoleAlternative: hawtio1. Starthawtio

2. OpeninWebBrowser

3. SelectConnectàLocal

4. ConnecttoActiveMQ

5. ClicknewAgentURLtolaunchbrowsing

6. SelectActiveMQ

java -jar hawtio-app.jar --port 8090

http://localhost:8090/hawtio

ActiveMQ Message Broker<execution environment>

Spring-Boot Application<execution environment>

REST Controller

Message Listener

TopicQueue

MagicSuppliesApplication

§ SpringBootWebApplication§ SpringBoot1.4.0§ JMSListeners- ActiveMQ

MagicSuppliesWeb Services1. StartActiveMQ *

2. StartWebApplication

3. OpenBrowser

4. ViewHealthCheckResponse

*WebApplicationrequiresActiveMQ started


mvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m”ORcd magic-supplies/targetjava -jar magic-supplies-1.0.0.jar

http://localhost:8080/info

{"build":{"description":"Magic SuppliesWebApplication","name":"Magic SuppliesWebAppication","version":"1.0.0","artifact":"magic-supplies"}}

P2PHidden Costs

BottomLineSOAbyMarcRix

WebservicesAwebservicedoesNOT trulydecoupletheconsumerandprovider

P2PIntegrationsThecostofmaintainingP2Pintegrationsincreasesexponentiallyasthenumberoftheconnectionsincreases

#ofconnections

$

gain

value

cost

loss

MessagingFoundations

Enterprise IntegrationPatterns*

*A.K.A. MessageExchangePatterns(MEP)

Publish Subscribe Domain (Topics)

Point-to-Point Domain (Queues)

Enterprise IntegrationPatterns

Channel Message FIlterSquareTriangle ChannelSquare SquareSquare

GuaranteedMessageDelivery

FilterMessages

AsynchronousRequestReply

Message OrientedArchitecture(MOA)

MessageSender Receiver

Broker

TopicQueue

createsprocesse

s

Delivers

Connects/ to Connects/ to

Hosted//by

JavaMessageService(JMS)*

*Alternative– AdvancedMessageQueuingProtocol(AMQP)

Message

JMS & Custom Properties

Payload

Properties(a.k.a.Headers)

§ JMS*(e.g.JMSType,JMSCorrelationID,JMSDeliveryMode,JMSExpiration)

§ Custom(e.g.MimeType,Token)

Payload(a.k.a.Body)

§ Text§ Object§Map§ Bytes§ Stream

JavaMessageService(JMS)

Producer

Publisher

Message Broker<JMS Providert>

Topic

Queue

Subscriber

Consumer<Listener>

Subscriber

Message Store

JMS Selector

Channel

Channel

ApacheActiveMQ

IntegrationOptions

JavaMessageService(JMS)AdvancedMessageQueuingProtocol(AMQP)

DeploymentFlexibility

Stand-aloneEmbedded

AdvancedTopologies

Master-SlaveHighAvailability(HA)FederatedNetwork

Support

ActiveOpenSourceCommunityCommercial24X7Options

Languages- Transport• Java-Scala– TCP/NIO• Ruby,Perl,Python– Stomp• C#(NMS)–TCP/NIO

MessagingExercises

Exercise:JMSSender1. StartActiveMQ

2. BuildClient

3. ExecuteJAR

4. ViewMessagefromActiveMQ orhawtio (test.alchemy)

java -jar message-client-1.0.0.jar

mvn clean installcd target



Spring Application<execution environment>

ActiveMQ Standalone Broker<execution environment>

Alchemy Queue

MessageSender

Channel

message-client/src/main/java/cogito/online/application/MessageClient:sendAlchemyAlert(..)

Demo:JMSListeners

magic-supplies/src/main/java/cogito/online/application/MagicSuppliesConfigurationmagic-supplies/src/main/java/cogito/online/messaging/SingleOrderProcessingMessageListenermagic-supplies/src/main/java/cogito/online/messaging/OrderProcessingMessageListener

Srping-Boot Application<execution environment>

MessageListener

Configuration

Producer <Message Client>


Queue

Message Store

Channel

Exercise:JMSListener1. StartActiveMQ

2. StartWebApplication

3. ExecuteJAR

4. ViewActivityviaActiveMQ orhawtio

5. ViewMagic-SuppliesConsoleLog

java -jar message-client-1.0.0.jar magic.orderjava -jar message-client-1.0.0.jar magic.orders



cd magic-supplies/targetjava -jar magic-supplies-1.0.0.jar

Demo:JMSSubscribers

magic-supplies/src/main/java/cogito/online/application/MagicSuppliesConfigurationmagic-supplies/src/main/java/cogito/online/messaging/AlertProcessingTopicListener.java


MessageSubscriber

Configuration



Topic

Channel

Exercise:JMSSubscribers1. StartActiveMQ

2. StartMagicSupplies

3. ExecuteJAR

4. ViewActivityviaActiveMQ (orhawtio)

5. ViewMagic-SuppliesConsoleLog

java -jar message-client-1.0.0.jar magic.alerts


Open http://localhost:8161/admin/ {admin/admin}Open http://localhost:8161/admin/topics.jsp

cd magic-supplies/targetjava -jar magic-supplies-1.0.0.jar

MessagingNetworks

ActiveMQ Broker Topologies

ActiveMQinAction byBruceSnyder,Dejan Bosanac,andRobDavies

§ EmbeddedBroker(VMTransport)

§ StandaloneBroker(a.k.a.Client-Server)

§ LoadBalancedBrokers

§ NetworkedBrokers(a.k.a.StoreandForward)

failover:(tcp://BrokerA:61616,tcp://BrokerC:61616)?randomize=true

ActiveMQ HighAvailabilityandDisasterRecovery

ActiveMQinAction byBruceSnyder,Dejan Bosanac,andRobDavies

§MasterSlave– FileSystem§ KahaDB§ SAN-NFS4

§MasterSlave– JDBC§ RDBMS(e.g.PostgreSQL)

§MasterSlave– Replicated§LevelDB &Zookeeper

§ FailoverProtocol(usedomainaliases)

failover:(tcp://PrimaryBroker:61616,tcp://SecondaryBroker:61616)?randomize=false

ActiveMQ HighAvailabilityExample

failover:(tcp://PrimaryBroker:61616,tcp://SecondaryBroker:61616)?randomize=false

Linux VM

Queue

<Execution Environment>ActiveMQ (Master)

Topic

Linux VM

Queue

<Execution Environment>ActiveMQ (Slave)

Topic

<NFS Mount>

Kaha DB

ActiveMQ SimpleHighAvailabilityExercise1. Createthefollowingdirectorystructure

2. UnpackActiveMQ twicerenamingtomasterandslave

3. UpdateMaster../conf/activemq.xml:Name,Management,Persistence,andNetworkConfigurations

../cluster

../cluster/data

../cluster/master

../cluster/slave

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="master" dataDirectory="${activemq.data}”><managementContext>

<managementContext createConnector="true" connectorPort="1099"/> </managementContext><persistenceAdapter>

<kahaDB directory="<<your location>>"/cluster/data"/> </persistenceAdapter><networkConnectors>

<networkConnector uri="static:(tcp://localhost:61617)"/> </networkConnectors>

ActiveMQ HighAvailabilityExercise4. UpdateSlave../conf/activemq.xml:Name,Management,

Persistence,Network,TransportConfigurations

5. UpdateSlave../conf/jetty.xml:jettyPort

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="slave" dataDirectory="${activemq.data}">

<networkConnectors>

<networkConnector uri="static:(tcp://localhost:61616)"/> </networkConnectors><!– update – also remove other transports à<transportConnector name="openwire" uri="tcp://0.0.0.0:61617?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>

<property name="port" value="8162"/>

ActiveMQ HighAvailabilityExercise6. StarttheMasterandthentheSlaveBroker

7. OpenMasterWebConsole– QueuesPage

8. CreateaQueueandSendaMessage

9. StoptheMasterBroker

10. OpenSlaveWebConsole– QueuesPage

11. ViewtheQueueandpersistedMessage

cd ../cluster/<<master || slave>>/bin./activemq start OR ./activemq.bat



cd ../cluster/<<master || slave>>/bin./activemq stop OR Ctrl+C

ActiveMQ FederatedTopologyExampleLinux VM

Queue


Topic

Linux VM

Queue


Topic

<NFS Mount>

Kaha DB

Linux VM

Queue


Topic

Linux VM

Queue


Topic

<NFS Mount>

Kaha DB

Linux VM

Queue


Topic

Linux VM

Queue


Topic

<NFS Mount>

Kaha DB

WebNetworkSegment

ServiceNetworkSegment

DataNetworkSegment

ActiveMQ ErrorHandling

§ DeadLetterQueue(DLQ)§ Review(WebConsole)§ Redeliver(Plug-in)§ Remediate(Policy)

§MonitorDLQDepth§ Alerts(Splunk Example)

§ FailedMessageTransactions§ Rollback§ DLQ

§ ThroworLogException

ActiveMQ Security

§ AuthenticationandAuthorization§WebConsole§ JMXConnector(BindtospecificPort#)§ QueuesandTopics

§ Auditing§MonitorLogs§ PeriodicallyAuditAccess

§ Confidentiality§ Transport(SSL)§Messages(Encryption)§ KahaDB Files(Encryption)§ LogFiles(Encryption)

AdvancedMessaging

ApacheCamel– EnterpriseIntegrationLibrary

CamelInAction byClausIbsenandJonathanAnstey

§ DomainSpecificLanguages(DSL)– Java,Scala,SpringDSL

§ RouteBuilders– createEndpoints(i.e.from&to)usingComponents(e.g.protocol)andProcessors(e.g.Mediation,Enrichment

§ RouteEngine– LoadsandexecutesRoutes

ApacheCamelMessage

JMS

JMS & Custom Properties

Payload

Camel (IN)

Camel Headers

Body

Camel (OUT)

Camel Headers

Body

HTTP Request

HTTP Headers

Body

ApacheCamelRouteProcessing/** Splitter - xpath expression*/

from("activemq:emagic.orders").split(splitXPath).parallelProcessing().

to("activemq:emagic.order");

Camel Route Engine

Camel Route

<JMS Component>Endpoint (From)

<EIP Processor>Some Processing

<DB Component>Endpoint (To)

CamelMessage

Configurations

/** Content Based Routing - Inpatient-Outpatient Events* http://camel.apache.org/jsonpath.html*/

from("activemq:event.ingestion").process(new TrackingIdProcessor()).choice().

when().jsonpath("$[?(@.class==inpatient)]").to("activemq:event.inpatient").

otherwise().to("activemq:event.outpatient").

end().to("activemq:event.audit");

Spring-Boot&ApacheCamelJava Runtime Environment (JRE)

Spring IOC Container

Camel Route Engine

Camel Route

<JMS Component>Endpoint (From)

<EIP Processor>Some Processing

<DB Component>Endpoint (To)

CamelMessage

Configurations

/** Spring Boot – Camel Route Application */@Configuration@ImportResource("classpath:camel-route-spring.xml")public class MagicRouterApplication {

public static void main(String[] args) {SpringApplication.run(MagicRouterApplication.class, args);

}}

Spring Boot Application<Executable JAR>

Message Broker<JMS Provider>

Queue<magic.order>

Content Router

Queue<priority.orders>

Queue<emagic.orders>

Camel Routes

Queue<emagic.order>

Splitter



MessageSubscriber

Configuration

MessageListener

Producer <Console>

Exercise:ContentBasedRouting

Exercise:ContentBasedRouting1. BuildprojectandimportintoIDE(e.g.Eclipse,IntelliJ)

2. ViewMagicRouteBuilder.java &MagicRouterApplication.java

3. Viewcamel-route-spring.xml

4. ExecuteSpringBootApplication

5. ExecuteMessageClient

cd camel-standalone-routermvn clean installmvn eclipse:eclipse

mvn spring-boot:run ORjava –jar ./target/camel-standalone-router-1.0.0.jar

java -jar message-client-1.0.0.jar emagic.orders

Exercise:ContentBasedRouting6. OpenActiveMQWebConsoleandviewpriority.order queue

7. ViewMagicSupplieslogforordersprocessed(magic.order)

8. ViewCamelStandalone– JConsole

9. ConnecttoCamel– hawtio


jconsole {pid}

Camel Unit Testing@OverrideprotectedCamelContext createCamelContext()throwsException{

CamelContext context=super.createCamelContext();

//setuproutestoMockEndpointscontext.addRoutes(new RouteBuilder(){

publicvoidconfigure(){

//testxpath splitfrom("direct:split").

split(new XPathBuilder (MagicRouteBuilder.splitXpath)).to("mock:order");

});

returncontext;}

@TestpublicvoidtestSplitLogic()throwsException{

//SetexpectationsMockEndpoint order=getMockEndpoint("mock:order");order.expectedMessageCount(3);

//readinasmallbatchStringxml=IOUtils.toString(this.getClass().

getResourceAsStream("batch.xml"),"UTF-8");

//executethetesttemplate.sendBody("direct:split",xml);

//shouldbethreeordersinthebatchorder.assertIsSatisfied();

}

MagicRouteBuilderTest.java

Camel FunctionalTesting1. DownloadandinstallApacheJMeter

2. Copy$ACTIVEMQ_HOME/activemq-all-5.13.1.jarto$JMETER_HOME/lib

3. StartJMeter - ./jmeter.sh orjmeter.bat

4. Opencamel-magic-router/src/test/jmeter/Mock CamelPublisher.jmx

5. Runtest

Otherpossibleframeworks

• RobotFramework

• SOAPUI

Message Broker<JMS Provider>

Producer <JMeter>

Queue<emagic.dead>

Queue<emagic.orders>

Camel Routes

Splitter Dead Letter Channel

Demo:DeadLetterChannel

errorHandler(deadLetterChannel("activemq:emagic.dead").maximumRedeliveries(1).redeliveryDelay(1000));

java -jar message-client-1.0.0.jar emagic.bad

ExtensibleMessaging

ActiveMQ Master Broker<execution environment>

ActiveMQ Slave A Broker<execution environment>

ActiveMQ Slave B Broker<execution environment>

Spring Boot Application 1<Executable JAR>

Camel Routes



Camel Routes Camel Routes

Spring-Boot&Camel– ActiveMQ Integration

PrimaryBenefits:§ Resiliency– CamelRoutefailuredoesnottakedownBroker§ Extensibility– SwapoutMessageBrokerorEIPFramework§ ContinuousDevelopment– NoBrokerorWebServerdowntime

CompetingConsumers

ActiveMQ Broker<execution environment>

Spring Boot App 1<Executable JAR>

Camel Routes


Camel Routes


Camel Routes

Queue<emagic.order>

GoldenPath – Processedonceinorder

Bad Producer

Redis<execution environment>

Key-Value Store

Idempotent CompetingConsumers

Spring-Boot&Camel– DeploymentOptions

Linux VM

Camel Route

<Execution Environment>Spring-Boot Camel

Camel Route

Properties Log

Linux VM

Camel Route


Camel Route

Properties Log

Port: 2100

Camel Route


Camel Route

Properties Log

Port: 2101

§ SingleApplicationperVM

§MultipleApplicationsperVM

§ IaaS Container(e.g.Docker)

§ PaaS Container(e.g.CloudFoundry)

Exercise:EnrichmentandTransformation1. Update:ReplaceSimpleChoiceRoutewithMediationRoute

2. StopSpringBoot,BuildProject,andStartSpringBoot

XmlJsonDataFormat xmlJsonFormat = new XmlJsonDataFormat();xmlJsonFormat.setForceTopLevelObject(true);

from("activemq:emagic.order").choice().

when().simple("${in.body} contains 'Houdini'").process(new Processor() {public void process(Exchange exchange) {

exchange.getIn().setHeader("VIP", "true");}

}).to("activemq:priority.order").

otherwise().marshal(xmlJsonFormat).transform(body().regexReplaceAll("@", "")).to("activemq:magic.order");

mvn clean installmvn spring-boot:run ORjava –jar ./target/camel-standalone-router-1.0.0.jar

Exercise:EnrichmentandTransformation3. ExecuteMessageClient

4. OpenActiveMQWebConsoleandviewpriority.order queue

6. OpenActiveMQWebConsoleandviewpriority.order queue

7. ViewMagicSupplieslogforordersprocessed(magic.order)

8. ConnecttoCamel&ActiveMQ – hawtio

Open http://localhost:8161/admin/ {admin/admin}Open http://localhost:8161/admin/queues.jspOpen http://localhost:8161/admin/topics.jsp



Exercise:Wire-Tap1. ReplaceRoutingw/:Wire-TaptoSplitterandEvaluationRoute


3. ExecuteMessageClient&ViewResults

from("activemq:emagic.order").wireTap("direct:ministry").

to("activemq:magic.order");

from("direct:ministry").choice().

when().simple("${in.body}contains'Elder'").log("ILLEGALMAGICALERT").to("activemq:topic:magic.alerts").

otherwise().log("...offintotheether");



Open http://localhost:8161/admin/queues.jspOpen http://localhost:8161/admin/topics.jsp Wire Ttap

BonusExercise:IdempotentConsumer(In-Memory)

?

Idempotent Consumer

1. ViewIdempotentConsumerRouteBuilder inCamel-Standalone-Router

2. StartCamel-StandaloneRouter


4. OpenActiveMQWebConsole(orviewviahawtio)

5. ViewMagicSupplieslogforordersprocessed(magic.order)• Only two of three sent messages should be present.

mvn spring-boot:run ORjava –jar ./target/camel-standalone-router-1.0.0.jar

java -jar message-client-1.0.0.jar unique.orders

BonusExercise:IdempotentConsumer(Redis)

?

Idempotent Consumer

1. DownloadInstallRedis &Start• Mac:brew install redis• Linux:yum install redis

• Windows:https://github.com/rgl/redis/downloads

2. Updatecamel-route-spring.xml – Uncomment


 <bean id="redisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">

<property name="hostName" value="localhost"/> <property name="port" value="6379"/>

</bean>

<bean id="redisTemplate" class="org.springframework.data.redis.core.RedisTemplate"> <property name="connectionFactory" ref="redisConnectionFactory"/>

</bean>

BonusExercise:IdempotentConsumer(Redis)1. UpdateIdempotentConsumerRouteBuilder – Comment:

2. UpdateIdempotentConsumerRouteBuilder - Uncomment


4. RepeatIn-MemorySteps3-5

from("activemq:unique.order").idempotentConsumer(header("uniqueId"),MemoryIdempotentRepository.memoryIdempotentRepository(200)).


..@AutowiredRedisTemplate<String, String> redisTemplate;..from("activemq:unique.order").

idempotentConsumer(header("uniqueId"), RedisIdempotentRepository.redisIdempotentRepository(redisTemplate, "camel-repo")).



Messaging2.0

AdditionalEnvironmentSetup§ DownloadandinstallRabbitMQ (3.6.2+)

§MacStandalone orHomebrew§WindowsStandalone

§ Add$RABBITMQ_HOME/sbin toPATH

§ Commands(WindowsasaService)§ Start:rabbitmq-server -detached§ Status:rabbitmqctl status

§ Stop:rabbitmqctl stop

§ InstallManagementPlugin

§ VerifyManagementConsole(guest/guest):

rabbitmq-plugins enable rabbitmq_management

http://localhost:15672/

Mac&LinuxO/S– Update/etc/hosts127.0.0.1 <<name of your laptop>>

AdditionalEnvironmentSetup§ Download(Clone)CodeFromGithub

§ TestHarness-https://github.com/jtdeane/amqp-client§ AMQPConsumer- https://github.com/jtdeane/amqp-consumer§ AMQPProducer- https://github.com/jtdeane/amqp-producer§ AMQPComplexEventProcessor(CEP)- https://github.com/jtdeane/amqp-cep

Hello Bunny!

1. StartRabbitMQ

2. Buildandimportamqp-client intoIDE

3. SendAMQPMessage– RunasJUnit Test

4. ViewExchange,Queue,andMessageinManagementConsole

5. ReceiveAMQPMessage– RunasJUnit Test

mvn clean install

mvn test -Dtest=ProducerFunctionalTest

mvn test -Dtest=ConsumerFunctionalTest

http://localhost:15672/#/exchanges/%2F/test.direct

http://localhost:15672/#/queues/%2F/magic.greetings

amqp-client/src/test/java/ws/cogito/magic/messaging/ProducerFunctionalTest.javaamqp-client/src/test/java/ws/cogito/magic/messaging/ConsumerFunctionalTest.java

AdvancedMessagingQueueing Protocol(AMQP)

RabbitMQ implement0.9.1ofAMQPStandard

AMQPMessageMessage

Properties{Standard or Broker Defined}

Headers{Custom}

Payload{Stream of bytes}

Property Purpose

ContentType ThecontenttypeofthemessageasaMediaType.

ContentEncoding Theencodingto andfrombytes(UTF-8)

RoutingKey UsedbyanExchangeformessagerouting

DeliveryMode Persistent(2)ornon-persistent(1)

Reply To FormessagingforwardingorRequest-ReplyPattern

Expiration Time-to-Live (TTL)inMilliseconds

Priority MovemessageupQueue(defaultis0)

CustomHeaderExample:TrackingID

AMQPVirtualHosts, Exchanges&RoutingKeys§ VirtualHostsprovidelogicalcontainersforExchanges&Queues

§ ReferredtoasVHost§ DefaultVirtualHost(/)– Donotdelete!§ SuggestedNamingScheme:<owner>.<domain || purpose>.vhost

§ ExchangesroutemessagesbasedonaRoutingKey§ DirectExchange:Point-to-Point(P2P)§ TopicExchange: Publish-Subscribe§ Fanout Exchange: Broadcast

§DefaultExchanges(AMQP)– Donotdelete!§ SuggestedNamingScheme:<owner>.<domain || purpose>.<Exchange Type>

§ RoutingKeysrepresentahierarchicalnamespace§ ws.cogito.magic.sales.orders§ ws.cogito.magic.finance.bills

Message Broker

Virtual Host

Publisher ExchangeMessage

AMQPQueues andBindingKeys§ QueuesareassociatedwithExchangesviaBindingKeys

§ Durable orTransient§ SuggestedNamingScheme:<owner>.<hierarchy>.<entities>

§ acme.magic.finance.invoices§ acme.magic.sales.orders

§ BindingKeysoftenmatchQueueNamesexactlyforDirectExchanges§ acme.magic.finance.invoices

§ BindingKeysoftenusespecialcharactersforTopicExchanges§ * (star)cansubstituteforexactlyoneword§ # (hash)cansubstituteforzeroormorewords

§ acme.magic.finance.*§ acme.magic.#

RabbitMQ SimulatorDemo

http://tryrabbitmq.com/

AMQP Connections,ChannelsandAcknowledgements§ Connections arelonglivedTCPconnectionswhileChannelsmultiplexedinteractionswithinaConnection.

§MessageAcknowledgementsareautomaticbydefaultbutcanbemanual

§Messagescanalsoberejected orre-queued orsenttoDeadLetterQueue(DLQ)

AMQPDeclarations

ConflictingDeclarationswillfail!

AMQPExercises

AMQPExercisesSetup1. StartRabbitMQ

2. CreateUser(client/client)– AdminTag

3. CreateVirtualHost(VHost)– cogito

4. AssociateUserclient toVHost cogito&/

5. Buildandbootamqp-client

6. ViewdeclaredExchanges

http://localhost:15672/#/users

http://localhost:15672/#/vhosts

http://localhost:15672/#/vhosts/cogito

mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"

http://localhost:15672/#/exchanges/cogito/magic.directhttp://localhost:15672/#/exchanges/cogito/magic.topic

Code:amqp-client/src/main/java/ws/cogito/magic/utilities/AmqpDeclarationsConfiguration.java

AMQPExercisesSetup1. CreateQueuemagic.tests *

2. CreateQueueDirect&TopicBindings**

3. CreateUsers)– AdminTag• consumer/consumer• producer/producer• evaluator/evaluator

4. AssociateUserstoVHost cogito&/

http://localhost:15672/#/queues

Exchange: magic.direct, Queue & Routing Key: magic.tests

Exchange: magic.topicTopic: magic.testing.eventsRouting Key: magic.testing.#

Knuffle-Bunny-Cautionary*Shortenedfortesting,shouldbews.cogito.magic….etc**Normallythereshouldbeonequeueperconsumer

AMQPSender

Note:normallythereshouldbeonequeueperconsumer;thisisjustfortesting


AMQPSender1. IssueHTTPRequestviacurlorRESTclient(e.g.Postman)- Direct

2. ViewmessageinQueueandthenpurgeQueue

3. IssueHTTPRequestviacurlorRESTclient(e.g.Postman)- Topic

4. ViewmessageinQueueandthenpurgeQueue

5. ViewCode

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1125, "customer":"Houdini", "item":"Cards", "amount":6}' http://localhost:9999/amqp/magic.direct/magic.tests

http://localhost:15672/#/queues/cogito/magic.tests

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1126, "customer":"Teller", "item":"Dice", "amount":4}' http://localhost:9999/amqp/magic.topic/magic.testing.foo

http://localhost:15672/#/queues/cogito/magic.tests

/amqp-client/src/main/java/ws/cogito/magic/utilities/web/AmqpController.java

AMQPListener&Subscriber


1. StartRabbitMQ

2. Buildandbootamqp-consumer

3. ViewQueuesandBindings

4. ViewDeclarationCode

5. Bootamqp-client

AMQP Listener


http://localhost:15672/#/queues/cogito/magic.eventshttp://localhost:15672/#/queues/cogito/magic.sales.orders

/amqp-consumer/src/main/java/ws/cogito/magic/AmqpDeclarationsConfiguration.java


6. SendOrderMessageviaamqp-client

7. Viewamqp-client andamqp-consumer consolelogs

6. ViewOrderListenerCode

7. Stopamqp-consumerandsendanotherOrderMessage

8. ViewmessageviaManagementConsole

9. Bootamqp-consumerandviewconsolelogs

AMQP Listener

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1127, "customer":"Blain", "item":"Marbles", "amount":2}' http://localhost:9999/amqp/magic.direct/magic.sales.orders

/amqp-consumer/src/main/java/ws/cogito/magic/messaging/OrdersMessageListener.java/amqp-consumer/src/main/java/ws/cogito/magic/MessageLogging.java

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1128, "customer":"Blackstone", "item":"Scarf", "amount":4}' http://localhost:9999/amqp/magic.direct/magic.sales.orders

1. SendEventMessageviaamqp-client


6. ViewEventListenerCode

7. SendanotherEventMessageviaamqp-client


AMQP Subscriber

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"event":21, "location":"Las Vegas", "title":"Penn and Teller", "date":"2016-12-31"}' http://localhost:9999/amqp/magic.topic/magic.events.shows

/amqp-consumer/src/main/java/ws/cogito/magic/messaging/EventsMessageListener.java

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"event":22, "location":"Clearwater", "title":"ArchConf", "date":"2016-12-12"}' http://localhost:9999/amqp/magic.topic/magic.events.conferences

RabbitMQ Architecture

RabbitMQ HighAvailabilityTopologyRabbitMQ Server

Message BrokerVirtual Host

RabbitMQ Server


Mirrored Queues

Exchange Exchange

Queue A

Queue B

Queue A

Queue B

§ RabbitMQ ServersareClustered – SharedSecret– Erlang Cookie

§ RabbitMQ HAPolicy:rabbitmqctl set_policy ha-all "" '{"ha-mode":"all"}'

§ ClientsconnecttosetofaddressesorLoadBalancercanproxyClients(e.g.HAProxy)

RabbitMQ Federated Topologies

RabbitMQ Server RabbitMQ Server

Finance Cluster

RabbitMQ Server

RabbitMQ Server RabbitMQ Server

Sales Cluster

RabbitMQ Server

Federation Plugin

Shovel Plugin

§ FederationPlugin– Both“upstream”and“downstream”Consumersreceivemessage

§ ShovelPlugin– ForwardsMessagefrom“upstream”Queuetoremote“downstream”Exchange

RabbitMQ Security§ UsershaveVHost Read,Write,andConfigurepermissions

§ LDAPPlugin – Authentication-Authorization

§ TransportLevelSecurity(TLS)– Port5671§ TLSConnectionFactoryGist

§EncryptionatRest(Mnesia DB&Logs)– e.g.VormetricRabbitMQ Server


DB

User Security

Logs

Transport Level Security

Encryption at Rest

RabbitMQ Operations§WebConsole

§ HTTPRESTAPI (guest/guest)– ListoutVirtualHosts

§ NewRelic &Zabbix Plugins

§ CommandLineInterface(CLI)&Tool

1. DownloadCommandLineToolin$RABBITMQ_HOME/sbin

2. Mayneedtoupdatepermissionsonfile

3. ListoutVirtualHosts

http://localhost:15672/cli/

chmod 777 rabbitmqadmin

rabbitmqadmin list vhosts

http://localhost:15672/api/vhosts

AdvancedAMQPPatterns

AMQPComplexEventProcessing


1

3

2

AMQP– ComplexEventProcessing1. StartRabbitMQ

2. Buildandbootamqp-cep

3. ViewQueuesandBindings


5. Bootamqp-consumer &amqp-client


http://localhost:15672/#/queues/cogito/magic.evaluationshttp://localhost:15672/#/queues/cogito/magic.alerts

/amqp-cep/src/main/java/ws/cogito/magic/AmqpDeclarationsConfiguration.java


AMQP– ComplexEventProcessing6. PublishEventviacurlorRESTclient(e.g.Postman)

7. Viewamqp-client, amqp-consumer, amqp-cep consolelogs

8. ViewtheEvaluationMessageListener

9. PublishEventviacurlorRESTclient(e.g.Postman)

10. Viewamqp-client, amqp-consumer, amqp-cep consolelogs• Notehowanalertisnotraisedbecauseofthelocation

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"event":21, "location":"Las Vegas", "title":"Penn and Teller", "date":"2016-12-31"}' http://localhost:9999/amqp/magic.topic/magic.events.shows

/amqp-cep/src/main/java/ws/cogito/magic/messaging/EventsMessageListener.java

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"event":22, "location":"Clearwater", "title":"ArchConf", "date":"2016-12-12"}' http://localhost:9999/amqp/magic.topic/magic.events.conferences

AMQP– WhynotSpringIntegrationsorApacheCamel?@Beanpublic IntegrationFlow amqpFlow() {

return IntegrationFlows.from(Amqp.inboundGateway(this.rabbitConnectionFactory, queue())).transform("hello "::concat).transform(String.class, String::toUpperCase).get();

}@Beanpublic IntegrationFlow amqpOutboundFlow() {

return IntegrationFlows.from("amqpOutboundInput").handle(Amqp.outboundAdapter(this.amqpTemplate)

.routingKeyExpression("headers.routingKey")).get();

context.addRoutes(new RouteBuilder() {public void configure() {

from("rabbitmq://localhost?exchange=direct_logs&exchangeType=direct&bindingKey=ERROR").setBody().simple("ERROR: ${in.body}").to("stream:out");

from("rabbitmq://localhost?exchange=direct_logs&exchangeType=direct&bindingKey=WARN").setBody().simple("WARN: ${in.body}").to("stream:out");

}});

SpringIntegrations– Immature- poorlydocumentedDSL

ApacheCamel– ImmatureAMQP&RabbitMQ Components

AMQPAsynchronousRequest-Reply– TempQueue


1

2

3

StartswithPOSThttp://localhost:9000/priority/orders

4

AMQP– Asynchronous Request-Reply – TempQueue1. StartRabbitMQ

2. Buildandbootamqp-producer

3. ViewQueueandBinding


5. Bootamqp-consumer


http://localhost:15672/#/queues/cogito/magic.priority.orders

/amqp-producer/src/main/java/ws/cogito/magic/AmqpDeclarationsConfiguration.java


AMQP– Asynchronous Request-Reply – TempQueue6. IssuePriorityOrderRequestviacurlorRESTclient(e.g.Postman)

7. ViewthePriorityOrderController:processOrder

8. Viewamqp-consumer andamqp-producer consolelogs• NotetheTrackingIDandReplyTo Queue

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1129, "customer":"Penn", "item":"Rabbit", "amount":6}' http://localhost:9000/priority/orders

/amqp-consumer/src/main/java/ws/cogito/magic/web/PriorityController.java

“Beware of time-outs & downtime”

AMQPAsynchronousRequest-Reply– CorrelationID

http://tryrabbitmq.com/StartswithPOSThttp://localhost:9000/priority/vip/orders

12

3

4

AMQP– Asynchronous Request-Reply – CorrelationID1. StartRabbitMQ

2. Buildandbootamqp-producer

3. ViewQueueandBinding


5. Bootamqp-consumer


http://localhost:15672/#/queues/cogito/magic.priority.orders

/amqp-producer/src/main/java/ws/cogito/magic/AmqpDeclarationsConfiguration.java


AMQP– Asynchronous Request-Reply– CorrelationID1. StartRabbitMQ

2. Buildandbootamqp-producer and amqp-consumer

3. IssueVIPOrderRequestviacurlorRESTclient(e.g.Postman)

4. ViewthePriorityOrderController:processOrderVip

5. View amqp-consumer andconsolelogsamqp-producer• NotetheTrackingID,CorrelationID,andReplyTo Queue

6. ViewthePriorityMessageListener

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1130, "customer":"Angel", "item":"quarters", "amount":6}' http://localhost:9000/priority/vip/orders

/amqp-consumer/src/main/java/ws/cogito/magic/web/PriorityController.java

/amqp-producer/src/main/java/ws/cogito/magic/messaging/PriorityMessageListener.java

Questions&FeedbackQuestions&Feedback

MyContactinformation:

[email protected]://jeremydeane.net

https://github.com/jtdeane/magic-supplies

https://github.com/jtdeane/message-client

https://github.com/jtdeane/camel-magic-router

https://github.com/jtdeane/camel-standalone-router

https://github.com/jtdeane/amqp-client

https://github.com/jtdeane/amqp-consumer

https://github.com/jtdeane/amqp-producer

https://github.com/jtdeane/amqp-cep

enterprise messaging foundations

Software