making communication across boundaries simple with azure service bus

PowerPoint Presentation

Yves Goeleven & Sean FeldmanSolution Architects & Azure MVPshttp://particular.net Azure Service BusSetting up communication across boundaries

1Setting up communication across network boundaries

Introduction to ServiceInsight for NServiceBus

Azure Service BusSecure & reliable communication between software components across network boundaries.

Inside cloud servicesBetween datacentersBetween branch officesHybrid solutionsIoT & Mobile devicesWhen do you need Azure Service Bus ?

2


Azure Service BusDNS (mynamespace.servicebus.windows.net): Scale Unit allocationLoad Balancer: High AvailabilityGateway Node: Authorization & entity lookupBroker Node: Servicing entity operations storeStore: SQL Azure Database or Storage ContainerHigh Level Architecture OverviewScale UnitScale UnitDNSLBLBGatewayGatewayGatewayGatewayClientBrokerBrokerBrokerBrokerStoreStoreStoreStoreStoreStoreStoreStoreStoreStoreStore

3


Azure Service BusEntity: a generic term for constructs in an Azure Service Bus namespaceEntity ?

QueueSubscriptionSubscriptionTopicSending ClientSending ClientReceiving ClientReceiving ClientReceiving ClientReceiving ClientEach entity is a SQL Azure Database or Storage Blob under the hood!Messaging EntitiesQueueTopicSubscriptionRule

Other (out of scope): Relay, EventHub, Consumer Group, ...

4

Connectivity

5


Azure Service Bus2 Ways to connectNamespaceManager: to create & manage entitiesMessagingFactory: to send/receive

Connectivity

6


Azure Service BusNamespaceManagerNamespace AddressTokenProvider

Methods: Get, Exists, Create, Update, Delete

Connecting to manage entitiesNamespaceManagerAdditional optionsREST API: https://docs.microsoft.com/en-us/rest/api/servicebus/ARM: https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-resource-manager-overviewTokenProviderREST APIHTTPSCreation is really expensive!C#

700.NamespaceManager.linq


Azure Service BusExceptions to deal with:

Connectivity TimeoutsAuthorization (manage)Broker ExceptionsTransientNon-transientConcurrency conflicts

What can possibly go wrong?DNSLBGatewayClientBroker timeout timeout Exception Exception UnauthorizedClient? Concurrency conflict timeoutC#

801.ConnectivityProblems.linq


Azure Service BusMessagingFactoryNamespace AddressTokenProviderConnection Settings

Maintains the network connection

MessageSender/ReceiverCreated by FactoryLogical connection

Connecting to send/receive messagesMessagingFactoryMessageSenderEstablishing a physical connection is expensive!MessageReceiverTCP/WebSocketConnectionAMQP/SBMP

MessagingFactoryMessageSenderMessageReceiverTCP/WebSocketConnectionAMQP/SBMPBrokerGatewayNodeBrokerGatewayNodeC#

902.MessagingFactory.linq


Azure Service BusYou want to receive/send as long as your application lives.

Physical connection will drop!

Creating a factory every time is not an option: establishing connection is expensive!

Lifecycle managementMaintain instances/poolsManage connection & open new if closed

Additional Connectivity ChallengesDNSLBGatewayFactory Local interruption Hardware failures RebootFactoryC#

1003.MaintainingConnectivity.linq

Reliable Message Exchange: Receiving

11


Azure Service Bus2 Ways to receive and process messagesReceive And DeletePeek Lock

Receiving messages

12


Azure Service BusMessage deleted upon receive. Cannot rollback received message.

Single shot processingException occurs = message lost!

Cloud environment is very susceptible to exceptions!Advise against this mode for workflow style processing

Receive and Delete

MessageReceiver

Receive And Delete

DeleteC#

1304. ReceiveAndDelete


Azure Service BusHide message on receiveTemporal lockExplicit complete/abandon after processing

Multi shot processingException occurs = RetryRetry counterCounter exceeded => DLQ

Slow processing = retries!!!RenewLock -> more gotchas there, see performance section

Peek Lock

MessageReceiver

Peek LockDLQ

Temporal Lock

DeleteC#

1405.PeekLock


Azure Service BusWhat Topology to use?Queue per message typeQueue per receiver

Serialization ChallengesInternal serialization or not?Need to deserialize same way!What CLR Type (Q per endpoint)

Concurrency ChallengesAll code must be thread safe

Handling Multiple Message TypesReceiverReceiver

Sender

Receiver

Sender

Is this the right way to construct a message?

new BrokeredMessage({json})

Sender

ReceiverReceiver

C#

1506. Deserialization


Azure Service BusMaxDeliveryCount To avoid locking on processing problematic messages, a message is automatically deadlettered after number of deliveries.

ChallengesPoisonous vs errored messagesBuilding a backing off strategy for retries (3rd party web service)Monitoring for DLQed messages over multiple entitiesDead Letter Queue vs Business Errors

DLQ

MaxDeliveryCount failed processing

Receiving Client

10... failed processing

Receiving Client

2 failed processing

Receiving Client

1C#

1607.DLQ.linq

Reliable Message Exchange: Sending

17


Azure Service BusChallenges well coverThrottlingMessage sizeFlooding entitiesTransactions

Sending messages

18


Presentation titleUsage exceeding quotas, the broker will throw a ServerBusyException.

Mitigation: retry after 10 seconds.

Retries can fail as well.

Cant retry the same message object.

Exception Handling (User Code) succeeded

Sending Client... throttled

Sending Client throttled

Sending ClientOperations can and will fail. Based on exception, need to be retried.C#

19104.ServerBusyException.Retries.linq


Azure Service Bus

Message size (including headers)Standard tier: 256KBPremium tier: 1MB

Headers: up to 64KB

MessageSizeExceededExceptionMessage Size

Sending Client

Size < MAX

Sending Client

Size > MAXWarning! Message size is not known until its sent outTo get an accurate value for the size of a BrokeredMessage, you should read the Size property after completing the Send/Receive operation on the BrokeredMessage.

20


Azure Service BusScenario:An orphaned subscription queue receiving messages that are not processed.

ChallengesOverflooding subscriptionsOverall namespace performance degradationOverflooding Subscriptions

SubscriptionSubscriptionTopicSending ClientReceiving ClientReceiving Client

SolutionsUse ForwardingGotchaTTL on subscription

ForwardingReceiving ClientSending Client

21


Azure Service BusAll messages succeed or fail atomic

NOT entirely what you think!Single queue onlyNo other transactional resources allowed

Transactions

QueueSending Client

Queue ASending Client

Queue B

C#

22106.Transactions.linq


Azure Service Bus

All messages succeed or fail - atomic

Surprise!

A hard limit of 100 messages orTransactionSizeExceededException

Prevents usage of batch completion, has negative performance impactTransactions and Number of Messages

Sending Client

1

100

Sending Client

1

100

101C#

23107.Transactions.100.linq


Azure Service BusAll messages succeed or fail atomic

NOT entirely what you think!Abandon of incoming message can not be included!Complete can be includedSingle queue onlyIf complete included, sends must go to receive queueNo other transactional resources allowed

Transactions with Receive Operations

QueueClient

Queue AClient

Queue B

Complete

Complete

24


Azure Service Bus

A hidden gem in a heap of features

Work around for the single queue transaction limit!Uses receive queue as a transfer queue

Use limited within namespace

Send Via

myQueueReceiving Client

queueASending Client

queueB

atomic operationforwardingtransfer queue

C#

25108.SendVia.linq

Performance

26


Azure Service BusDOs and DONTsFactoriesBatchingCoding practicesPrefetching

Performance improvements

27


Azure Service BusRemember

Using multiple senders helps.More factories helps more.

Multiple FactoriesMessagingFactoryMessageSenderMessageSenderTCP/WebsocketConnectionAMQP/SBMP

MessagingFactoryMessageSenderMessageSenderTCP/WebsocketConnectionAMQP/SBMPBrokerGatewayNodeBrokerGatewayNodeC#

28201.HighPerformanceSending.linq


Azure Service Bus2 KindsBatch flush intervalBatched send

Batch sizeStandard tier: 256KBPremium tier: 1MB

MessageSizeExceededExceptionBonus! Total message size is not known until its sent outExtra Bonus! Max 100 messages when sending inside transactionscope

Batch Sending

Sending Client

1

N

Sending Client

1

NBatch size < MAXBatch size > MAX

29low throughput + low latency = disable BatchFlushInterval


Azure Service BusDont!

Do!Coding Practices That Kill Native Batchingforeach(var message in messages){await sender.SendAsync(message)}var tasks = new List()foreach(var message in messages){ tasks.Add( sender.SendAsync(message) )}await Task.WhenAll(tasks)

30Note: for simplicity .ConfigureAwait(false) is omitted.


Azure Service BusHow it worksLoad additional messages when receivingPrefetched messages are locked on the gatewayEach client maintains its own in-memory cache of processingPrefetchingMessagingFactoryMessageReceiverBrokerGatewayNode

C#GotchasCannot renew locksPrefetch size depends on the processing code speedUnlocked prefetched messages are not evicted from cachePrefetched messages with expired TTL are not evicted from cache

31201.HighPerformanceReceiving.linq


Azure Service BusService Bus is a great service!With many powerfull features

Combining those features while maintaining reliability & performance is the challenge!Its a balancing act

32

Webinar #2December 14th

33

Q&A

34

Thank you

35


Azure Service BusNeed to set up secure communication between your software components across network boundaries?

Yves Goeleven and Sean Feldman will show you what it takes to use the Azure Service Bus for this purpose.

Learn how to:Set up and maintain the connection with the broker.Ensure reliable message exchange.Optimize performance.Abstract.

36

making communication across boundaries simple with azure service bus

Software