sparkplug specification version 1.0 - amazon s3specification.pdf · sparkplug specification version...
Post on 07-Aug-2019
227 Views
Preview:
TRANSCRIPT
Page1SparkplugSpecificationVersion1.0
SparkplugSpecification
MQTTTopicNamespaceandPayloadDefinition
Version1.0
Page2SparkplugSpecificationVersion1.0
RevisionNumber Date Author Description1.0 5/26/16 ArlenNipper InitialRelease
Page3SparkplugSpecificationVersion1.0
TableofContentsTableofContents.........................................................................................................................................3
TableofFigures............................................................................................................................................6
1. Introduction.........................................................................................................................................7
2. Background..........................................................................................................................................8
3. InfrastructureComponents..................................................................................................................9
3.1. MQTTServer(s)...........................................................................................................................9
3.2. MQTTEdgeofNetwork(EoN)Node...........................................................................................9
3.3. Device.........................................................................................................................................9
3.4. MQTTEnabledDevice(Sparkplug)..............................................................................................9
3.5. MQTTHostNode......................................................................................................................10
3.6. MQTTApplicationNode............................................................................................................10
3.7. Security.....................................................................................................................................10
4. LeveragingStandardsandOpenSource.............................................................................................11
4.1. OASISMQTTV3.1.1Specification.............................................................................................11
4.2. EclipseFoundationIoTResources.............................................................................................11
4.2.1. Paho......................................................................................................................................11
4.2.2. Kura......................................................................................................................................11
4.3. RaspberryPiHardware.............................................................................................................11
5. GeneralMessageFlow.......................................................................................................................12
5.1. State..........................................................................................................................................12
6. SparkplugMQTTTopicNamespace....................................................................................................13
6.1. SparkplugTopicNamespaceElements.....................................................................................13
6.1.1. namespaceElement.............................................................................................................13
6.1.2. group_idElement.................................................................................................................13
6.1.3. message_typeElement.........................................................................................................13
6.1.4. edge_node_idElement.........................................................................................................14
6.1.5. device_idElement................................................................................................................14
6.2. ExampleIgnitionFolderStructure............................................................................................14
7. SparkplugMQTTPayloads..................................................................................................................16
7.1. SparkplugMQTTPayloadEncoding..........................................................................................16
7.1.1. GoogleProtocolBuffers........................................................................................................16
Page4SparkplugSpecificationVersion1.0
7.1.2. LeveragingExistingKuraProtocolBufferSupport................................................................17
7.2. MQTTEoNBirthandDeathCertificate.....................................................................................18
7.2.1. EoNDeathCertificate(NDEATH)..........................................................................................18
7.2.1.1. Birth/DeathSequenceNumber-bdseq.......................................................................19
7.2.2. EoNBirthCertificate(NBIRTH).............................................................................................19
7.2.2.1. Birth/DeathSequenceNumber-bdseq.......................................................................19
7.2.2.2. MessageSequenceNumber-seq................................................................................19
7.2.2.3. UTCMillisecondsTimestamp-timestamp..................................................................20
7.2.2.4. OptionalEoNConfigurationandControlParameters..................................................20
7.2.2.5. IgnitionFolderStructureafteranEoNNodeBirthCertificate.....................................21
7.3. MQTTEoNNodeDataPayload(NDATA)..................................................................................22
7.3.1. MessageSequenceNumber-seq........................................................................................23
7.3.2. UTCMillisecondsTimestamp-timestamp...........................................................................23
7.3.3. Dynamicparametervaluesthathavechanged....................................................................23
7.4. MQTTDeviceBirthandDeathCertificate.................................................................................23
7.4.1. MQTTDeviceBirthCertificate(DBIRTH)...............................................................................24
7.4.1.1. MessageSequenceNumber-seq................................................................................24
7.4.1.2. UTCMillisecondsTimestamp-timestamp..................................................................24
7.4.1.3. RequiredDeviceProcessVariableMap.......................................................................24
7.4.1.1. IgnitionTagStructureafterDeviceBIRTHMessage....................................................25
7.4.2. MQTTDeviceDeathCertificate(DDEATH)...........................................................................27
7.4.2.1. MessageSequenceNumber-seq................................................................................27
7.4.2.2. UTCMillisecondsTimestamp-timestamp..................................................................27
7.4.2.3. OptionalDeathCertificateParameters........................................................................27
7.5. MQTTDeviceDataMessages(DDATA).....................................................................................28
7.5.1. MessageSequenceNumber-seq........................................................................................28
7.5.2. UTCMillisecondsTimestamp-timestamp...........................................................................28
7.5.3. AnyTag/PVvaluethathaschanged.....................................................................................28
7.6. IgnitionGatewayBirthandDeathCertificates.........................................................................29
7.6.1. IgnitionDeathCertificatePayload(STATE)...........................................................................29
7.6.1. IgnitionBirthCertificatePayload..........................................................................................29
7.7. IgnitiontoEoNNodeDataCommand(NCMD).........................................................................30
7.8. IgnitiontoDeviceDataCommand(DCMD)..............................................................................30
Page5SparkplugSpecificationVersion1.0
8. SparkplugMQTTSessionManagementandMessageFlow...............................................................32
8.1. PrimaryIgnitionSessionEstablishment....................................................................................33
8.2. EoNNodeSessionEstablishment.............................................................................................35
8.3. MQTTDeviceSessionEstablishment........................................................................................37
8.4. GeneralMQTTapplicationsandnon-primaryIgnitionGateway..............................................39
9. SparkplugMQTTDataandCommandMessages...............................................................................40
9.1. EoNNDATAandNCMDMessages............................................................................................40
9.2. DeviceDDATAandDCMDMessages........................................................................................42
10. SparkplugManagementofMultipleMQTTServers.......................................................................45
10.1. MultipleMQTTServerTopology...............................................................................................45
10.2. PrimaryIgnitionGatewaySTATEinMultipleMQTTServerTopologies....................................48
11. SparkplugPersistentversusNon-PersistentConnections..............................................................51
12. GlossaryofTerms...........................................................................................................................52
13. ContactInformation.......................................................................................................................53
Page6SparkplugSpecificationVersion1.0
TableofFiguresFigure1-IgnitionMQTTInfrastructureOverview.......................................................................................7
Figure2-MQTTSCADAInfrastructure.........................................................................................................9
Figure3-SimpleMQTTInfrastructure.......................................................................................................12
Figure4–InitialMQTTEngineTagProviderFolderStructure...................................................................15
Figure5–IgnitionMQTTEngineFolderStructureusingtheSparkplugTopicNamespace.......................15
Figure6-KuraProtocolBufferDefinition..................................................................................................18
Figure7-IgnitionTagStructureafterEoNNodeBIRTH............................................................................22
Figure8-IgnitionTagStructureafterDeviceBIRTHmessage...................................................................26
Figure9-HostSessionEstablishment........................................................................................................33
Figure10-EoNNodeMQTTSessionEstablishment..................................................................................35
Figure11-MQTTDeviceSessionEstablishment.......................................................................................37
Figure12-EoNNodeNDATAandNCMDMessageFlow...........................................................................42
Figure13-DeviceDDATAandDCMDMessageFlow.................................................................................44
Figure14-SingleMQTTServerTopology..................................................................................................45
Figure15-MultipleMQTTServerTopology..............................................................................................46
Figure16-MQTTClienttagsinIgnition.....................................................................................................47
Figure17-EoNMQTTmetrictagsinIgnition............................................................................................48
Figure18-IgnitionSTATEflowdiagram....................................................................................................49
Page7SparkplugSpecificationVersion1.0
1. IntroductionThisdocumentprovidestheopenandfreelyavailablespecificationforhowEdgeofNetworkorMQTTenabledenddevicescommunicateinMQTTInfrastructureintheInductiveAutomationIgnitionApplicationPlatform.ThedocumentdetailsthestructureandimplementationrequirementsforcompliantMQTTclientdevicestoutilizeCirrusLinksuiteofMQTTModules.ThecomponentsofthebaselineMQTTinfrastructureincludeIgnition,oneormoreMQTTServersandattheCirrusLinkMQTTEngineModule.AdescriptionoftheCirrusLinkMQTTmodulesareasfollows:
TheMQTTEngineModule:AddsthefunctionalitytotheIgnitionplatformtobidirectionalcommunicatewithMQTTenablededge-of-the-networkdevicessecurelyviaanMQTTServer.
TheMQTTDistributorModule:AddsanMQTTservertotheIgnitionplatformthatenablesMQTTclientstosecurelyconnect,publish,andsubscribetodata.
TheMQTTTransmission Module: EnablestheTAGswithinIgnitiontobepublishedasMQTTclientdatawithintheMQTTInfrastructure.ItisabridgefromIgnitionTAGstoMQTT.
Belowisasamplesolutiondiagram.
Figure1-IgnitionMQTTInfrastructureOverview
Page8SparkplugSpecificationVersion1.0
2. BackgroundMQTTwasoriginallydesignedasamessagetransportforRealTimeSCADAsystems.WithinthecontextoftheMQTTspecification,therearemechanismsdefinedtoprovidethestateofanMQTTsessiontoprovideSCADAHostapplicationsthecurrentstateofanyMQTTdevicenodeintheinfrastructure.TheMQTTspecificationdoesnotspecifytheTopicNamespaceorPayloadrepresentationofthedatabeingpublishedand/orsubscribed.Thiswasintendedtoprovideflexibilityinfuturesystemimplementations.
ThepurposeofthisdocumentistoremaintruetotheoriginalnotionofkeepingtheTopicNamespaceandmessagesizestoaminimumwhilestillmakingtheoverallmessagetransactionsbetweenMQTTDeviceNodesandMQTTEngineinstalledontheIgnitionGatewaysimple,efficientandeasytounderstandandimplement.
Page9SparkplugSpecificationVersion1.0
3. InfrastructureComponentsThissectiondetailstheinfrastructurecomponentsimplementedwithuseofInductiveAutomationIgnitionPlatformandCirrusLinkMQTTModules.
Figure2-MQTTSCADAInfrastructure
3.1. MQTTServer(s)MQTTenabledinfrastructurerequiresoneormoreMQTTServersarepresentintheinfrastructure.TheMQTTServercomponentinthearchitectureisrequiredtobecompliantwiththelatestMQTTV3.1.1specificationandissizedtoproperlymanageallMQTTmessagetraffic.
Onecanimplementtheuse(ifrequired)ofmultipleMQTTserversforthepurposeofredundancy,highavailability,andscalabilitywithinanygiveninfrastructure.
3.2. MQTTEdgeofNetwork(EoN)NodeInthecontextofthisdocument,anMQTTEdgeofNetwork(EoN)NodeisanyV3.1.1compliantMQTTClientapplicationthatmanagesanMQTTSessionandprovidesthephysicaland/orlogicalgatewayfunctionsrequiredtoparticipateintheTopicNamespaceandPayloaddefinitionsdescribedinthisdocument.TheEoNNodeisresponsibleforanylocalprotocolinterfacetoexistinglegacydevices(PLCs,RTUs,FlowComputers,Sensors,etc.)and/oranylocaldiscreteI/O,and/oranylogicalinternalprocessvariables(PVs).
3.3. DeviceTheDevicerepresentsanyphysicalorlogicaldeviceconnectedtotheMQTTEoNNodeprovidinganydata,processvariablesormetrics.
3.4. MQTTEnabledDevice(Sparkplug)Thisrepresentsanydevice,sensor,orhardwarethatdirectlyconnectstoMQTTinfrastructureusingacompliantMQTT3.1.1connectionwiththepayloadandtopicnotationasoutlinedinthisSparkplugspecification.
Page10SparkplugSpecificationVersion1.0
3.5. MQTTHostNodeTheMQTTHostNodeisanyMQTTClientapplicationthatsubscribestoPVmessagesandpublishesPVCommandmessagesdefinedinthisdocument.InmostSCADAinfrastructureimplementations,therewillbeonlyonePrimaryMQTTHostNoderesponsibleforthemonitoringandcontrolofagivengroupofMQTTDeviceNodes.ThisdoesnotprecludeanynumberofadditionalMQTTHostNodesparticipatingintheinfrastructurethatareineitherapuremonitoringmode,orintheroleofahotstandbyshouldthePrimaryMQTTHostNodegooffline.
WithintheInductiveAutomationIgnitionplatform,itbecomestheHostNodewheretheMQTTEnginemoduleenablesitsMQTTsessionsforsubscribingandpublishingdata.MQTTEoNNodesandassociatedDevicesthatfollowthisspecificationareautomaticallyrecognizedbyMQTTEngineandbecomeapartoftheIgnitiontagstructuredynamically.ThisspecificationdetailsthefolderandtagstructurecreatedforeachleveloftheTopicNamespace.
3.6. MQTTApplicationNodeAMQTTApplicationNodeisanynon-primaryMQTTClientapplicationthatconsumestherealtimePVmessagesoranyotherdatabeingpublishedwithproperpermissionandsecurity.
3.7. SecurityThereareseverallevelsofsecurityandaccesscontrolconfiguredwithinanMQTTinfrastructure.FromapureMQTTclientperspective,theclientdoesneedtoprovideauniqueClientID,andanoptionalUsernameandPassword.AlthoughaccesscontrolisnotmandatedintheMQTTspecificationforuseinMQTTServerimplementations,AccessControlList(ACL)functionalityisavailableformostMQTTServerimplementations.TheACLofanMQTTServerimplementationisusedtospecifywhichTopicNamespaceanyparticularMQTTClientcansubscribetoandpublishon.ExamplesareprovidedonhowtosetupandmanageMQTTClientcredentialsandsomeconsiderationsonsettingupproperACL’sontheMQTTServers.
TheMQTTspecificationdoesnotspecifyanyparticularTCP/IPsecurityschemeasitwasenvisagedthatTCP/IPsecuritywould(anddid)changeovertime.AlthoughthisdocumentwillnotspecifyanyTCP/IPsecurityschemaitwillprovideexamplesonhowtosecureanMQTTinfrastructureusingTLSsecurity.
Page11SparkplugSpecificationVersion1.0
4. LeveragingStandardsandOpenSourceTheSparkplugspecificationleveragesasmuchopensourcedevelopmentanddataencodingaspossibleasthisisacorebeliefwithinCirrusLinkSolutions.
4.1. OASISMQTTV3.1.1SpecificationTheSparkplugspecificationspecifiesthatMQTTClientsintheinfrastructureadheretotheMQTTV3.1.1specification.Thespecificationdocumentationrefersto“mqtt-v3.1.1-os.doc”:
http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html
AlsoreferredisanaddendumdocumenttotheMQTTV3.1.1specificationdocumentthatdiscussesbestpracticesforimplementingsecurityonMQTTTCP/IPnetworks:
http://docs.oasis-open.org/mqtt/mqtt-nist-cybersecurity/v1.0/mqtt-nist-cybersecurity-v1.0.doc
4.2. EclipseFoundationIoTResourcesTheEclipseFoundationisanexcellentresourceforopensourcesoftwaresupportindustrystandards.WithintheEclipseFoundationisanInternetofThings(IoT)workinggroupprovidingawealthofinformation.
http://iot.eclipse.org/
4.2.1. PahoPahoisanEclipseFoundationprojectthatoffersexcellentresourcesformature,compliantMQTTClientandMQTTServerimplementationsandwellasadditionalresourcesforallthingsMQTT.
http://www.eclipse.org/paho/
4.2.2. KuraKuraisanotherEclipseFoundationprojectunderIoTresources.KuraisaJava/OSGi-basedframeworkforIoTgateways,orEoNNodesasdefinedinthisspecification.KuraalsoprovidesopensourceresourcesfortheGoogleProtocolBufferrepresentationofMQTTpayloadsasdefinedinthisspecification:
https://www.eclipse.org/kura/
4.3. RaspberryPiHardwareForthesakeofkeepingtheSparkplugspecificationasrealworldaspossible,areferenceimplementationofanEoNNodeandassociatedDeviceisprovidedfortheexamplesandscreenshotsinthisdocument.AllofthiswasimplementedonRaspberryPihardwarerepresentingtheEoNNodewithaPibrellaI/OboardrepresentingtheDevice.
Page12SparkplugSpecificationVersion1.0
5. GeneralMessageFlowThissectiondiscussesthegenerictopologyshowninFigure2-MQTTSCADAInfrastructureidentifyinghoweachofthecomponentoftheinfrastructureinteracts.
Atthesimplestlevelthereareonlytwocomponentsrequiredasshownbelow.AnMQTTClientandanMQTTServer.Withpropercredentials,anyMQTTClientcanconnecttotheMQTTServerwithoutanynotionofotherMQTTClientapplicationsthatareconnected,andcanissuesubscriptionstoanyparticularMQTTmessage(data)thatitmightbeinterestedinaswellasstartpublishinganydatathatithas.ThisisoneoftheprincipalnotionsofIIoT,thatisthedecouplingintelligentdevicesfromanydirectconnectiontoanyoneconsumerapplication.
Figure3-SimpleMQTTInfrastructure
5.1. StateInanynetworkarchitecture,networkconnectionStateisimportant.InSCADA,connectionStateisextremelyimportant.StateisthesessionawarenessoftheMQTTEoNandtheMQTTServer.Theveryreasonthatthemajorityofthismarketsectorarestillusinglegacypoll/responseprotocolstomaintainanotionoftheStateoftheconnectionbetweentheSCADAapplicationandtheconnecteddevices.Ipoll,Igetaresponse,IknowtheStateofalloftheI/Opoints,butnowImustpollagainbecausethatStatemayhavechanged.
ManyimplementationsofsolutionsusingMQTTtreatitasasimple,stateless,pub/substatemachine.TheisquiteviableforIoTandsomeIIoTapplications,howeveritisnottakingadvantageofthecapabilityofMQTTbasedinfrastructures.
FortheproperimplementationofanyMQTTbasedwithinaSCADAsystem,thebuiltinsessionstatemechanismofMQTTmustbeunderstoodandproperlyimplemented.Ifreal-timecontrolorstateisnotusedforthedataandapplication,stateisnotrequiredandthesystemwilloperatebasedonusingtime-stampeddataasLKG(last-known-good)values.(CHECK)
OneoftheprimaryapplicationsforMQTTasitwasoriginallydesignedwastoprovidereliableSCADAcommunicationsoverVSATtopologies.Duetopropagationdelayandcost,itwasnotfeasibletouseapoll/responseprotocol.Insteadofapoll/responseprotocolwhereallofthedatawassentinresponsetoeverypoll,MQTTwasusedto“publish”informationfromremotesitesonlywhenthedatachanged.ThistechniqueissometimescalledReportbyExceptionorRBE.ButinorderforRBEtoworkproperlyinrealtimeSCADA,the“state”oftheenddeviceneedstobeknownatalltimes.Inotherwords,theSCADAhostapplicationslikeIgnitioncouldonlyrelyonRBEdataarrivingreliablyifitcouldbeassuredofthestateoftheMQTTsession.
Page13SparkplugSpecificationVersion1.0
6. SparkplugMQTTTopicNamespaceInordertogetaworkingMessageOrientedMiddleware(MOM)basedSCADAsystemworkingusingMQTT,thefirstthingthatmustbedefinedisaTopicNamespacetoworkwithin.ThebeautyofMQTTisthefactthatyoucanjustcomeupwithanarbitrarytopiclike“Portland/Temperature”,connecttoanMQTTServer,andstartpublishingthetemperaturevalue.InorderforthisdatatobeusefultootherMQTTClientapplicationsthatwanttoconsumethetemperaturevalues,theTopicNamespaceneedstobeunderstoodbyeveryoneparticipatinginthedataexchange.
EveryMQTTmessagepublishedconsistofatopicandapayloadcomponent.Thesecomponentsarethe“overhead”ofanMQTTmessageasmeasuredinbytesonthewire.TheSparkplugspecificationisdesignedtokeepthesecomponentsmeaningfulandeasytounderstand,butnottogetsoverboseastonegativelyimpactbandwidth/timesensitivedataexchange.
6.1. SparkplugTopicNamespaceElementsAllMQTTclientsusingtheSparkplugspecificationwillusethefollowinggeneraltopicstructure:
namespace/group_id/message_type/edge_node_id/[device_id]6.1.1. namespaceElement
ThenamespaceelementoftheTopicNamespaceistherootelementthatwilldefineboththestructureoftheremainingnamespaceelementsaswellastheencodingusedfortheassociatedpayloaddata.ForthisinitialversionoftheSparkplugspecification,theUTF-8stringconstantforthenamespaceelementwillbe:
“spAv1.0”ThiselementidentifiesthemessageasbeingaSparkplug“sp”definedmessageusingversion“A”oftheSparkplugspecificationencodingschemeversion“1.0”.
6.1.2. group_idElementThegroup_idelementoftheTopicNamespaceprovidesforalogicalgroupingofMQTTEoNandDevicesintotheMQTTServerandbackouttotheconsumingMQTTClients.Theformatofthegroup_idelementisnotdictatedinthatitcanbeanyvalidUTF-8alphanumericStringwiththeexceptionofthereservedcharactersof‘+’(plus),‘/’(forwardslash),and‘#’(numbersign).Tominimizeoverheadifthatisagoalofyourapplication,itshouldbeassmallaspossible.Examplesofwherethe[group_id]mightbeusedincludeOil/GasapplicationswhereMQTTDevicesonaphysicalpipelinesegmentallhavethesame[group_id].PlantfloorapplicationsmaygroupMQTTDevicesbasedonlogicalcellormanufacturinglinerequirements.
6.1.3. message_typeElementThemessage_typeelementoftheTopicNamespaceprovidesanindicationastowhattheMQTTPayloadofmessagewillcontain(Note:TheSparkplugMQTTPayloaddefinitionsaredefinedindetaillateristhisdocument).Thefollowingmessage_typeelementsaredefinedfortheSparkplugTopicNamespace:
• NBIRTH–BirthcertificateforMQTTEoNNodes.
Page14SparkplugSpecificationVersion1.0
• NDEATH–DeathcertificateforMQTTEoNNodes.• DBIRTH–BirthcertificateforMQTTDevices.• DDEATH–DeathcertificateforMQTTDevices.• NDATA–Nodedatamessage.• DDATA–Devicedatamessage.• NCMD–Nodecommandmessage.• DCMD–Devicecommandmessage.• STATE–Criticalapplicationstatemessage.
Thespecificationforeachofthesemessage_typeelementsaredetailedlaterinthisdocument.
6.1.4. edge_node_idElementTheedge_node_idelementoftheSparkplugTopicNamespaceuniquelyidentifiestheMQTTEoNNodewithintheinfrastructure.Theformatoftheedge_node_idcanbevalidUTF-8alphanumericStringwiththeexceptionofthereservedcharactersof‘+’(plus),‘/’(forwardslash),and‘#’(numbersign).Thetopicelementedge_node_idtravelswitheverymessagepublishedandshouldbeasshortaspossible.
6.1.5. device_idElementThedevice_idelementoftheSparkplugTopicNamespaceidentifiesanMQTTDeviceattachedtotheMQTTEoNNode.Notethatthedevice_idisanoptionalelementwithintheTopicNamespaceassomemessageswillbeeitheroriginatingordestinedtotheedge_node_idandthedevice_idwouldnotberequired.Theformatofthedevice_idisavalidUTF-8alphanumericStringwiththeexceptionofthereservedcharactersof‘+’(plus),‘/’(forwardslash),and‘#’(numbersign).Thedevice_idmustbeuniquefromotherdevicesconnectedtothesameEoNNode,butcanbeduplicatedfromEoNNodetootherEoNNodes.
6.2. ExampleIgnitionFolderStructureUponinstallingtheMQTTEnginemoduleinIgnitionandconfiguringoneormoreavailableMQTTServers,anew“TagProvider”isaddedtotheIgnitionTagstructurenamed“MQTTEngine”.UntilEdgeofNetworkdevicesjointheMQTTinfrastructure,theMQTTEnginestructureprimarilyprovidesmetricsontheMQTTEngineinternalMQTTClientsconnectedtothedefinedMQTTServers.
Page15SparkplugSpecificationVersion1.0
Figure4–InitialMQTTEngineTagProviderFolderStructure
ButonceanEoNNodeconnectstotheMQTTinfrastructure,MQTTEnginestartsusingtheTopicNamespacedefinedtobuildouttheIgnitionfolderstructurerepresentingEoNNodes,Devices,andthevariousmetric,parameter,andprocessvariablefolders.
Figure5–IgnitionMQTTEngineFolderStructureusingtheSparkplugTopicNamespace
Page16SparkplugSpecificationVersion1.0
7. SparkplugMQTTPayloadsTheMQTTmessagetransportspecificationdoesNOTdefineanydatapayloadformat,butjustlikeintheTopicNamespacesectionnamespacedesigndoeshavetotakeplace.ThesameistrueforthedatapayloadsbeingpublishedandsubscribedtobytheparticipatingMQTTclientsintheinfrastructure.ThissectionoftheSparkplugspecificationdefineshowanMQTTpayloadisencodedandthedatathatisrequired.NotethatSparkplugcansupportmultiplepayloadsdefinitions.TheinitialreleaseisbasedontheopensourceEclipseKura/GoogleProtobufimplementation.
SparkplugisaddressingtheabsolutefactthatthemajorityofdevicesconnectingintonextgenerationIIoTinfrastructureislegacyequipmentusingpoll/responseprotocols.ThismeanswemusttakeinaccountregisterbaseddatafromdevicesthattalkprotocolslikeModbus.TheexistinglegacyequipmentneedstoworkinconcertwithemergingIIoTequipmentthatareabletoleveragemessagetransportslikeMQTTnatively.
7.1. SparkplugMQTTPayloadEncodingTheSparkplugspecificationcreatesabandwidthefficientdatatransportforrealtimedevicedata.ForWANbasedSCADA/IIoTinfrastructuresthisequatestolowerlatencydataupdateswhileminimizingtheamountoftrafficandthereforecellularand/orVSATbandwidthrequired.ForLANbasedSCADAinfrastructures,thisequatestoenablinghigherthroughputofrealtimedatatoconsumerapplicationswithoutrequiringextremenetworkingtopologiesand/orequipment.
ThereisaplethoraofdataencodingtechnologiesavailablethatcanALLbeusedinconjunctionwithMQTT.ConventionalITtoolingprovidesXMLandJSONasthemostpopularencodingtechnologies.Butneitheroftheseencodingtechnologiesareefficientinrepresentinglargequantitiesofregisterbased,realtimeprocessvariables.ThesetechnologiesallhavetheirplaceinITdesignstrategies.Sparkplugselectedanexisting,open,andhighlyavailableencodingschemethatefficientlyencodesregisterbasedprocessvariables.TheencodingtechnologyselectedforSparkplugisGoogleProtocolBuffersalsoreferredtoasGoogleprotobufs.
7.1.1. GoogleProtocolBuffers“ProtocolBuffersareawayofencodingstructureddatainanefficientyetextensibleformat.”
GoogleProtocolBuffers,sometimesreferredtoas“Googleprotobufs”,providetheefficiencyofpackedbinarydataencodingwhileprovidingthestructurerequiredtomakeiteasytocreate,transmit,andparseregisterbasedprocessvariablesusingastandardsetoftools.GoogleProtocolBuffersdevelopmenttoolsareavailablefor:
• C• C++• C#• Java• Python• GO• JavaScript
AdditionalinformationonGoogleProtocolBufferscanbefoundat:
Page17SparkplugSpecificationVersion1.0
https://developers.google.com/protocol-buffers/
7.1.2. LeveragingExistingKuraProtocolBufferSupportAlthoughtheSparkplugspecificationcouldarbitrarilydefineaGoogleProtocolBufferdefinitiontodefineMQTTpayloads,itisthegoalofthisversion“A”ofthespecificationistofollowopensourceandexistingstandardswhereverpossible.TheEclipseSoftwareFoundationsponsorsamatureInternetofThingsGatewayprojectcalledKura(seethetechnicalreferencesectionattheendofthisdocumentformoreinformation).TheKuraprojecthasalreadydefinedaGoogleProtocolBuffersschemathattargetsthetypeofmachineProcessVariableinformationthatwewanttohaveintheSparkplugMQTTmessagepayloads.TheProtocolBufferschemacanbefoundatthefollowingURL:
https://github.com/eclipse/kura/blob/KURA_1.4.0_RELEASE/kura/org.eclipse.kura.core.cloud/src/main/protobuf/kurapayload.proto
Page18SparkplugSpecificationVersion1.0
Figure6-KuraProtocolBufferDefinition
TheimportantthingtonoteinthisProtocolBufferschemaisthattherequiredparametersarethenameandtypeparameters.FormostusecaseapplicationsinSCADA,thinkofthisschemaprovidinganefficientwaytoencodekey:valuepairsofatag(orPLCregisternumber)andanassociatedvalue.ThevalueenumerationsherecoveralloftherequireddatatypesthatProcessVariablesrequireinclude:
• DOUBLE• FLOAT• INT64• INT32• BOOL• STRING• BYTES
TherearesomemetadatastructuresdefinedwithinthecurrentKuraProtocolBuffersschemathatcanbeutilized(optionally)aswell.ThisincludestheINT64timestampthatusedtotimestampUTCmillisecondvaluesaswellastheKuraPositionstructureforanyLocationBasedservicesthatmightbeprovidedbysomeMQTTEoNNodesorDevices.
7.2. MQTTEoNBirthandDeathCertificateAcriticalaspectforMQTTinarealtimeSCADAapplicationismakingsurethattheprimaryMQTTHostNodeisabletoknowthe“STATE”ofanyEoNand/orDeviceintheinfrastructurewithintheMQTTKeepAliveperiod(refertosection3.1.2.10intheMQTTSpecification).ToimplementthestateaknownWillTopicandWillMessageisdefinedandspecified.TheWillTopicandWillMessageregisteredintheMQTTCONNECTsessionestablishment,collectivelymakeupwhatwearecallingtheDeathCertificate.NotethatthedeliveryoftheDeathCertificateuponanyMQTTclientgoingofflineunexpectedlyispartoftheMQTTprotocolspecification,notpartofthisSparkplugspecification(refertosection3.1CONNECTintheMQTTSpecificationforfurtherdetailsonhowanMQTTSessionisestablishedandmaintained).
UnliketheDeathCertificatemechanismwhichispartoftheMQTTtransport,theBirthCertificateisaSparkplugdefinition.TheBirthCertificateisalogicalreciprocaloftheDeathCertificatethatisusedtoconveythefactthattheassociateMQTTEoNNodeand/orMQTTDevicenowhasanMQTTsessionestablishedandcannowstartprovidingrealtimeProcessVariableinformation.
ThefirstMQTTmessagethatanEoNNodeMUSTpublishuponthesuccessfulestablishmentofanMQTTSessionisanEoNBIRTHCertificate.
7.2.1. EoNDeathCertificate(NDEATH)TheDeathCertificatetopicforanMQTTEoNNodeis:
namespace/group_id/NDEATH/edge_node_id
TheDeathCertificatetopicandpayloaddescribedherearenot“published”asanMQTTmessage,butusedasprovidedparameterswithintheMQTTCONNECTcontrolpacketwhenthisMQTTEoNNodefirstestablishestheMQTTClientsession.
Page19SparkplugSpecificationVersion1.0
ImmediatelyuponreceptionofanEoNDeathCertification,MQTTEnginewillsettheassociatedIgnitiontagdataqualityto“STALE”forALLtagsandparametersassignedtoanyMQTTDeviceconnectedtothisEoNNode.
TheMQTTpayloadassociatedwiththistopicwillbeaGoogleProtocolBufferusingtheKuraschemacontainingthefollowingoptionalparameters:
7.2.1.1. Birth/DeathSequenceNumber-bdseq‘bdseq’:INT32:[sequence_number] //Birth/DeathSequenceNumber
TheBirth/DeathsequencenumberisarequiredSparkplugspecificationfeaturethatisutilizedwhentheinfrastructurecontainsmultipleMQTTServers.ThebdseqmetricvalueMUSTbethesamevalueusedintheassociatedBirthCertificatepayloaddefinedbelow,andMUSTbeanincrementingvalueonallsubsequentMQTTsessionestablishments.
7.2.2. EoNBirthCertificate(NBIRTH)TheBirthCertificatetopicforanMQTTEoNNodeis:
namespace/group_id/NBIRTH/edge_node_id
TheEoNBirthCertificatecontainseverythingrequiredtobuildoutanIgnitiontagtreestructureforallmetricsandparametersforthisEoNNode.UponthereceptionofavalidEoNBirthCertificate,MQTTEnginecreatesthetagfolderinformationrequiredtorepresenttheEoNNodeifitdoesnotalreadyexist,andpopulatetheprovidedmetricinformationintotheappropriatefolders.TheONLINEstateofthisEoNNodewillbesettoTRUEalongwiththeassociatedONLINEDateTimeparameter.NotethattheEoNBirthCertificateONLYindicatesthenodeitselfisonlineandinanMQTTSession,butanyattachedMQTTDeviceswillstillhave“STALE”dataqualityuntilthosedevicescomeonlinewiththeirassociatedBirthCertificates.
TheMQTTpayloadassociatedwiththistopicwillbeaGoogleProtocolBufferusingtheKuraschemacontainingthefollowingoptionalparameters:
7.2.2.1. Birth/DeathSequenceNumber-bdseq‘bdseq’:INT32:[session_number] //Birth/DeathSequenceNumber
TheBirth/DeathsequencenumberisarequiredSparkplugspecificationfeatureutilizedwhentheinfrastructurecontainsmultipleMQTTServers.ThebdseqmetricvalueMUSTbethesamevalueusedintheassociatedDeathCertificatepayloaddefinedabove,andMUSTbeanincrementingvalueonallsubsequentMQTTsessionestablishments.
7.2.2.2. MessageSequenceNumber-seq‘seq’:INT32:[message_sequence_number] //MessageSequenceNumber
TheMessageSequenceNumberisarequiredparameterthatguaranteespropermessageorderingbetweenanMQTTEoNNodeandtheSCADA/Hostapplication.ForrealtimesolutionswhereProcessVariablesarebeingpublishedusingReportbyException(RBE),itiscriticalthatallanalogandstateProcessVariablesarriveinorder.TheMQTTspecificationstatethatallmessagesbedeliveredinthe
Page20SparkplugSpecificationVersion1.0
ordertheyarepublished.ButtheMessageSequenceNumberprovidesanadditionlevelofmonitoringandsafetyininstanceswherethismightnotbethecase.ThevalueoftheMessageSequenceNumberstartsatavalueofzero(0)withthisNBIRTHCertificatemessageandincrementoneverysubsequentMQTTmessagesfromthisEoNNodeandeachsubsequentmessageincrementstheMessageSequenceNumberbyavalueofexactlyone(1)andincrementtoavalueof255,andthenrolloverto0again.AnyprimaryMQTTSCADAHostnodemonitorsthereceivedMessageSequenceNumber,andifanoutofordermessageisdetected,thenALLProcessVariabledatashouldbesetto“STALE”qualityandtheSCADAHostnodewillrequestanewMQTTSessionfromthisEoNNode.
7.2.2.3. UTCMillisecondsTimestamp-timestamp‘timestamp’:INT64:[current_utc_timestamp] //UTCMillisecondsTimestamp
UTCMillisecondsTimestampisanoptionalparameterthatcanbeprovidedintheEoNBirthCertificate.AlthoughanyMQTTHostnodecan(andshould)timestampdataonarrival,UTCtimestampsintheMQTTmessagescanprovideimportantlatencymetricstomonitortheoverallhealthoftheMQTTinfrastructure.InmanySCADAsystemimplementations,allinfrastructurecomponentshaveaccesstothesamemastertimeserver(forexampleusingamasterNTPserver).IncludingtheUTCMillisecondTimestampwitheveryMQTTmessagepublishedallowstheMQTTHostnodetheabilitytoproviderealtimelatencymetricsfromeveryMQTTEoNnodeinthesystem.ThislatencymetricmeasuresthetimeforthemessagepublishedfromtheEoNnodetraversingtheinfrastructureandarriveattheSCADAhost.ForcellularandVSATbasedsystems,thisisanimportantmetrictokeeptrackof.ItcanalsobeusedtoensurethattherearenoissueswiththemessagesgettingthrutheMQTTServerinfrastructure.
7.2.2.4. OptionalEoNConfigurationandControlParametersEachMQTTEoNNodecanprovideanoptionalnumberofmetricsandparametersforadditionalinformationaboutthenode.MQTTEngineplacestheseparametersintotheEoNNodetagfolderstructurewitheachparameterexposedasanIgnitiontag.ThedifferencebetweentheEoNNodeconfigurationandcontroltagsinthissectionversustheprocessvariabletagsdetailedlaterinthissectionisthatthesetagsusetheNDATAandNCMDmessagetypesandcanbesegregatedfromphysicaldevicecommand/controlaccess.ThisisanimportantaspectoftheSparkplugspecificationtounderstand.Usingpropercredentialandaccessmanagement,usersgainaccesstotheconfigurationparametersofanEoNNodewithouthavingaccesstothephysicaldevicecommand/controlDCMDmessages.
InformationalparametersliketheHardwareManufactureID,ModelNumber,SoftwareRevisionNumber,ConfigurationRevisionNumber,etc.canbeprovidedherealongwithanyEoNNodecontrolorconfigurationparameter.Byusinganappropriateparameterstructurehere,completeconfigurationdashboardscanbebuiltwiththeIgnitiontooling.UsingtheRaspberryPiexamplecode(https://github.com/Cirrus-Link/Sparkplug),thefollowingEoNNodefolderstructureandtagsareexposed:
//Rootleveltag‘UpTimems’:INT64:[upTimeStart] //UpTimeinms//NodeControlfoldertags‘NodeControl/Rebirth’:BOOL:false //Rebirthcommandcoil‘NodeControl/Reboot’:BOOL:false //Rebootcommandcoilintothe
Page21SparkplugSpecificationVersion1.0
‘NodeControl/Nextserver’:BOOL:false //MovetonextMQTTServercommandcoil‘NodeControl/scan_rate_ms’:INT32:250 //I/Oscanrateinmilliseconds//Propertiesfoldertags‘Properties/NodeManf’:STRING:’Raspberry’ //Manufacture‘Properties/HardwareVersion’:STRING:’Pi2ModelB’ //HWVersion‘Properties/SoftwareVersion’:STRING:’v1.0.0’ //SWVersion‘Properties/ConfigChangeCount’:INT32:1 //PersistentconfigurationchangecounterNotethatthetagfolderhierarchyasitappearsinIgnitioncanbecontrolledbythestructureoftheBirthCertificatetagnames.Intheexamplegivenabove,the“UpTimems”tagwillappearattherootleveloftheNode.TagsthatcontrolaspectsoftheEoNNodeareplacedintoafoldercalled“NodeControl”.Other,informationonlytagsareshownabovebeingplacedintothe“Properties”folder.TheonlyEoNNodetagstructurethatismandatoryintheSparkplugspecificationisthe“NodeControl/Rebirth”nodecontrolBoolean.AnyMQTTClientthatconnectstotheMQTTServerinfrastructurewillwanttorequestthatanewBirthCertificateneedstobepublishedinordertolearnaboutthenodeandassociatedDevices.MQTTEnginewillassumethiscontroltagexistsforanyEoNNodeconnectingintotheinfrastructure.
7.2.2.5. IgnitionFolderStructureafteranEoNNodeBirthCertificateUsingtheactualparametersgivenintheexamplesabove,thereferenceRaspberryPicodecanpublishanEoNNodeBirthCertificateonthefollowingMQTTtopic:
spA1.0/SparkplugDevices/NBIRTH/JavaRaspberryPi
Uponreceiptofthismessage,MQTTEnginebuildsoutthefollowingfolder/tagstructurewithinIgnition:
Page22SparkplugSpecificationVersion1.0
Figure7-IgnitionTagStructureafterEoNNodeBIRTH
AftertheEoNNodeBirthCertificatehasbeenpublished,alloftheparametersdefinedintheBIRTHmessagepayloadarenowavailableasIgnitiontags.TheseparametertagshavethesamepropertiesasanyotherIgnitiontagvalue.Asshowninthetagbrowserviewabove,theEoNNodeisonlineandprovidesspecificinformationaboutthenodethatcanbedisplayedindashboardviewsfortheenduser.Variousnodecommandsarenowavailablelikesendinganewscan_rate_msparametertoreadtheassociatedI/Otanewmillisecondperiod.CommandsliketheRebootnodeshownabovecanbedynamicallybuiltintothisstructureveryquickly.MetricslikethecurrentnodeTCP/IPaddressandtheConfigurationChangeCountercanbeusedforalarminganddiagnosticswithinIgnition.
7.3. MQTTEoNNodeDataPayload(NDATA)OnceanMQTTEoNNodeisonlinewithproperBirthCertificateswe’reinamodeofquiescentReportbyException(RBE)ortimebasedreportingofanyparameterinformationthatchanges.ThisenablestheadvantageofthenativeContinuousSessionAwarenessofMQTTtomonitortheSTATEofallconnectedMQTTEoNNodedevicesandtorelyonReportbyException(RBE)messagesforanyparameterormetricstatechangeovertheMQTTsessionconnection.
Asdefinedabove,theDataTopicforanMQTTEoNNodeis:
Page23SparkplugSpecificationVersion1.0
namespace/group_id/NDATA/edge_node_id
ThepayloadfortheDatamessageisaGoogleProtocolBufferstructureusingtheKuraschema.
7.3.1. MessageSequenceNumber-seq‘seq’:INT32:[message_sequence_number] //MessageSequenceNumber
TheMessageSequenceNumberisarequiredparameterthatguaranteespropermessageorderingbetweenanMQTTEoNNodeandtheHostapplication.
7.3.2. UTCMillisecondsTimestamp-timestamp‘timestamp’:INT64:[current_utc_timestamp] //UTCMillisecondsTimestamp
UTCMillisecondsTimestampisanoptionalparameterandifprovidedisthecurrentUTCtimestampvalueatthepointthismessageispublished.
7.3.3. DynamicparametervaluesthathavechangedAnyparametervaluethathaschangedvalueisaddedtothisstructureinthesameformatusedfortheparameterlistintheEoNBirthCertificateabove:
‘TagName’:ParameterDataType:[CurrentParameterValue]
FromtheexampleEoNBirthCertificateabove,anyparametersthatchangevaluesispublishedandplacedintothecorrespondingtagsinIgnitionoranyotherinterestedMQTTclientapplication.Forexample,ifsomeoneweretoupdatetheconfigurationontheexampleRaspberryPiexampleabovethefollowingparameterswouldlikelychangevalue.
‘Properties/SoftwareVersion’:STRING:’v1.0.1’ //SWVersionchangedto1.0.1‘Properties/ConfigChangeCount’:INT32:2 //Persistentconfigurationchangecounter
ThenewvalueswouldbepublishedinaNDATAmessagewiththetopicof:
spAv1.0/SparkplugDevice/NDATA/JavaRaspberryPi
ThecorrespondingtagvaluesintheParameterfoldershowninFigure7-IgnitionTagStructureafterEoNNodeBIRTHabovewouldbedynamicallyupdatedwiththenewsoftwareversionnumberandthenewconfigurationchangecounter.
7.4. MQTTDeviceBirthandDeathCertificateSparkplugspecifieshowaMQTTEoNNodeusestheMQTTtransportlayerDeathCertificatealongwiththeSparkplugdefinedBirthCertificate.MQTTDeviceBirthandDeathcertificatesaredefinedandmanagedbytheSparkplugspecificationi.e.theyareapplicationlevelpayloadspublishedbytheEoNNodeandarenotpartoftheMQTTtransportspecification.
Page24SparkplugSpecificationVersion1.0
7.4.1. MQTTDeviceBirthCertificate(DBIRTH)TheTopicNamespaceforaBirthCertificateforanMQTTdeviceis:
namespace/group_id/DBIRTH/edge_node_id/device_id
TheMQTTEoNNodeisresponsibleforthemanagementofallattachedphysicaland/orlogicalMQTTDevices.OncetheEoNNodehaspublisheditsBirthCertificate,MQTTEngineensuresthattheIgnitiontagfolderstructurehastheEoNnodeinanONLINEstate.Buteachphysicaland/orlogicaldeviceconnectedtothisnodewillstillneedtoprovidethisBirthCertificatebeforeMQTTEnginecreates/updatesthetagfolderstructure(ifthisisthefirsttimethisdevicehasbeenseen)andsetanyassociatedProcessVariabletagsinIgnitiontoa“GOOD_DATA”state.
7.4.1.1. MessageSequenceNumber-seq‘seq’:INT32:[message_sequence_number] //MessageSequenceNumber
TheMessageSequenceNumberisaparameterthatguaranteespropermessageorderingbetweenanMQTTEoNNodeandtheHostapplication.
7.4.1.2. UTCMillisecondsTimestamp-timestamp‘timestamp’:INT64:[current_utc_timestamp] //UTCMillisecondsTimestamp
UTCMillisecondsTimestampisanoptionalparameterandifprovidedisthecurrentUTCtimestampvalueatthepointthismessageispublished.
7.4.1.3. RequiredDeviceProcessVariableMapThefolder/tagstructureforDeviceprocessvariablesandmetricsisidenticaltothewaythatEoNNodeparametersarebuilt.ThisisacriticalpartoftheoverallSparkplugspecificationasitprovidesallofthetaginformationandcurrentstatethatMQTTEnginerequirestobuildoutthetagstructureinIgnitionandprovidecurrentProcessVariableinformation.EachtagisametricentryintheGoogleProtocolBufferstructureusingtheKuraschema.Eachentryprovidesthefollowingdataforeachtag:
‘TagName’:PVDataType:[CurrentPVValue]
Withthisinformation,MQTTEnginecreatesthetagstructurewithinIgnition(ifthisisthefirsttimethisdevicehasconnected)foreachtag.NotethatinmostimplementationsthisistheonlytimethatALLtagsarepublished.SinceSparkplugisleveragingthe“ContinuousSessionAwareness”ofMQTTusingtheBirth/DeathCertificates,onceallcurrenttagvaluesarepublished,subsequentdatamessagescansafelyreportonlytagvaluesthathavechangedwithintheMQTTsession.
TagNamesintheSparkplugspecificationcanbedefinedinasimpleflattagstructurewherealloftheprocessvariabletagsappearbelowtheassociatedDeviceinthetaghierarchy.Butinmanycasesitmakessensetoorganizethetagsinahierarchicalfolderstructuretomakeiteasiertoviewandmanagethetags.The“TagName”parameterintheBirthCertificatepayloadcanprovideahierarchicaltagpathandMQTTEnginewillcreatetherequiredfolderstructurewithinIgnition.FortheRaspberryPireferenceimplementation,thefollowingdevicefolderstructureandprocessvariabletagsdemonstratesbothaflattagstructureandwellascreatingmorecomplextagpathstobetterorganizetaggroups:
//PlacesometagsattherootleveloftheDevicetagstructure‘button’:BOOL:[currentbuttoninputstate] //PibrellaButtoninput
Page25SparkplugSpecificationVersion1.0
‘buttoncount’:INT32:[currentcount] //Internalcounterforbuttonpresses‘buttoncountsetpoint’:INT32:[setpoint] //Read/writesetpointformaxbuttoncount‘buzzer’:BOOL:false //Pibrellabuzzer//Createafoldercall“Inputs”forthePibrelladigitalinputprocessvariables.
‘Inputs/a’:BOOL:[currentinputstate] //Pibrelladigitalinput_astate‘Inputs/b’:BOOL:[currentinputstate] //Pibrelladigitalinput_bstate‘Inputs/c’:BOOL:[currentinputstate] //Pibrelladigitalinput_cstate‘Inputs/d’:BOOL:[currentinputstate] //Pibrelladigitalinput_dstate//Createafoldercalled“Ouputs”forthePribrelladigitalouputs‘Outputs/e’:BOOL:[currentoutputstate] //Pibrelladigitaloutput_estate‘Outputs/f’:BOOL:[currentoutputstate] //Pibrelladigitaloutput_fstate‘Outputs/g’:BOOL:[currentoutputstate] //Pibrelladigitaloutput_gstate‘Outputs/h’:BOOL:[currentoutputstate] //Pibrelladigitaloutput_hstate//CreateasubfolderunderOutputscalledLEDsforassociatedLEDcontrol/state‘Outputs/LEDs/green’:BOOL:[currentLEDstate] //PibrellaGREENLEDstate‘Outputs/LEDs/red:BOOL:[currentLEDstate] //PibrellaREDLEDstate‘Outputs/LEDs/yellow:BOOL:[currentLEDstate] //PibrellaYELLOWLEDstate
[*Note:ItistheresponsibilityoftheEoNDeviceapplicationto“upcast”unsignedintegervaluestothenextintegerresolutionthatIgnitiontagssupport.Forexample,aModbusunsigned16-bitintegervaluewouldbeupcasttoasigned32bitsothatthefullresolutionoftheunsignedintegervaluecanberepresented.If32-bitunsignedintegerprocessvariablesarepresent,accumulatororcountervaluesforexample,thesewouldneedtobe“upcast”to64-bitintegervalues.]
ANYfolder/tagvaluecanbepopulatedintheTagMap.ThemapaboverepresentsphysicalandcalculatedRaspberryPiI/OvaluesbutotherrealtimetagvaluescalculatedbytheMQTTEoNNodeconnectedtothedevicecanbeaddedtothisstructureaswell.
OncetheMQTTDeviceBirthCertificateisdeliveredtoMQTTEngine,thephysical/logicaldeviceisconsideredonlineandabletodeliverdiscretetagdataupdatesinrealtimeusingtheDDATAmessagetopicandreceivecommandsusingtheDCMDmessage.
7.4.1.1. IgnitionTagStructureafterDeviceBIRTHMessageUsingtheRaspberryPireferenceimplementationswiththeparameterandprocessvariablemappingsaboveaDeviceBirthCertificateispublishedonthetopic:
spA1.0/SparkplugDevices/DBIRTH/JavaRaspberryPi/Pibrella
Uponreceiptofthismessage,MQTTEngineautomaticallybuildsoutthefollowingtagstructureinIgnition:
Page26SparkplugSpecificationVersion1.0
Figure8-IgnitionTagStructureafterDeviceBIRTHmessage
OncetheinitialtagstructureisbuiltoutinIgnitionfromtheinformationintheDeviceBirthCertificate,anysubsequentprocessvariablechangesarepublishedusingtheDDATAmessagestructuredetailedinsection0below.
Page27SparkplugSpecificationVersion1.0
7.4.2. MQTTDeviceDeathCertificate(DDEATH)Asdefinedabove,theDeathCertificateforanMQTTdeviceis:
namespace/group_id/DDEATH/edge_node_id/device_id
Intypicalusecasescenarios,itistheresponsibilityoftheMQTTEoNNodetoindicatetherealtimestateofeitherphysicallegacydeviceusingpoll/responseprotocolsand/orlocallogicaldevices.Regardlessofthedevicebeingmonitoredandprovidingtherealtimetaginformation,intheeventthatanytaginformationcouldbebadorquestionable,itistheresponsibilityoftheEoNNodetopublishaDeathCertificateonbehalfoftheenddevice.
Formostusecases,thereceptionofaMQTTDeviceDeathcertificateonlyrequirestheoptionalparametersoftheMessageSequenceNumber(seq)andtheUTCMillisecondTimestamp(timestamp).
7.4.2.1. MessageSequenceNumber-seqMessageSequenceNumber–‘seq’:INT32:[message_sequence_number]
TheMessageSequenceNumberisarequiredparameterthatguaranteespropermessageorderingbetweenanMQTTEoNNodeandtheHostapplication.
7.4.2.2. UTCMillisecondsTimestamp-timestampUTCMillisecondsTimestamp–‘timestamp’:INT64:[current_utc_timestamp]
UTCMillisecondsTimestampisanoptionalparameterandifprovidedisthecurrentUTCtimestampvalueatthepointthismessageispublished.
7.4.2.3. OptionalDeathCertificateParametersWheretheusecasedictates,optionalDeathCertificateparameterscanbeprovidedforanyextradetailsthatmightberequiredforwhytheDeviceDeathoccurred.Forexample,intypicalusecasescenariosthelegacypoll/responsedevicebeingmonitoredtimedouton“N”numberofpoll/responseattemptstheMQTTpayloadfortheDeathcouldinclude:
‘death_cert_cause’:STRING:’NoResponse’ //Noresponsefromenddevice
or:
‘death_cert_cause’:STRING:’CRCError’ //CRCErrorfromenddevice
Page28SparkplugSpecificationVersion1.0
7.5. MQTTDeviceDataMessages(DDATA)OnceanMQTTEoNNodeandassociatedMQTTDevicesareallonlinewithproperBirthCertificateswe’reinamodeofquiescentReportbyException(RBE)reportingofanyPVinformationthatchanges.ThistakesadvantageofthenativeContinuousSessionAwarenessofMQTTtomonitortheSTATEofallconnectedMQTTEoNNodedevicesandcanrelyonReportbyException(RBE)messagesforanyPVstatechangeovertheMQTTsessionconnection.
Asdefinedabove,theDataTopicforanMQTTdeviceis:
namespace/group_id/DDATA/edge_node_id/device_id
ThepayloadfortheDatamessageisaGoogleProtocolBufferstructureusingtheKuraschema.
7.5.1. MessageSequenceNumber-seq‘seq’:INT32:[message_sequence_number] //MessageSequenceNumber
TheMessageSequenceNumberisarequiredparameterthatguaranteespropermessageorderingbetweenanMQTTEoNNodeandtheHostapplication.
7.5.2. UTCMillisecondsTimestamp-timestamp‘timestamp’:INT64:[current_utc_timestamp] //UTCMillisecondsTimestamp
UTCMillisecondsTimestampisanoptionalparameterandifprovidedisthecurrentUTCtimestampvalueatthepointthismessageispublished.
7.5.3. AnyTag/PVvaluethathaschangedAnytag/PVvaluethathaschangedvaluecanbeaddedtothisstructureinthesameformatusedforthePVMapintheDeviceBirthCertificateabove:
‘TagName’:PVDataType:[CurrentPVValue]
UsingtheRaspberryPireferenceimplementationexamplegivenabove,ifanyofthePibrellaprocessvariablevalueschange,thenaDDATAmessageiscreatedandpublished.Forexample,ifthebuttononthePibrellaboardwerepressedthenthefollowingmessagewouldbepublished:
spA1.0/SparkplugDevice/DDATA/JavaRaspberryPi/Pibrella[‘button’:BOOL:true]
AsmanyprocessvariablescanbepublishedasrequiredwithintheDDATAmessagepayload.ThesimpleexampleaboveshowsonlythebuttonstateencodedintotheGoogleProtocolBufferpayload.Butifalloftheinputschangedatthesametimethenthepayloadcouldcontainallfouroftheinputstates:
‘Inputs/a’:BOOL:[newstate]‘Inputs/b’:BOOL:[newstate]‘Inputs/c’:BOOL:[newstate]‘Inputs/d’:BOOL:[newstate]
spAv1.0/SparkplugDevices/DDATA/JavaRaspberryPi/Pibrella[protobufofallinputstates]
Page29SparkplugSpecificationVersion1.0
7.6. IgnitionGatewayBirthandDeathCertificatesIninfrastructureswheremultipleMQTTServersprovideredundancyandscalability,theMQTTEoNNodesneedtobeawareofthe“state”ofthePrimaryHost.ThisisaccomplishedwithauniquesetofBirth/DeathCertificatesthattheHostMQTTClientMUSTpublishwhenanewMQTTsessionisobtained.InthesamemannerusedfortheMQTTEoNNodeBirth/Deathcertificategeneration,theHostBirth/DeathcertificateswilluseacombinationofthebuiltinMQTTWillTopicandWillPayloadDeathCertificateinconjunctionwithanapplicationlevelBirthCertificatethatispublishedasanMQTTmessage.
ThetopicthataHostnodewilluseisidenticalforboththeBirthandtheDeathcertificate.TheTopicNamespacewillbe:
STATE/scada_host_id
ItusesanaspectoftheMQTTtransportcalleda“RETAINED”publishinordertomaintainthecurrentstateofthePrimaryHostMQTTClientsessionstatetoallavailableMQTTServers.HowtheHoststateisusedisexplainedindetailinthesectiononSparkplugMQTTSessionManagement.
7.6.1. IgnitionDeathCertificatePayload(STATE)WhentheprimaryIgnitionMQTTclientestablishesanMQTTsessiontotheMQTTServer(s),theDeathCertificatewillbepartoftheWillTopicandWillPayloadregisteredintheMQTTCONNECTtransaction.TheWillTopicasdefinedabovewillbe:
STATE/scada_host_id
TheWillPayloadwillbetheSTRING“OFFLINE”.
TheWillRETAINflagwillbesettoTRUE,andtheWillQoSwillbesetto0.
Withtheseparametersset,theMQTTClientfortheHostisanMQTTmessagepublishedonthetopicofscada_host_id/STATE,withastringpayloadof“OFFLINE”anditwillbeaRETAINEDMQTTmessage.
7.6.1. IgnitionBirthCertificatePayloadThefirstmessageaprimaryIgnitionGatewayMUSTpublishisaBirthCertificate.TheIgnitionDeathCertificateisregisteredabovewithintheactualestablishmentoftheMQTTsessionandispublishedasapartofthenativeMQTTtransportiftheMQTTsessionterminatesforanyreason.
TheBirthCertificatethatisdefinedhereisanapplicationlevelmessagepublishedbytheHostMQTTClientapplications.
ThetopicusedfortheHostBirthCertificateisidenticaltothetopicusedfortheDeathCertificate:
STATE/scada_host_id
TheBirthCertificatePayloadistheSTRING“ONLINE”.
TheRETAINflagfortheBirthCertificateissettoTRUE,andtheQualityofService(QoS)issetto0.
Page30SparkplugSpecificationVersion1.0
7.7. IgnitiontoEoNNodeDataCommand(NCMD)TheIgnitionGatewaytoEoNNodecommandtopicaboveprovidestheTopicNamespaceusedtosendcommandsfromtheIgnitionGatewaytoanyconnectedEoNNodes.FromanMQTTEngineperspective,thismeanssendinganupdatedtagvaluetoanassociatedparametermappedintheEoNBirthCertificateparameterslist.
namespace/group_id/NCMD/edge_node_id
TheMQTTPayloadisencodedusingthesameGoogleProtocolBuffersKuraSchemadefinedabove.
‘TagName’:PVDataType:[new_PV_value]
ThisbecomesaVERYconvenientmechanismforthedisplayofconfigurationparametersandcontrolsthatmightbeonvariousEoNdevicesorevenaddingnewonesdynamically.IntheexamplegivenabovefortheEoNNodeBirthCertificate,someEoNNodecontrolsprovidedcannowbewrittentousingthismessage.Byclickingonthe‘Nextserver’coilintheIgnitiontagstructure,aKuraencodedpayloadispublishedtocausetheEoNNodetodisconnectfromthecurrentMQTTServerandwalktothenextoneinitsconfigurationtableshowninthefollowingstructureoftopic[payload]:
//SendcommandtowalktonextavailMQTTServerspAv1.0/Group_001/NCMD/EoN_001[‘Nextserver’:BOOL:true]AnytimeanewMQTTApplicationclientjoinstheMQTTinfrastructureitmightwanttoseeacompletelynewsetofBirthCertificatesfromthisEoNNodetogetinsyncwithallavailabledata.Usingthepropergroup_idanddevice_idinthetopicstructure,acommandmessageissenttocausetheEoNNodetoreissueallBirthCertificateinformationas:
//SendcommandtoReissueallEoNandDeviceBirthCertsspvA1.0/Group_001/NCMD/EoN_001[‘Rebirth’:BOOL:true]
WithproperpermissionsinIgnition,thesystemcouldwritetotheRebootcoilmappedoutintheexampleabovetocausetheremoveEoNNodetoreboot:
//RebootthetargetEoNNodespvA1.0/Group_001/NCMD/EoN_001[‘Reboot’:BOOL:true]
7.8. IgnitiontoDeviceDataCommand(DCMD)TheIgnitionGatewaytoDevicecommandtopicaboveprovidestheTopicNamespaceusedtosendcommandsfromIgnitiontoanyconnectedDevice.FromanMQTTEngineperspective,thismeanssendinganewtagvaluetoanassociatedparametermappedintheDeviceBirthCertificateparametersortaglist.
namespace/group_id/DCMD/edge_node_id/device_id
TheMQTTPayloadisencodedusingthesameGoogleProtocolBuffersKuraSchemadefinedabove.
‘TagName’:PVDataType:[new_PV_value]
Page31SparkplugSpecificationVersion1.0
UsingtheMQTTDevicetagsmapprovidedintheDeviceBirthCertificateexampleabove,commandscanbeissuedtoanywritablePVwithintheIgnitiontagstructure:
//SendacommandtoturntheGREENLEDOnspAv1.0/SparkplugDevices/DCMD/JavaRaspberryPi/Pibrella[‘Outputs/LEDs/green’:BOOL:true]
Page32SparkplugSpecificationVersion1.0
8. SparkplugMQTTSessionManagementandMessageFlowAnMQTTbasedSCADAsystemisuniqueinthattheHostnodeisNOTresponsibleforestablishingandmaintainingconnectionstothedevicesasisthecaseinmostexistinglegacypoll/responsedeviceprotocols.WithanMQTTbasedSCADAsystem,boththeHostapplicationaswellasthedevicesestablishMQTTSessionswithacentralMQTTServerorSevers.Thisisthedesiredfunctionalityasitprovidesthenecessarydecouplingfromanyoneapplicationandanygivendevice.AdditionalMQTTclientscanconnectandsubscribetoanyoftherealtimedatawithoutimpactingtheprimarySCADAHostapplication.
DuetothenatureofrealtimeSCADAsolutions,itisveryimportantfortheprimarySCADAHostandallconnectedMQTTEoNNodestohavetheMQTTSessionSTATEinformationforeachother.InordertoaccomplishthistheSparkplugTopicNamespacedefinitionsforBirth/DeathcertificatesalongwiththedefinedpayloadsprovidebothstateandcontextbetweentheSCADAHostMQTTclientandtheassociateddevicesideMQTTClients.Inmostusecasesandsolutionscenariostherearetwoprimaryreasonsforthis“designation”ofaprimarySCADAHost:
1. OnlythePrimaryIgnitionHostshouldhavethepermissiontoissuecommandstoMQTTDevices.2. InhighavailabilityandredundancyusecaseswheremultipleMQTTServersareused,MQTTEoN
NodesneedtobeawareofwhetherIgnitionhasnetworkconnectivitytoeachandeveryMQTTServerintheinfrastructure.IfthePrimaryIgnitionSTATEshowsthatanEoNNodeisconnectedtoanMQTTServerthatthePrimaryIgnitionGatewayisNOTconnectedto,thentheEoNNodeshouldwalktothenextavailableMQTTServerwhereSTATEfortheIgnitionGatewayis‘ONLINE’.
Page33SparkplugSpecificationVersion1.0
8.1. PrimaryIgnitionSessionEstablishmentOncetheMQTTEnginemoduleisinstalledinIgnition,theIgnitionGatewayConsoleprovidesanewMQTTEngineSettingstabinConfigurationàMQTTEngineàSettings.ThisconfigurationmenuallowsforthedefinitionofoneormoreMQTTServersthatarepresentintheinfrastructureaswellasifthisIgnitioninstanceisaPRIMARYinstanceintheinfrastructure(Note:ForIgnitioninstancesthatarenotprimary,refertosection8.4,GeneralMQTTapplicationsandnon-primaryIgnitionGateway.below).OncetheMQTTEnginemoduleisinstalledandconfigured,itwillimmediatelytrytocreateaHostMQTTSessionwiththeconfiguredMQTTServerinfrastructure.NotethattheestablishmentofanMQTTHostNodesessionisasynchronousofanyotherMQTTClientsession.IfEoNnodesarealreadyconnectedtotheMQTTServerinfrastructure,MQTTEnginewillbeabletosynchronizewiththem.IfassociatedEoNnodesarenotconnected,MQTTEnginewillregisterthemwhentheypublishtheirBirthCertificate.
Figure9-HostSessionEstablishment
ThesessiondiagraminFigure9-HostSessionEstablishmentshowsaverysimpletopologywithasingleMQTTServer.Thestepsoutlinedinthesessiondiagramaredefinedasfollows:
1. MQTTEnginewilltrytocreateanMQTTSessionusingtheMQTTCONNECTControlPacket(refertosection3.1intheMQTTV3.1.1specification).ADeathCertificateisconstructedintotheWillTopicandWillPayloadoftheoftheConnectControlPacketwithaWillQoS=0andWillRetain=true.TheMQTTCONNECTControlPacketisacknowledgedassuccessfulwithavalidCONNACKControlPacket.Fromthispointforwardintime,theMQTTServerisreadytodeliveraHostDeathCertificateanytimetheMQTTEngineMQTTClientlosesTCP/IPconnectivitytotheMQTTServer.
Page34SparkplugSpecificationVersion1.0
2. OnceanMQTTSessionhasbeenestablished,MQTTEnginewillpublishanewSTATEmessageasdefinedininsection7.6.1,IgnitionBirthCertificatePayload.AtthispointMQTTEnginecanupdatetheMQTTClientmetrictagsinIgnitionwithacurrentstateofONLINE.
3. WiththeMQTTSessionestablished,andaSTATEBirthCertificatepublished,MQTTEnginewillissueanMQTTsubscriptionforthedefinedSparkplugTopicNamespace,“spv1.0/#”.MQTTEngineisnowreadytostartreceivingMQTTmessagesfromanyconnectedEoNnodewithintheinfrastructure.SinceMQTTEngineisalsorelyingontheMQTTSessiontotheMQTTServer(s),theavailabilityofServerstoIgnitionisalsobeingmonitoredandreflectedintheMQTTClientmetricstagfolderinIgnition.
4. IfatanypointintimeMQTTEnginelosesTCP/IPconnectivitywiththedefinedMQTTServer(s),theONLINEstateoftheServerisimmediatelyreflectedintheMQTTClientmetricstagfolderinIgnition.AlltagdataassociatedwithanyMQTTEoNNodethatwasconnectedtothatMQTTServerwillupupdatedtoa‘STALE’dataquality.
Page35SparkplugSpecificationVersion1.0
8.2. EoNNodeSessionEstablishmentAnyEoNNodeintheMQTTinfrastructuremustestablishanMQTTSessionpriortoprovidinginformationforconnectedMQTTDevicenodes.MostimplementationsofanMQTTEoNNodeforrealtimeSCADAwilltrytomaintainapersistentMQTTSessionwiththeMQTTServerinfrastructure.ButthereareusecaseswheretheMQTTSessiondoesnotneedtobepersistent.Ineithercase,anEoNNodecantrytoestablishanMQTTsessionatanytimeandiscompletelyasynchronousfromanyotherMQTTClientintheinfrastructure.TheonlyexceptiontothisruleistheusecasewheretherearemultipleMQTTServersandaPrimaryHostapplication.
Figure10-EoNNodeMQTTSessionEstablishment
ThesessiondiagraminFigure10-EoNNodeMQTTSessionEstablishmentshowsaverysimpletopologywithasingleMQTTServer.Thestepsoutlinedinthesessiondiagramaredefinedasfollows:
1. TheEoNNodeMQTTclientwillattempttocreateanMQTTsessiontotheavailableMQTTServer(s)usingtheMQTTCONNECTControlPacket(refertosection3.1intheMQTTV3.1.1specification).TheDeathCertificateconstructedintotheWillTopicandWillPayloadfollowstheformatdefinedinsection7.2.1,EoNDeathCertificate(NDEATH).TheMQTTCONNECTControlPacketisacknowledgedassuccessfulwithavalidCONNACKControlPacket.Fromthispointforwardintime,theMQTTServerisreadytodeliveranEoNNodeDeathCertificatetoanysubscribingMQTTClientanytimeTCP/IPconnectivityislost.
2. OnceanMQTTSessionhasbeenestablished,theEoNNodeMQTTclientwillpublishanapplicationlevelBirthCertificateasdefinedinsection7.2.2,EoNBirthCertificate(NBIRTH).AtthispointMQTTEnginewillhavealloftheinformationrequiredtobuildouttheEoNNodemetricsandparametersintoanassociatedIgnitiontagfolderstructureandshowtheEoNNodeinan“ONLINE”state.
Page36SparkplugSpecificationVersion1.0
3. AfterpublishingtheEoNNodeBirthCertificate,thelastthingtheEoNNodeMQTTclientneedstodoistoissueasubscriptiontospecificSparkplugdefinedtopics.ThesubscriptiontoNCMDleveltopicsensuresthatEoNtargetedmessagesfromMQTTEnginearedelivered.ThesubscriptiontoDCMDensuresthatDevicetargetedmessagesfromMQTTEnginearedelivered.InapplicationswithmultipleMQTTServersanddesignatedPrimaryHostapplications,thesubscriptiontoSTATEinformstheEoNNodethecurrentstateofthePrimarySCADAHost.AtthispointtheEoNNodehasfullycompletedthestepsrequiredforestablishingavalidMQTTSessionwithMQTTEngineandIgnition.
4. IfatanypointintimetheEoNNodeMQTTClientlosesTCP/IPconnectivitytothedefinedMQTTServer(s),aDeathCertificateisissuebytheMQTTServeronbehalfoftheEoNNode.UponreceiptoftheDeathCertificate,MQTTEnginewillsetthestateoftheEoNNodeto‘OFFLINE’andupdatealltimestampmetricsconcerningtheconnection.Anydefinedmetricandparametertagswillbesettoa‘STALE’dataquality.
Page37SparkplugSpecificationVersion1.0
8.3. MQTTDeviceSessionEstablishmentTheSparkplugspecificationisprovidedtogetrealtimeprocessvariableinformationfromexistingandnewenddevicesmeasuring,monitoringandcontrollingaphysicalprocessintoandMQTTMOMinfrastructureandtheIgnitionIndustrialInternetofThingsapplicationplatform.InthecontextofthisdocumentanMQTTdevicecanrepresentanythingfromexistinglegacypoll/responsedrivenPLCs,RTUs,HARTSmartTransmitter,etc.,tonewgenerationautomationandinstrumentationdevicesthatcanimplementaconformantMQTTclientnatively.
TheprecedingsectionsinthisdocumentdetailhowMQTTEngineinteractswiththeMQTTServerinfrastructureandhowthatinfrastructureinteractswiththenotionofanMQTTEoNNode.Buttoalargeextentthetechnicalrequirementsofthosepiecesoftheinfrastructurehavealreadybeenprovided.FormostusecasesinthismarketsectortheprimaryfocuswillbeontheimplementationoftheSparkplugspecificationbetweenthenativedeviceandtheEoNNodeAPI’s.
Inordertoexposeandpopulatethemetric,parameter,andprocessvariabletaginformationfromanyintelligentdevice,thefollowingsimplesessiondiagramoutlinestherequirements:
Figure11-MQTTDeviceSessionEstablishment
ThesessiondiagraminFigure11-MQTTDeviceSessionEstablishmentshowsasimpletopologywithalloftheSparkplugelementsinplacei.e.Ignition,MQTTEngine,MQTTServer(s),MQTTEoNNodeandthiselement,theMQTTDeviceelement.Thestepsoutlinedinthesessiondiagramaredefinedasfollows:
1. ThisflowdiagramassumesthatatleastoneMQTTServerisavailableandoperationalwithintheinfrastructure.WithoutatleastasingleMQTTServertheremainderoftheinfrastructureisunavailable.
2. TheSessionEstablishmentofIgnitionusingtheMQTTEnginemoduleisdescribedinsection8.1,PrimaryIgnitionSessionEstablishment.TheestablishmentofthissessionisactuallyVERYarbitrarybasedonusecaseasEoNNodescanandwillestablishsessionswithMQTTinfrastructureswithorwithoutthiscomponentacross“N”numberofHostapplications.
3. TheSessionEstablishmentoftheassociatedMQTTEoNNodeisdescribedinsection8.2,EoNNodeSessionEstablishment.ThisflowdiagramassumesthattheEoNNodesessionhasalreadybeenestablishedwithMQTTEngine.
Page38SparkplugSpecificationVersion1.0
Dependingonthetargetplatform,theEoNNodemaybeaphysical“EdgeofNetwork”gatewaydevicedevicepollingphysicallegacydevicesviaModbus,AB,DNP3.0,HART,etc.,aMQTTenabledsensororsensorordevice,oritmightbealogicalimplementationofoneoftheCirrusLinkreferenceimplementationsforprototypeEoNNodesrunningontheRaspberryPIplatform.Regardlessofthetheimplementation,atsomepointthedeviceinterfacewillneedtoprovideastateandassociatedassociatedmetrics,parameters,andprocessvariabletopublishtotheMQTTinfrastructure.State#4inState#4inthesessiondiagramrepresentsthestateatwhichthedeviceisreadytoreportallofitsitsmetadataandprocessvariableinformationtotheMQTTEoNNodeasdefinedinSparkplug.ItistheistheresponsibilityoftheEoNNode(logicalorphysical)toputthisinformationinaformdefinedin0,
definedin0,
Page39SparkplugSpecificationVersion1.0
4. MQTTDeviceBirthCertificate(DBIRTH).UponreceivingtheDBIRTHmessage,MQTTEnginecanbuildoutthepropermetric,parameter,andtaginformationinIgnition.
FollowingtheSparkplugspecificationinsection0,
Page40SparkplugSpecificationVersion1.0
5. MQTTDeviceDataMessages(DDATA),allsubsequentmetric,parameter,andprocessvariableinformationarepublishedtoMQTTEngineonaReportbyException(RBE)basisusingtheDDATAmessageformat.
6. InatanytimetheMQTTDevice(logicalorphysical)cannotproviderealtimeinformation,theMQTTEoNNodespecificationrequiresthatanMQTTDeviceDeathCertificatebepublished.ThiswillinformMQTTEnginethatallmetric,parameter,andprocessvariableinformationinIgnitionbesettoa‘STALE’dataquality.
Page41SparkplugSpecificationVersion1.0
8.4. GeneralMQTTapplicationsandnon-primaryIgnitionGateway.Asnotedabove,thereisthenotionofaPrimaryIgnitionGatewayinstanceintheinfrastructurethathastherequiredpermissionstosendcommandtoDevicesandthefactthatallEoNNodesneedtobeawareofthefactthatthePrimaryIgnitionGatewayisconnectedtothesameMQTTServeritsconnectedtooritneedstowalktoanotheroneintheinfrastructure.ButbothoftheseareknownrequirementsofamissioncriticalSCADAsystem.
ButunlikelegacySCADAsystemimplementations,allrealtimeprocessvariableinformationbeingpublishedthrutheMQTTinfrastructureisavailabletoanynumberofadditionalMQTTClientsinthebusinessthatmightbeinterestedinsubsetsifnotalloftherealtimedata.
TheONLYdifferencebetweenaPrimaryIgnitionMQTTclientandallotherclientsthatnon-primaryClientdoNOTissuetheSTATEBirth/Deathcertificates.
Page42SparkplugSpecificationVersion1.0
9. SparkplugMQTTDataandCommandMessagesLookingbackinthisdocumentwe’vedescribedthefollowingcomponents:
• IgnitionGateway• MQTTEngine• MQTTServer(s)• EdgeofNetwork(EoN)Nodes• Devices• TopicNamespace• PayloadEncoding• BirthCertificates• DeathCertificates• STATEMessages• Ignition,EoNNode,andDeviceSessionEstablishment
AllofthesespecificationsanddefinitionsgettotheprimarygoalofSparkplug,thatistodeliverarichsetofrealtimeDeviceprocessvariabledataextremelyefficientlytomanydataconsumerswithintheEnterprisewhilestillprovidingabestinclassCommand/ControlSCADAsystem.
ThedisruptivenotionoftheemergingIIoTmindsetisthatintelligentdevicesshouldbesmartenoughtodeliverprocessvariabledatatotheinfrastructurewhenitisrequired.Butthefactofthematteristhattheexistingpopulationof100’sofmillionsofthesmartdevicesneedtobe“asked”ifsomethinghaschangedusingpoll/responseprotocols.Thisiswhywe’reseeingtheemergenceofedgedevicesthroughouttheindustrialsector.ForthedecadeormorethatitwilltakefordevicemanufacturestoembedIIoTtechnologynatively,thesolutionbeingemployedtodayistoplacethiscapabilityinsmallembeddeddevicesclosertothedataproducersthemselves.SowithintheSparkplugspecificationthesedevicescalledEdgeofNetworkNodes(EoN)representthisnewclassofGateway,EdgeController,EdgeofNetworkNode,ProtocolGateway,andmanymoreacronymsforthesameclassofdevices.ThecapabilityofthesedevicesareinanextremerangeoflowpowermicrocontrollerstomulticoreIntelandARMbasedprocessors.TheoperatingsystemsrangefromfullembeddedLinuxkernelsandWindowsembeddedtosmallbaremetalRTOS’s.Regardlessofthecategorythesegatewaydevicesfallinto,thesimplicity,ofMQTTandtheSparkplugspecificationshouldbeavailableacrosstheboard.
ThissectionoftheSparkplugspecificationgoesintodetailonhowmetric,parameter,andprocessvariabledataarepublished/subscribewithinanMQTTinfrastructureinrealtimeandtheresultingtaginformationthatIgnitioncanread/writeto.
9.1. EoNNDATAandNCMDMessagesWe’llstartthissectionwithadescriptionofhowmetricandparameterdataispublishedtoIgnitionfromanEoNNodeintheMQTTinfrastructure.ThedefinitionofanEoNNodeisgenericinthatitcanrepresentbothphysical“EdgeofNetworkGateway”devicesthatareinterfacingwithexistinglegacyequipmentandalogicalMQTTendpointfordevicesthatnativelyimplementtheSparkplugspecification.SectionXXXXXabovedefinestheBirthCertificateMQTTPayloadandthefactthatitcanprovideanynumberofmetricandparametertagsthatwillbeexposedinIgnition.Someofthesetagswillbe“readonly”suchas:
Page43SparkplugSpecificationVersion1.0
• EoNManufacture• EoNDeviceType• EoNSerialNumber• EoNSoftwareVersionNumber• EoNConfigurationChangeCount• EoNPosition(ifGPSdeviceisavailable)• EoNCellularRSSIvalue(ifcellularisbeingused)• EoNPowerSupplyvoltagelevel• EoNTemperature
Othermetricsmaybedynamicand“read/write”tagssuchas:
• EoNRebirthcommandtorepublishallEoNandDeviceBirthCertificates.• EoNNextservercommandtomovetonextavailableMQTTServer.• EoNRebootcommandtoreboottheEoNNode.• EoNPrimaryNetwork(PRI_NETWORK)where1=Cellular,2=Ethernet
TheimportantpointtorealizeisthatthetagsexposedinIgnitionforuseinthedesignofapplicationsarecompletelydeterminedbywhatmetricandparametervaluesarepublishedintheEoNBirthCertificate.EachspecificEoNNodecanbestdeterminewhatdatatoexpose,andhowtoexposeit,anditwillautomaticallyappearintheIgnitiontagstructure.ParameterscanevenbeaddeddynamicallyatruntimeandwithanewBirthCertificate,theseparameterswillautomaticallybeaddedtotheIgnitiontagstructure.
TheotherVERYimportantdistinctiontomakehereisthatEoNNodeNDATAandNCMDmessagesaredecoupledfromtheDevicelevelcommandandcontrolmessagesofDDATAandDCMD.ThisdecouplingintheTopicNamespaceisimportantbecauseitallowsinteractionfromallMQTTClientsinthesystem(tothelevelofpermissionandapplication)withtheEoNNodes,butNOTtothelevelofsendingDevicecommands.MQTTEngineprovidesaconfigurationparameterthatwillBLOCKoutputDDATAandDCMDmessagesbutstillallowNDATAandNCMDmessagestoflow.Inthismanner,multipleIgnitionsystemscanbeconnectedtothesameMQTTinfrastructure,butonlytheoneswithDevicecommandsenabledcanpublishDevicecommands.
ThefollowingsimplemessageflowdiagramdemonstratesthemessagesusedtoupdateachangingcellularRSSIvalueinIgnitionandsendingacommandfromIgnitiontotheEoNNodetouseadifferentprimarynetworkpath.
Page44SparkplugSpecificationVersion1.0
Figure12-EoNNodeNDATAandNCMDMessageFlow
1. AssumingMQTTServerisavailable.2. AssumingthatMQTTEngineasanestablishedMQTTSessionwiththeMQTTServer(s).3. TheEoNNodehasanestablishedMQTTSessionandtheBirthCertificatehasbeenpublished.
Ignitionnowhasalldefinedmetricandparametertagsandtheircurrentvalue.4. TheEoNNodeismonitoringitslocalcellularRSSIlevel.ThelevelhaschangedandnowtheEoN
NodewantstopublishthenewvaluetotheassociatedtaginIgnition.5. Fromandoperationalrequirement,theEoNNodeneedstobetoldtoswitchitsprimary
networkinterfacefromcellulartoEthernet.FromIgnitionthenewvalueiswrittentothetagandMQTTEnginewillautomaticallypublishthenewvaluetotheEoNNodeparameters.
9.2. DeviceDDATAandDCMDMessagesTheultimateintentoftheSparkplugspecificationistofacilitatethepublishingofrealtimeprocessvariabledatafromexistingdevicesinthefieldorontheplantfloorintoaMessageOrientedMiddlewareinfrastructure.ItisrecognizedthatthisrepresentsaHUGEplethoraofdevicesacrossthespectrumintheIndustrialdevicemarketsectorspanningdecadesofinstalledandlegacyinfrastructure.ThroughtheuseofMQTTandtheSparkplugspecification,Ignitioncanbecometheultimate“SwissArmyKnife”astheIndustrialApplicationDevelopmentplatformfortheemergingIIoTinfrastructures.
ThereasonthatthenotionofanEoNNodeispresentinSparkplugisarealizationthatifexistingsmartdeviceinfrastructurecannotbeaddressedwithemergingtoolsanddevelopmentplatforms,themigrationfromlegacymethodologies,protocols,infrastructures,anddatasiloswillneveroccurandtheenvisagedbenefitsofFactory4.0willnevercometofruition.EoNNodescanrepresentphysicalhardwareintheplantorinthefieldthatcanaddressconvertinglegacypoll/responseprotocols(whileprovidingtheoftenrequirednetworksecurityatthesametime)intorealtimeMQTTmessages.Atthesametime,EoNNodescanrepresentalogicalsoftwareagentinnextgenerationdevicesthatcannativelyproviderealtimeprocessvariablesviaMQTT.ByusingGoogleProtocolBuffersandtheKura
Page45SparkplugSpecificationVersion1.0
schemainconcertwiththeIgnitiontagstructure,devicescanbedecoupledfromapplicationsandvaluabledeviceinformationcanbemuchmoreeasilysharedwithintheenterprise.
ButultimatelytheEoNNodesareonlyinplacetosupporttheultimatedataproducer/consumer,therealdevicesontheplantfloororinthefield.TheSparkplugspecificationprovidesawaytonametheprocessvariablesacquiredfromthesedevicesalongwithdefiningtheappropriatedatatypeandcurrentvalue.OncetheprocessvariablesarepublishedintoanMQTTinfrastructure,theycanbeconsumedbyanynumberofapplicationsareareinterestedinthevalues,includingoneormoreIgnitionGatewayinstances.
Section0,
Page46SparkplugSpecificationVersion1.0
MQTTDeviceBirthCertificate(DBIRTH)abovedefinestheDeviceBirthCertificateMQTTPayloadthatdefinesalloftheprocessvariablesbeingprovidedbytheassociateddevice.UponthereceiptoftheBirthCertificate,MQTTEnginebuildsouttheentireDevicefolderundertheassociatedEoNNodeandcreatesallofthedefinedtagswithcurrentvaluesandstates.ThetagnamingconventionisleftentirelytotheimplementationoftheSparkplugspecificationontheEoNNode.Forexample,ifanEoNNodewasusingtheModbusprotocoltopollaPLClocally,thenthetagnamesusedforIgnitionmightremaintheconventionalModbusregisteraddressesof0xxxx,1xxxx,3xxxx,and4xxxx.ConverselytheEoNNodemightprovidethecapabilityofassigningmoremeaningfultagnamestotheModbusregisterprocessvariablessuchasSuctionPressure,TankLevel,ProcessTemp,etc.AnEoNNodeprovidingtheinterfacetoHARTenables4-20maSmartTransmitterscouldusetheactualHARTassignedprocessvariablesnamessuchasPV1,PV2,LRV,URV,etc.
Regardlessofthetagnamingconventionused,uponreceiptoftheDeviceBirthCertificate,MQTTEnginewillcreatetheprocessvariabletagstructurewithinIgnitionwheretheywillbeimmediatelyusablebytheIgnitionplatform.Inthefollowingmessageflowdiagram,we’llusetheexampleofaModbusPLC.ThisexamplewillalsoassumethattheEoNNodeconnectedtothePLCcanprovidedescriptivetagnamestoeachoftheModbusregisteraddresses:
• ValveOpen //mapModbusCoil#1totheValveOpencommand.• ValveClose //mapModbusCoil#2totheValveClosecommand.• ValveLS1 //mapModbusStatusInput10,001totheValveLimitSwitch#1• ValveLS2 //mapModbusStatusInput10,002totheValveLimitSwitch#2• SuctionPressure //mapModbusInputRegister30,001totheSuctionPressure• DischargePressure //mapModbusInputRegister30,002totheDischargePressure• Setpoint //mapModbusHoldingRegister40,001tothesetpointvalue
Page47SparkplugSpecificationVersion1.0
Figure13-DeviceDDATAandDCMDMessageFlow
1. AssumingMQTTServerisavailable.2. AssumingthatMQTTEnginehasanestablishedMQTTSessionwiththeMQTTServer(s).3. AssumingthattheEoNNodehasanestablishedMQTTSessionwiththeMQTTServer(s).4. UponreceivingtheDeviceBirthCertificate,MQTTEnginecreates/updatestheDevicefolderin
IgnitionandallassociatedtagsdefinedforthisDevice.5. TheEoNNodeispollingtheModbusPLClocallyanddetectsthattheSuctionPressurein
ModbusRegister30,001haschangedvalues.ThisisimmediatelyplacedintothedefinedDeviceMQTTPayloadandpublished.Uponreceipt,MQTTEngineupdatestheSuctionPressuretagvalue.
6. The‘ValveOpen’BooleaniscommandedonanIgnitiondashboard.MQTTEnginecreatestheDCMDpayloadrequiredbytheEoNNodetosendaModbusForceCoilCommandtoCoil#1.
7. TheValveOpencommandcausestheMotorOperatedvaluetostartmovingwhichinturnchangesthestateofthetwoMOVlimitswitches.ThenextModbuspollresultsinboth‘ValveLS1’and‘ValveLS2’inchangingstate.BothchangesareplacedintoaDDATAMQTTpayloadandpublished.MQTTEnginereceivestheDDATAmessageandimmediatelyupdatestheassociatedtagvaluesinIgnition.
8. OnthenextModbuspoll,theDischargePressurechangesvalue.ThenewvalueispublishedtotheMQTTServer.MQTTEnginereceivesthemessagesandimmediatelyupdatestheDischargePressuretag.Thevalvethatwascommandedin#6abovefinallygoesfroman“intransit”statetofullyopen.ThiscausesbothlimitswitchinputstothePLCtochangestate.TheEoNNodepicksupthisstatechange,formatsaDDATAmessagewiththenewvaluesandpublishesittotheMQTTServer.MQTTEnginereceivesthemessageandimmediatelyupdatestheassociatedtagsinIgnition.
Page48SparkplugSpecificationVersion1.0
10. SparkplugManagementofMultipleMQTTServersTherearemanyinstanceswheretheMQTTinfrastructurewillcontainmultipleMQTTServers.ThereareseveralreasonwhymultipleMQTTServersareutilized.Certainly,applicationswhereredundancy,highavailability,anddisasterrecoveryrequiremultipleMQTTservers.MultipleMQTTServerscanalsobedeployedwherescalabilityacrossalargenumberofMQTTEoNNodesisrequired.OtherhybridmodelsuseonpremiseMQTTserversastheprimaryandCloudbasedMQTTServerand/orServicesassecondaryandtertiary.
RegardlessofthereasonformultipleMQTTServers,thereareseveralarchitecturalconsiderationsthatneedtobeaddressedbytheSparkplugspecification.
10.1. MultipleMQTTServerTopologyForacomparisonofthetopologies,thefollowingdiagramshowsatypicalinfrastructurewithasingleMQTTServerinstance:
Figure14-SingleMQTTServerTopology
ThesingleMQTTServertopologyshownaboveisaperfectlyviablesolutioninmanyinstances.ButitdoesrequireahighlyavailableMQTTServerandpresentsasinglepointoffailurewithinthesystem.
InkeepingwiththeKISSprincipal(KeepItSimple)allofthemultipleMQTTServertopologiesshownon-clusteredMQTTServers.ThemarkettodayoffersmanychoicesofMQTTServers,someofwhichareclusteredinstancesofMQTTServers.ButasabaselinefunctionalityandtokeepthingverysimplethecurrentSparkplugspecificationassumesthateachandeveryMQTTServerisastandaloneinstance.OneoftheuniquefeaturesofMQTTEngineisthatitseamlesslymanagesthemulti-servertopologies.
Page49SparkplugSpecificationVersion1.0
Figure15-MultipleMQTTServerTopology
InthedrawingshowninFigure15above,therearemultipleMQTTServersavailableinthetopology.Inthistopology,MQTTEngineconnectstoallavailableMQTTServersintheinfrastructureandprovidesdetailedmetricsoneachinstance.Thesemetricsinclude:
• NumberofEoNNodesconnectedtoeachMQTTServer• LatencycheckinmillisecondsbetweenMQTTEngineandtheMQTTServer.• TheMQTTClientIDusedforthisMQTTSession• TheOfflineDateTimetimestampofwhentheMQTTServerlastwentoffline.• TheOnlineDateTimetimestampofwhentheMQTTServercameonline.• ThecurrentrealtimestateoftheconnectiontotheMQTTServer• NumberofMQTTmessagespersecondbeingprocessedbythisMQTTServer• TheMQTTServerURL
Page50SparkplugSpecificationVersion1.0
Figure16-MQTTClienttagsinIgnition
NotethatalloftheIgnitionGatewayinstancesandwellasanyother“LineofBusiness”(LOB)applicationcanestablishMQTTSessionswithallavailableMQTTServers.Inthismanner,theEoNNodesarefreetodeterminethebestnetworkpathandMQTTServeravailabilitytoestablishtheirMQTTSessionwith.
ThistopologydoesrequirethatEoNNodesusingtheSparkplugspecificationkeepatableofavailableMQTTServersandbeabletoconnecttoanynumberofthembasedonnetworkandserveravailability.
UsingthistopologyanyEoNNodecanuseanyoftheavailableTCP/IPnetworksandconnecttoanyoneoftheavailableMQTTServers.FromtheIgnitionandLOBapplicationsideofthetopology,theydon’treallycarewhichoftheNnumberofMQTTServerstheEoNNodesconnectto.TheBirth/DeathcertificatemanagementautomaticallytakescareofknowwhichEoNNodesareconnectedatanypointintime,andjustasimportant,whichnodesarenotconnected.MQTTEnginekeepsmetricsoneveryEoNNodeconnectingintotheMQTTinfrastructureandincludes:
• CurrentMQTTServerthisEoNNodeisconnectedto.• DateTimetimestampofwhenthisEoNwentOfflinelast.• DateTimetimestampofwhenthisEoNNodecameOnlinelast.• ThenumberofBirthCertificatessentbythisEoNNode.• ThenumberofDeathCertificatesseenfromtheMQTTServersonbehalfofthisEoNNode.• CurrentOnlinestateofthisEoNNode• TotalnumberofactualbytesreceivedfromthisEoNNode.• TotalnumberofactualbytessenttothisEoNNode.
Page51SparkplugSpecificationVersion1.0
Figure17-EoNMQTTmetrictagsinIgnition
10.2. PrimaryIgnitionGatewaySTATEinMultipleMQTTServerTopologiesForimplementationswithmultipleMQTTServers,thereisreallyonlyoneadditionalaspectthatneedstobeunderstoodandmanagedproperly.WhenmultipleMQTTServersareavailablethereisthepossibilityof“stranding”andEoNNodeifthePrimarycommand/controlIgnitionGatewaylosesnetworkconnectivitytooneoftheMQTTServers.InthisinstancetheEoNNodewouldstayproperlyconnectedtotheMQTTServerpublishinginformationnotknowingthatIgnitionwasnotabletoreceivethemessages.WhenusingmultipleMQTTServers,theprimaryIgnitionGatewayinstancemustbeconfiguredtopublishaSTATEBirthCertificateandallEoNNodesneedtosubscribetothisSTATEmessage.
MQTTEnginehasaconfigurationoptionthatspecifieswhetherthisIgnitionGatewayinstanceisa“Primary”command/controlinstanceornot.IfitisaprimaryinstancetheneverytimeMQTTEngineestablishesanewMQTTSessionwithanMQTTServer,theSTATEBirthCertificatedefinedinsectionaboveisthefirstmessagethatispublishedafterasuccessfulMQTTSessionisestablished.
EoNNodedevicesinaninfrastructurethatprovidesmultipleMQTTServerscanestablishasessiontoanyoneoftheMQTTServers.Uponestablishingasession,theEoNNodeshouldissueasubscriptiontotheSTATEmessagepublishedbyIgnition.SincetheSTATEmessageispublishedwiththeRETAINmessageflagset,MQTTwillguaranteethatthelastSTATEmessageisalwaysavailable.TheEoNNode
Page52SparkplugSpecificationVersion1.0
shouldexaminethepayloadofthismessagetoensurethatitisavalueof“ONLINE”.Ifthevalueis“OFFLINE”,thisindicatesthethePrimaryIgnitionGatewayhaslostitsMQTTSessiontothisparticularMQTTServer.ThisshouldcausetheEoNNodetoterminateitssessionwiththisMQTTServerandmovetothenextavailableMQTTServerthatisavailable.ThisuseoftheSTATEmessageinthismannerensuresthatanylossofconnectivitytoanMQTTServertothePrimaryIgnitionGatewaydoesnotresultinEoNNodesbeing“stranded”onanMQTTserverbecauseofnetworkissues.ThefollowingmessageflowdiagramoutlineshowtheSTATEmessageisusedwhenthree(3)MQTTServersareavailableintheinfrastructure:
Figure18-IgnitionSTATEflowdiagram
1. WhenanEoNNodeisconfiguredwithmultipleavailableMQTTServersintheinfrastructureitshouldissueasubscriptiontothePrimaryIgnitionGatewaySTATEmessage.TheEoNNodesarefreetoestablishanMQTTSessiontoanyoftheavailableserversoveranyavailablenetworkatanytimeandexaminethecurrentSTATEvalue.IftheSTATEmessagepayloadis‘OFFLINE’thentheEoNNodeshoulddisconnectandwalktothenextavailableserver.
2. Uponstartup,ifMQTTEngineisconfiguredtobethePrimaryIgnitioninstance,theMQTTSessionwillbeconfiguredtoregisteranIgnitionDEATHCertificatethatindicatesSTATEis‘OFFLINE’withthemessageRETAINflagsettotrue.ThenanIgnitionBIRTHCertificatewillbepublishedwithaSTATEpayloadof‘ONLINE’.
3. AstheEoNNodewalksitsavailableMQTTServertable,itwillestablishanMQTTSessionwithaserverthathasaSTATEmessagewithapayloadof‘ONLINE’.TheEoNNodecanstayconnectedtothisserveraslongasitsMQTTSessionstaysintactanditdoesnotreceiveanIgnitionDEATHCertificate.
4. HavingasubscriptionregisteredtotheMQTTServerontheSTATEtopicwillresultinanychangetothecurrentIgnitionSTATEbeingreceivedimmediately.Inthiscase,anetworkdisruptioncausestheIgnitionMQTTSessiontoserver#2tobeterminated.ThiswillcausetheMQTTServer,onbehalfofthenowterminatedIgnitionMQTTClienttopublishtheDEATHcertificateto
Page53SparkplugSpecificationVersion1.0
anyonethatiscurrentlysubscribedtoit.UponreceiptoftheIgnitionDEATHCertificatethisEoNNodewillmovetothenextMQTTServerinitstable.
5. TheEoNNodemovedtothenextavailableMQTTServerandsincethecurrentSTATEonthisserveris‘ONLINE’,itcanstayconnected.
6. Inthemeantime,thenetworkdisruptionbetweenIgnitionandMQTTServer#2hasbeencorrected.MQTTEnginehasanewMQTTSessionestablishedtoserver#2withanupdateBirthCertificateof‘ONLINE’.NowMQTTServer#2isreadytoacceptnewEoNNodesessionrequests.
Page54SparkplugSpecificationVersion1.0
11. SparkplugPersistentversusNon-PersistentConnectionsPersistentconnectionsareintendedtoremainconnectedtotheMQTTinfrastructureatalltimes.TheyneversendanMQTTDISCONNECTmessageduringnormaloperation.ThisfactletsMQTTEngineprovidetherealtimestateofeverypersistentnodeintheinfrastructurewithintheconfiguredMQTTKeepAlivetimeperiodusingtheBirth/Deathmechanismsdefinedabove.
Butinsomeusecases,suchassendingGPScoordinatesforassettrackingorotherIOTapplicationswithperiodicdatafromsensors,MQTTenableddevicesdonotneedtoremainconnectedtotheMQTTinfrastructure.Intheseusecases,alltheDeviceneedstodoistoissueanMQTTDISCONNECTcontrolpacketpriortogoingofflineinordertoleavetheMQTTinfrastructure“gracefully”.InthiscaseanMQTTDeviceorassociatedDeviceDEATHcertificatewillmostnormallynotbeseen.SystemdesignersjustneedtobeawarethatthetagsinIgnitioninthiscasewillrepresent“LastKnownGood”valueswithatimestampofthisdatawherethecurrentstateoftheoftheMQTTDeviceisnotarealtimeindication.Ignitiontagtimestampvaluescanbeusedtodeterminewhenthevaluesfromthisnodewerelastupdated.
Non-persistentMQTTEnabledDevicesshouldstillregisteraproperDEATHCertificateupontheestablishmentofanMQTTsession.InthismannertheIgnitioncanstillhaveagoodrepresentationofLastKnownGoodprocessvariableversusthefactthattheMQTTsessionwasterminatedpriortotheEoNNodebeingabletocompleteitstransaction.
Page55SparkplugSpecificationVersion1.0
12. GlossaryofTermsThissectionwillbeprovidedinthenextversionoftheSparkplugspecification.
Page56SparkplugSpecificationVersion1.0
13. ContactInformationForanyquestionsregardingthisSparkplugspecificationorformoreinformation,pleaseusethefollowingdetails:CirruslinkSolutionsWebsite:www.cirrus-link.comPhone:844-924-7787Email: support@cirrus-link.com
top related