performance evaluation of publish/subscribe middleware ... · performance evaluation of...

Performance Evaluation of Publish/Subscribe Middleware Technologies for ATM (Air Traffic

Management) SystemsJuan M. Lopez-Soler, Jose M. Lopez-Vega, Javier Povedano-Molina and Juan J. Ramos-Munoz

Signal Theory, Telematics and Communications Dept.

This presentation is created by the University of Granada and based on ©SESAR JOINT UNDERTAKING material developed within the frame of the SESAR Programme, cofinanced by the EU and EUROCONTROL. Reprint of information included herein with approval of publisher and with reference to source code only.

Performance Evaluation of Publish/Subscribe Middleware Technologies for ATM SystemsJuan M. Lopez-Soler, Jose M. Lopez-Vega, Javier Povedano-Molina and Juan J. Ramos-Munoz

2/29

Presentation Outline

1.- Introduction1.1.- SESAR Program1.2.- Technologies evaluation

2.- Experimental setup2.1.- Test environment2.2.- Benchmarking Tool

3.- Results3.1.- Marshalling3.2.- Data Size Impact evaluation3.3.- Number of subscribers Impact evaluation3.4.- Impact evaluation of the Number of publishers 3.5.- Impact evaluation of Filter criteria

4.- Conclusions and future workReferences


3/29

1.- Introduction● The SESAR(Single European Sky ATM Research) programme is the European air traffic control infrastructure modernisation programme [1].

● SESAR aims at developing the new generation air traffic management system capable of ensuring the safety and fluidity of air transport worldwide over the next 30 years, reducing costs of ATM and making flying more environmentally friendly.

● SESAR is one of the most ambitious research and development projects ever launched by the European Community.

● Budget: 2.1 billion € : ● European Commision and TEN-T EA: 700 mio€● Eurocontrol: 700 mio€.● Industry: 700 mio €.

http://www.sesarju.eu

1.1.- SESAR program


4/29

http://www.eurocontrol.int/communications/public/standard_page/pens.html

Pan-European Network Service (PENS): a common IP based network service across the European region covering voice and data communication and providing efficient support to existing services and new requirements that are emerging from future Air Traffic Management (ATM) concepts [2].

1.- Introduction1.1.- SESAR program


5/29

●As a part of SESAR, the D06-GroundGround Technology & Service Option Evaluation (Step2) deliverable contains the results of the evaluation of the following middleware technologies in terms of performance and interoperability [3]:

● CORBA● DDS● Message Oriented Middleware (JMS, AMQP)● Collaborative databases● Web Services● REST● OGC (Open Geospatial Consortium)

● In this presentation we will focus on the results obtained for the LAN Performance evaluation of DDS, included in the deliverable.

1.- Introduction1.2.- Technologies evaluation


6/29







7/29

2.- Experimental Setup2.1.- Test Environment

● Common experimental environment: 4 machines in a Gigabit Ethernet LAN.

Brand: HP ProLiant ML110 G6 Xeon G6950. Processor: One Intel Pentium G6950 / 2.8 GHz. Dual Core. 64 bit. RAM: 4 GB. DDR3 SDRAM – ECC. 1333 MHz. DIMM 240-pin. Harddisk: One 250 GB – standard – Serial ATA Network adapter: PCI Express. HP NC 107i. Gigabit Ethernet Ubuntu Linux version 10.04.2 server amd64

● DDS environment: Real Time Innovations DDS implementation, ndds.4.5d, with TLS support.

pub Subi

Subi+1

Subi+2


8/29

2.- Experimental Setup2.2.- Benchmarking tool

● It is composed of two parts.● The publishing application publishes throughput data● The subscribing applications subscribe to the throughput data● Echo latency request/replies are embedded, gathered at the publisher's side.● Throughtput statistics are gathered at the subscribers side.● Extended to support “large data” (RTPS standard [4]) and TLS tests.

RTI PerfTest 1.1d


9/29

2.- Experimental Setup2.2.- Benchmarking tool

Middleware

In-Message

Marshalled Message

Middleware

Out-Message

M1

T1 T2

M2

M31

M41

T31 T41

Middleware

Out-Message

M32 M42

T32 T42

• MAXIMUM TROUGHPUT: measurements that gets the largest throughput. • MINIMUM LATENCY: measurements that gets the lowest latency.

Measures


10/29







11/29

3.- Experimental Results3.1.- Marshalling

The Marshalling Performance is the time taken to convert the application data structure into a technology transport message.

Middleware

In-Message

Marshalled Message

Middleware

Out-Message

M1

T1 T2

M2

M31

M41

T31 T41

Middleware

Out-Message

M32 M42

T32 T42

Marshalling time (T2-T1)

Demarshalling time (T4-T3)


12/29



13/29


- The use of larger topics can reduce the overhead.


14/29

3.- Experimental Results3.2.- Data Size Impact Evaluation

- Large data do not fit in a single UDPv4 datagram, needing further process (64KB limitation).- TLS processing overload increases for very large data.

Unicast1 subscriber1 publisher


15/29

3.- Experimental Results3.2.- Data Size Impact Evaluation

Unicast1 subscriber1 publisher

- The processing complexity of TLS and the use of TCP directly affects the latency.


16/29

3.- Experimental Results3.3.- Number of subscribers Impact

- Using unicast, the available bandwidth is shared, and therefore the throughput decreases proportionally with the number of subscribers.

UnicastTopic size=10KB


17/29


- The latency does not increases proportionally to the number of subscribers.



18/29


- Enabling multicast, the middleware take the most of the available bandwidth, with a similar performance for 8 or 16 subscribers.

MulticastTopic size=10KB


19/29


- Even for very large data topics, multiple subscribers slightly impact on the throughput when using multicast and best effort service.

MulticastTopic size=10MB


20/29

3.- Experimental Results3.4.- Number of publishers Impact


- The use of TLS, with TCP, and several publishers, improves the overall throughput.


21/29

3.- Experimental Results3.4.- Number of publishers Impact

MulticastTopic size=10KB

- The increase of the number of publishers has little impact on the reliable service throughput.


22/29

3.- Experimental Results3.5.- Impact eval. of Filter criteria

●DDS allows defining a rich set of filter criteria to optimize the publication/subscription of resources' consumption.

●To demonstrate the feasibility of this feature at the middleware level (instead of at the application level), two conditions were tested.

●Particularly, for one condition only 80% of the samples matched the filtering criteria (F=0.8), and for the other, the 100% matched it (F=1.0).


23/29


- Best effort and reliable modes achieve almost the theoretical maximum throughput when F=1.-For F=0.8, the throughput decreased, since only 80% of the received data matched the filtering criteria.

UnicastTopic size=10KB1 publisher16 subscribers


24/29


MulticastTopic size=10KB1 publisher16 subscribers

- As in previous tests, using multicast the throughput reaches the maximum available bandwidth.


25/29







26/29

4.- Conclusions and Future Work

● We have presented the results of the LAN Performance evaluation of DDS for SESAR (D06-GroundGround Technology & Service Option Evaluation (Step2)). ● As expected, the best performance in terms of latency is achieved for small topics. ●Large topics increase the complexity of the RTPS protocol (64KB submessage fragments), but can achieve better throughputs.● When using multicast, the number of subscribers has little impact. ● When using unicast, the bandwidth can be fairly shared among the number of subscribers.


27/29

4.- Conclusions and Future Work

Future work: ● To ease and automatize future benchmarkings, we are designing and implementing an autotuning benchmark tool. ● The large topic scenarios will be explored more in deep.● The Datagram TLS implementation will be used for comparing the results of the security support.


28/29

[1] SESAR Joint Undertaking webpage, http://www.sesarju.eu/about[2] Pan-European Network Service (PENS), http://www.eurocontrol.int/communications/public/standard_page/pens.html[3] Deliverable D06 under SESAR project 14.01.12 (“D06-GroundGround Technology & Service Option Evaluation (Step2)”). [4] OMG Document, ”The Real-time Publish-Subscribe Wire Protocol DDS Interoperability Wire Protocol Specification (DDS-RTPS) Version 2.1”.

Clipart and pictures:- Front page: photo by peterned, http://www.flickr.com/photos/peterned/2562527752/in/photostream/- http://openclipart.org/detail/155101/server-by-saisyukusanagi- http://www.eurocontrol.int/communications/public/standard_page/pens.html

References


29/29

Thanks for your attention!

For further information, visit our site (http://dtstc.ugr.es/tl) or contact us by e-mail ([email protected], [email protected]).

http://dtstc.ugr.es/tl

mailto:[email protected]

mailto:[email protected]

performance evaluation of publish/subscribe middleware ... · performance evaluation of...

Documents