coordination of cooperative cloud computing...

72
Coordination of Cooperative Cloud Computing Platforms Mathieu Lavallée Michel Mayrand Elisa Shahbazian Prepared by: OODA Technologies Inc. 4580 Circle Rd Montréal, QC H3W 1Y7 PWGSC Contract Number: W7707-145677/001/HAL Technical Authority: Anthony W. Isenor Contractor’s Publication Date: April 2017 The scientific or technical validity of this Contract Report is entirely the responsibility of the Contractor and the contents do not necessarily have the approval or endorsement of the Department of National Defence of Canada. Contract Report DRDC-RDDC-2017-C118 May 2017

Upload: docong

Post on 18-Mar-2018

227 views

Category:

Documents


6 download

TRANSCRIPT

Page 1: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Coordination of Cooperative Cloud Computing Platforms

Mathieu Lavalleacutee Michel Mayrand Elisa Shahbazian Prepared by OODA Technologies Inc 4580 Circle Rd Montreacuteal QC H3W 1Y7 PWGSC Contract Number W7707-145677001HAL Technical Authority Anthony W Isenor Contractorrsquos Publication Date April 2017

The scientific or technical validity of this Contract Report is entirely the responsibility of the Contractor and the contents do not necessarily have the approval or endorsement of the Department of National Defence of Canada

Contract Report

DRDC-RDDC-2017-C118

May 2017

Template in use (2010) SR Advanced Template_EN (051115)dotm

copy Her Majesty the Queen in Right of Canada as represented by the Minister of National Defence 2017

copy Sa Majesteacute la Reine (en droit du Canada) telle que repreacutesenteacutee par le ministre de la Deacutefense nationale

2017

Coordination of Cooperative Cloud ComputingPlatforms

Mathieu LavalleeMichel MayrandElisa Shahbazian

Prepared By OODA Technologies Inc4580 Circle RdMontreal (Qc) H3W 1Y75144764773

Prepared For Defence Research amp Development Canada Atlantic Research Centre9 Grove Street PO Box 1012Dartmouth NSB2Y 3Z7902-426-3100

Scientific Authority Anthony W IsenorContract Number W7707-145677001HALCall Up Number 17Project Design develop manage test andor implement specific software or data source modulesReport Delivery Date April 7 2017

The scientific or technical validity of this Contract Report is entirely the responsibility of thecontractor and the contents do not necessarily have the approval or endorsement of Defence RampDCanada

This page is intentionally left blank

Executive Summary

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

The first part of this report addresses literature and tools review on cloud environments theirframeworks and the standards used to connect them for exchanging resources The second partof this report describes the installation of two clouds using OpenStack framework and the config-uration necessary to allow intercloud communication The next step is the investigation on howto use intercloud capability for exchanging information between two clouds In that context wedescribe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

i

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

ii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Contents

Executive Summary i

Contents iii

List of Figures vii

List of Tables ix

List of Acronyms xi

1 Introduction 1

11 Background 1

12 Document Overview 2

2 Literature Review 3

21 Research Methodology 3

211 Keywords 4

212 Glossary 6

22 Standards 10

221 Standard Inventory 10

222 Relevant Standards 10

2221 Cloud Data Management Interface (CDMI) 10

2222 Open Cloud Computing Interface (OCCI) 10

23 Cloud environments 12

iii

Final Report for RISOMIA Call-up 17

231 Cloud levels of services 12

232 Cloud software review 14

24 Tool recommendation 15

25 Other recommended books and articles 16

251 Books 16

252 Articles 16

3 Multicloud Existing Applications 19

31 Success Stories 19

311 The Interop challenge 19

312 European Grid Infrastructure (EGI) 19

313 NeCTAR 20

32 On-going experiences and projects 20

321 Project Tricircle 20

322 Project Trio2o 21

323 Inter Cloud Resource Federation 22

4 OpenStack Components and Federation Architecture 25

41 Components roles 25

42 OpenStack Federation 28

421 Definition 28

422 Architecture 29

43 Federation architecture and mechanisms 29

431 Discovery 30

432 Management and monitoring 30

5 Installation Guide 31

51 Centos installation 32

511 Installation 32

iv

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

CONTENTS

52 OpenStack installation 33

521 Architecture 34

522 Installation 34

5221 Every computer node 34

5222 On the controller node 36

53 Federation configuration 37

531 Architecture 38

532 Configuration Identity Provider (IdP) 38

533 Configuration Service Provider (SP) 40

6 Applications of Communication Between Federated OpenStack Cloud 43

61 Federated OpenStack Features 43

62 IaaS application examples 43

621 Workload transfer 44

63 PaaS application examples 44

631 Database synchronization 44

632 Application provisioning 44

64 SaaS application examples 45

641 Exchange of services 45

7 At-sea Environment Limitation 47

8 Conclusion and Future Work 51

Bibliography 53

Appendix A Installation Internet references A-1

A1 OpenStack A-1

A2 OpenStack Federation A-1

A3 Shibboleth A-2

v

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 2: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Template in use (2010) SR Advanced Template_EN (051115)dotm

copy Her Majesty the Queen in Right of Canada as represented by the Minister of National Defence 2017

copy Sa Majesteacute la Reine (en droit du Canada) telle que repreacutesenteacutee par le ministre de la Deacutefense nationale

2017

Coordination of Cooperative Cloud ComputingPlatforms

Mathieu LavalleeMichel MayrandElisa Shahbazian

Prepared By OODA Technologies Inc4580 Circle RdMontreal (Qc) H3W 1Y75144764773

Prepared For Defence Research amp Development Canada Atlantic Research Centre9 Grove Street PO Box 1012Dartmouth NSB2Y 3Z7902-426-3100

Scientific Authority Anthony W IsenorContract Number W7707-145677001HALCall Up Number 17Project Design develop manage test andor implement specific software or data source modulesReport Delivery Date April 7 2017

The scientific or technical validity of this Contract Report is entirely the responsibility of thecontractor and the contents do not necessarily have the approval or endorsement of Defence RampDCanada

This page is intentionally left blank

Executive Summary

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

The first part of this report addresses literature and tools review on cloud environments theirframeworks and the standards used to connect them for exchanging resources The second partof this report describes the installation of two clouds using OpenStack framework and the config-uration necessary to allow intercloud communication The next step is the investigation on howto use intercloud capability for exchanging information between two clouds In that context wedescribe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

i

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

ii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Contents

Executive Summary i

Contents iii

List of Figures vii

List of Tables ix

List of Acronyms xi

1 Introduction 1

11 Background 1

12 Document Overview 2

2 Literature Review 3

21 Research Methodology 3

211 Keywords 4

212 Glossary 6

22 Standards 10

221 Standard Inventory 10

222 Relevant Standards 10

2221 Cloud Data Management Interface (CDMI) 10

2222 Open Cloud Computing Interface (OCCI) 10

23 Cloud environments 12

iii

Final Report for RISOMIA Call-up 17

231 Cloud levels of services 12

232 Cloud software review 14

24 Tool recommendation 15

25 Other recommended books and articles 16

251 Books 16

252 Articles 16

3 Multicloud Existing Applications 19

31 Success Stories 19

311 The Interop challenge 19

312 European Grid Infrastructure (EGI) 19

313 NeCTAR 20

32 On-going experiences and projects 20

321 Project Tricircle 20

322 Project Trio2o 21

323 Inter Cloud Resource Federation 22

4 OpenStack Components and Federation Architecture 25

41 Components roles 25

42 OpenStack Federation 28

421 Definition 28

422 Architecture 29

43 Federation architecture and mechanisms 29

431 Discovery 30

432 Management and monitoring 30

5 Installation Guide 31

51 Centos installation 32

511 Installation 32

iv

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

CONTENTS

52 OpenStack installation 33

521 Architecture 34

522 Installation 34

5221 Every computer node 34

5222 On the controller node 36

53 Federation configuration 37

531 Architecture 38

532 Configuration Identity Provider (IdP) 38

533 Configuration Service Provider (SP) 40

6 Applications of Communication Between Federated OpenStack Cloud 43

61 Federated OpenStack Features 43

62 IaaS application examples 43

621 Workload transfer 44

63 PaaS application examples 44

631 Database synchronization 44

632 Application provisioning 44

64 SaaS application examples 45

641 Exchange of services 45

7 At-sea Environment Limitation 47

8 Conclusion and Future Work 51

Bibliography 53

Appendix A Installation Internet references A-1

A1 OpenStack A-1

A2 OpenStack Federation A-1

A3 Shibboleth A-2

v

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 3: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Coordination of Cooperative Cloud ComputingPlatforms

Mathieu LavalleeMichel MayrandElisa Shahbazian

Prepared By OODA Technologies Inc4580 Circle RdMontreal (Qc) H3W 1Y75144764773

Prepared For Defence Research amp Development Canada Atlantic Research Centre9 Grove Street PO Box 1012Dartmouth NSB2Y 3Z7902-426-3100

Scientific Authority Anthony W IsenorContract Number W7707-145677001HALCall Up Number 17Project Design develop manage test andor implement specific software or data source modulesReport Delivery Date April 7 2017

The scientific or technical validity of this Contract Report is entirely the responsibility of thecontractor and the contents do not necessarily have the approval or endorsement of Defence RampDCanada

This page is intentionally left blank

Executive Summary

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

The first part of this report addresses literature and tools review on cloud environments theirframeworks and the standards used to connect them for exchanging resources The second partof this report describes the installation of two clouds using OpenStack framework and the config-uration necessary to allow intercloud communication The next step is the investigation on howto use intercloud capability for exchanging information between two clouds In that context wedescribe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

i

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

ii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Contents

Executive Summary i

Contents iii

List of Figures vii

List of Tables ix

List of Acronyms xi

1 Introduction 1

11 Background 1

12 Document Overview 2

2 Literature Review 3

21 Research Methodology 3

211 Keywords 4

212 Glossary 6

22 Standards 10

221 Standard Inventory 10

222 Relevant Standards 10

2221 Cloud Data Management Interface (CDMI) 10

2222 Open Cloud Computing Interface (OCCI) 10

23 Cloud environments 12

iii

Final Report for RISOMIA Call-up 17

231 Cloud levels of services 12

232 Cloud software review 14

24 Tool recommendation 15

25 Other recommended books and articles 16

251 Books 16

252 Articles 16

3 Multicloud Existing Applications 19

31 Success Stories 19

311 The Interop challenge 19

312 European Grid Infrastructure (EGI) 19

313 NeCTAR 20

32 On-going experiences and projects 20

321 Project Tricircle 20

322 Project Trio2o 21

323 Inter Cloud Resource Federation 22

4 OpenStack Components and Federation Architecture 25

41 Components roles 25

42 OpenStack Federation 28

421 Definition 28

422 Architecture 29

43 Federation architecture and mechanisms 29

431 Discovery 30

432 Management and monitoring 30

5 Installation Guide 31

51 Centos installation 32

511 Installation 32

iv

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

CONTENTS

52 OpenStack installation 33

521 Architecture 34

522 Installation 34

5221 Every computer node 34

5222 On the controller node 36

53 Federation configuration 37

531 Architecture 38

532 Configuration Identity Provider (IdP) 38

533 Configuration Service Provider (SP) 40

6 Applications of Communication Between Federated OpenStack Cloud 43

61 Federated OpenStack Features 43

62 IaaS application examples 43

621 Workload transfer 44

63 PaaS application examples 44

631 Database synchronization 44

632 Application provisioning 44

64 SaaS application examples 45

641 Exchange of services 45

7 At-sea Environment Limitation 47

8 Conclusion and Future Work 51

Bibliography 53

Appendix A Installation Internet references A-1

A1 OpenStack A-1

A2 OpenStack Federation A-1

A3 Shibboleth A-2

v

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 4: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

This page is intentionally left blank

Executive Summary

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

The first part of this report addresses literature and tools review on cloud environments theirframeworks and the standards used to connect them for exchanging resources The second partof this report describes the installation of two clouds using OpenStack framework and the config-uration necessary to allow intercloud communication The next step is the investigation on howto use intercloud capability for exchanging information between two clouds In that context wedescribe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

i

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

ii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Contents

Executive Summary i

Contents iii

List of Figures vii

List of Tables ix

List of Acronyms xi

1 Introduction 1

11 Background 1

12 Document Overview 2

2 Literature Review 3

21 Research Methodology 3

211 Keywords 4

212 Glossary 6

22 Standards 10

221 Standard Inventory 10

222 Relevant Standards 10

2221 Cloud Data Management Interface (CDMI) 10

2222 Open Cloud Computing Interface (OCCI) 10

23 Cloud environments 12

iii

Final Report for RISOMIA Call-up 17

231 Cloud levels of services 12

232 Cloud software review 14

24 Tool recommendation 15

25 Other recommended books and articles 16

251 Books 16

252 Articles 16

3 Multicloud Existing Applications 19

31 Success Stories 19

311 The Interop challenge 19

312 European Grid Infrastructure (EGI) 19

313 NeCTAR 20

32 On-going experiences and projects 20

321 Project Tricircle 20

322 Project Trio2o 21

323 Inter Cloud Resource Federation 22

4 OpenStack Components and Federation Architecture 25

41 Components roles 25

42 OpenStack Federation 28

421 Definition 28

422 Architecture 29

43 Federation architecture and mechanisms 29

431 Discovery 30

432 Management and monitoring 30

5 Installation Guide 31

51 Centos installation 32

511 Installation 32

iv

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

CONTENTS

52 OpenStack installation 33

521 Architecture 34

522 Installation 34

5221 Every computer node 34

5222 On the controller node 36

53 Federation configuration 37

531 Architecture 38

532 Configuration Identity Provider (IdP) 38

533 Configuration Service Provider (SP) 40

6 Applications of Communication Between Federated OpenStack Cloud 43

61 Federated OpenStack Features 43

62 IaaS application examples 43

621 Workload transfer 44

63 PaaS application examples 44

631 Database synchronization 44

632 Application provisioning 44

64 SaaS application examples 45

641 Exchange of services 45

7 At-sea Environment Limitation 47

8 Conclusion and Future Work 51

Bibliography 53

Appendix A Installation Internet references A-1

A1 OpenStack A-1

A2 OpenStack Federation A-1

A3 Shibboleth A-2

v

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 5: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Executive Summary

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

The first part of this report addresses literature and tools review on cloud environments theirframeworks and the standards used to connect them for exchanging resources The second partof this report describes the installation of two clouds using OpenStack framework and the config-uration necessary to allow intercloud communication The next step is the investigation on howto use intercloud capability for exchanging information between two clouds In that context wedescribe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

i

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

ii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Contents

Executive Summary i

Contents iii

List of Figures vii

List of Tables ix

List of Acronyms xi

1 Introduction 1

11 Background 1

12 Document Overview 2

2 Literature Review 3

21 Research Methodology 3

211 Keywords 4

212 Glossary 6

22 Standards 10

221 Standard Inventory 10

222 Relevant Standards 10

2221 Cloud Data Management Interface (CDMI) 10

2222 Open Cloud Computing Interface (OCCI) 10

23 Cloud environments 12

iii

Final Report for RISOMIA Call-up 17

231 Cloud levels of services 12

232 Cloud software review 14

24 Tool recommendation 15

25 Other recommended books and articles 16

251 Books 16

252 Articles 16

3 Multicloud Existing Applications 19

31 Success Stories 19

311 The Interop challenge 19

312 European Grid Infrastructure (EGI) 19

313 NeCTAR 20

32 On-going experiences and projects 20

321 Project Tricircle 20

322 Project Trio2o 21

323 Inter Cloud Resource Federation 22

4 OpenStack Components and Federation Architecture 25

41 Components roles 25

42 OpenStack Federation 28

421 Definition 28

422 Architecture 29

43 Federation architecture and mechanisms 29

431 Discovery 30

432 Management and monitoring 30

5 Installation Guide 31

51 Centos installation 32

511 Installation 32

iv

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

CONTENTS

52 OpenStack installation 33

521 Architecture 34

522 Installation 34

5221 Every computer node 34

5222 On the controller node 36

53 Federation configuration 37

531 Architecture 38

532 Configuration Identity Provider (IdP) 38

533 Configuration Service Provider (SP) 40

6 Applications of Communication Between Federated OpenStack Cloud 43

61 Federated OpenStack Features 43

62 IaaS application examples 43

621 Workload transfer 44

63 PaaS application examples 44

631 Database synchronization 44

632 Application provisioning 44

64 SaaS application examples 45

641 Exchange of services 45

7 At-sea Environment Limitation 47

8 Conclusion and Future Work 51

Bibliography 53

Appendix A Installation Internet references A-1

A1 OpenStack A-1

A2 OpenStack Federation A-1

A3 Shibboleth A-2

v

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 6: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

ii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Contents

Executive Summary i

Contents iii

List of Figures vii

List of Tables ix

List of Acronyms xi

1 Introduction 1

11 Background 1

12 Document Overview 2

2 Literature Review 3

21 Research Methodology 3

211 Keywords 4

212 Glossary 6

22 Standards 10

221 Standard Inventory 10

222 Relevant Standards 10

2221 Cloud Data Management Interface (CDMI) 10

2222 Open Cloud Computing Interface (OCCI) 10

23 Cloud environments 12

iii

Final Report for RISOMIA Call-up 17

231 Cloud levels of services 12

232 Cloud software review 14

24 Tool recommendation 15

25 Other recommended books and articles 16

251 Books 16

252 Articles 16

3 Multicloud Existing Applications 19

31 Success Stories 19

311 The Interop challenge 19

312 European Grid Infrastructure (EGI) 19

313 NeCTAR 20

32 On-going experiences and projects 20

321 Project Tricircle 20

322 Project Trio2o 21

323 Inter Cloud Resource Federation 22

4 OpenStack Components and Federation Architecture 25

41 Components roles 25

42 OpenStack Federation 28

421 Definition 28

422 Architecture 29

43 Federation architecture and mechanisms 29

431 Discovery 30

432 Management and monitoring 30

5 Installation Guide 31

51 Centos installation 32

511 Installation 32

iv

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

CONTENTS

52 OpenStack installation 33

521 Architecture 34

522 Installation 34

5221 Every computer node 34

5222 On the controller node 36

53 Federation configuration 37

531 Architecture 38

532 Configuration Identity Provider (IdP) 38

533 Configuration Service Provider (SP) 40

6 Applications of Communication Between Federated OpenStack Cloud 43

61 Federated OpenStack Features 43

62 IaaS application examples 43

621 Workload transfer 44

63 PaaS application examples 44

631 Database synchronization 44

632 Application provisioning 44

64 SaaS application examples 45

641 Exchange of services 45

7 At-sea Environment Limitation 47

8 Conclusion and Future Work 51

Bibliography 53

Appendix A Installation Internet references A-1

A1 OpenStack A-1

A2 OpenStack Federation A-1

A3 Shibboleth A-2

v

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 7: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Contents

Executive Summary i

Contents iii

List of Figures vii

List of Tables ix

List of Acronyms xi

1 Introduction 1

11 Background 1

12 Document Overview 2

2 Literature Review 3

21 Research Methodology 3

211 Keywords 4

212 Glossary 6

22 Standards 10

221 Standard Inventory 10

222 Relevant Standards 10

2221 Cloud Data Management Interface (CDMI) 10

2222 Open Cloud Computing Interface (OCCI) 10

23 Cloud environments 12

iii

Final Report for RISOMIA Call-up 17

231 Cloud levels of services 12

232 Cloud software review 14

24 Tool recommendation 15

25 Other recommended books and articles 16

251 Books 16

252 Articles 16

3 Multicloud Existing Applications 19

31 Success Stories 19

311 The Interop challenge 19

312 European Grid Infrastructure (EGI) 19

313 NeCTAR 20

32 On-going experiences and projects 20

321 Project Tricircle 20

322 Project Trio2o 21

323 Inter Cloud Resource Federation 22

4 OpenStack Components and Federation Architecture 25

41 Components roles 25

42 OpenStack Federation 28

421 Definition 28

422 Architecture 29

43 Federation architecture and mechanisms 29

431 Discovery 30

432 Management and monitoring 30

5 Installation Guide 31

51 Centos installation 32

511 Installation 32

iv

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

CONTENTS

52 OpenStack installation 33

521 Architecture 34

522 Installation 34

5221 Every computer node 34

5222 On the controller node 36

53 Federation configuration 37

531 Architecture 38

532 Configuration Identity Provider (IdP) 38

533 Configuration Service Provider (SP) 40

6 Applications of Communication Between Federated OpenStack Cloud 43

61 Federated OpenStack Features 43

62 IaaS application examples 43

621 Workload transfer 44

63 PaaS application examples 44

631 Database synchronization 44

632 Application provisioning 44

64 SaaS application examples 45

641 Exchange of services 45

7 At-sea Environment Limitation 47

8 Conclusion and Future Work 51

Bibliography 53

Appendix A Installation Internet references A-1

A1 OpenStack A-1

A2 OpenStack Federation A-1

A3 Shibboleth A-2

v

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 8: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

231 Cloud levels of services 12

232 Cloud software review 14

24 Tool recommendation 15

25 Other recommended books and articles 16

251 Books 16

252 Articles 16

3 Multicloud Existing Applications 19

31 Success Stories 19

311 The Interop challenge 19

312 European Grid Infrastructure (EGI) 19

313 NeCTAR 20

32 On-going experiences and projects 20

321 Project Tricircle 20

322 Project Trio2o 21

323 Inter Cloud Resource Federation 22

4 OpenStack Components and Federation Architecture 25

41 Components roles 25

42 OpenStack Federation 28

421 Definition 28

422 Architecture 29

43 Federation architecture and mechanisms 29

431 Discovery 30

432 Management and monitoring 30

5 Installation Guide 31

51 Centos installation 32

511 Installation 32

iv

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

CONTENTS

52 OpenStack installation 33

521 Architecture 34

522 Installation 34

5221 Every computer node 34

5222 On the controller node 36

53 Federation configuration 37

531 Architecture 38

532 Configuration Identity Provider (IdP) 38

533 Configuration Service Provider (SP) 40

6 Applications of Communication Between Federated OpenStack Cloud 43

61 Federated OpenStack Features 43

62 IaaS application examples 43

621 Workload transfer 44

63 PaaS application examples 44

631 Database synchronization 44

632 Application provisioning 44

64 SaaS application examples 45

641 Exchange of services 45

7 At-sea Environment Limitation 47

8 Conclusion and Future Work 51

Bibliography 53

Appendix A Installation Internet references A-1

A1 OpenStack A-1

A2 OpenStack Federation A-1

A3 Shibboleth A-2

v

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 9: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

CONTENTS

52 OpenStack installation 33

521 Architecture 34

522 Installation 34

5221 Every computer node 34

5222 On the controller node 36

53 Federation configuration 37

531 Architecture 38

532 Configuration Identity Provider (IdP) 38

533 Configuration Service Provider (SP) 40

6 Applications of Communication Between Federated OpenStack Cloud 43

61 Federated OpenStack Features 43

62 IaaS application examples 43

621 Workload transfer 44

63 PaaS application examples 44

631 Database synchronization 44

632 Application provisioning 44

64 SaaS application examples 45

641 Exchange of services 45

7 At-sea Environment Limitation 47

8 Conclusion and Future Work 51

Bibliography 53

Appendix A Installation Internet references A-1

A1 OpenStack A-1

A2 OpenStack Federation A-1

A3 Shibboleth A-2

v

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 10: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

A4 Keystone A-2

A5 Wiki and Blogs A-2

vi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 11: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

LIST OF FIGURES

List of Figures

21 OCCI Architecture 12

22 Types of cloud services 13

31 Tricircle Architecture 21

32 Trio2o Architecture 22

33 Inter Cloud Resource Federation Architecture 23

41 OpenStack component architecture 25

42 OpenStack component interaction 27

43 Single sign-on using SAML in a Web browser 29

51 Cloud federation proposed architecture 34

52 Identity Provider Configuration 38

53 Service Provider Configuration 40

71 DoD Enterprise Cloud Environment Source [15] 48

vii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 12: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

viii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 13: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

LIST OF TABLES

List of Tables

21 Primary search Keywords 4

22 Secondary Search Keywords 5

23 Glossary of cloud related definitions 6

24 Cloud standard list by Cloud Standards Customer Council (CSCC) 11

25 Available cloud software 14

25 Available cloud software 15

51 Configuration modification for each cloud controller 42

ix

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 14: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

x

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 15: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

List of Acronyms

List of Acronyms

API Application Programming Interface

AWS Amazon Web Service

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

DaaS Data Storage as a Service

DMTF Distributed Management Task Force

DNS Domain Name System

EGI European Grid Infrastructure

HTTP Hypertext Transfer Protocol

IaaS Infrastructure as a Service

IdP Identity Provider

NIST National Institute of Standards and Technology

OASIS Organization for the Advancement of Structured Information Standards

OCCI Open Cloud Computing Interface

OGF Open Grid Forum

PaaS Platform as a Service

REST Representational State Transfer

RHEL RedHat Enterprise Linux

SaaS Software as a Service

SNIA Storage Networking Industry Association

SP Service Provider

SSO Single Sign On

xi

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 16: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

xii

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 17: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 1

Introduction

11 Background

Cloud computing has infiltrated multiple aspects of technology solutions for industry as well aseveryday life It provides many benefits fault tolerance to hardware failure automatic updateand backup scalability flexibility accessibility facilitation of collaboration and many more Thecloud provides the resources to process real-time commercialfinancial transactions social networkanalysis data mining and all kinds of algorithms which benefit from a large farm of processorsor need to process a large quantity of data or both The tools for managingmonitoring a cloudhave evolved to a point that it is now possible for an individual to build his or her own cloud athome thanks to state-of-the-art open-source software Now the new trend is to understand how tocoordinate resources across multiple homogeneous or heterogeneous clouds and to create the stan-dards and the tools to achieve intercloud communication In the context of a maritime operationregrouping a fleet where each ship has its own private cloud the utility of intercloud communica-tion becomes obvious automatic database exchange or merge remote hardware resource sharingredundancy remote service providers identity authentication etc

Many questions arise naturally

bull What is the state of these cloud framework environments

bull Have they achieved a certain level of maturity Are they stable

bull Are they easy to use

bull What can we do with it And cannot be done

bull Are they secured

This study will try to provide some answers to these questions

The first part of this report reflects on the literature and tools review on cloud environments theirframeworks and the standards used to inter-connect them enabling exchanging resources The

1

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 18: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

second part of this report describes the installation of two clouds using OpenStack framework andthe configuration necessary to allow intercloud communication The next step is the investigationon how to use intercloud capability for exchanging information between two clouds In that contextwe describe what are the possibilities for intercloud communication in the context of an OpenStackfederation at different levels Infrastructure as a Service (IaaS) Platform as a Service (PaaS) andSoftware as a Service (SaaS) The main goal of this study is to assess the existing capabilities ofthe state-of-the-art cloud environments for at-sea interoperability assuming none of the constraintsand limitations that at-sea environments may impose A short description of such constraints andlimitations is included at the end of this report

12 Document Overview

This document is organized as follows

bull Section 2 presents a literature review of cloud technology which includes a description of cloudstandards developed by the open-source community and the industryrsquos main contributors

bull Section 3 enumerates previous success stories of multicloud deployment as well as on-goingprojects related to cloud federation

bull Section 4 presents the core components of the OpenStack architecture and their role in amulticloud environment

bull Section 6 presents what is possible to do in a federated multicloud environment

bull Section 7 presents possible limitations of cloud computing in an at-sea environment and whatneeds to be done to overcome them

bull Section 8 presents the lessons learned possible future work and concluding remarks

2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 19: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 2

Literature Review

21 Research Methodology

Very early in the literature review we made these observations

1 Cloud usage has been spreading to many aspects of our society and there is no sign that itwill slow down

2 Cloud development is now accessible not only to large corporations but also to individuals

3 Cloud technology is evolving extremely fast

Consequently it becomes clear that reviewing out-of-date information on cloud technologies wouldbe a waste of energy For these reasons we focused only on the very recent tools developed forcloud deployment and standards

What is happening in cloud software development and why is it going so fast Many contributorsare working in parallel with each team implementing and testing new code to obtain the certi-fication to merge their features It leads to frequent releases For example in a previous report[1] more than half of an open-source cloud framework (Openshift) was redesigned within the timeframe of the analysis of the tool We observe a similar situation in the evolution of the virtualizationsoftware Docker which is widely used in cloud environments and where each release introduces ma-jor features and is rarely backward compatible Sadly a side effect of this fast-track development ispoor documentation not in the sense of low quantity (see httpsdocsopenstackorg) but inthe sense of its usefulness (multiple out-of-date versions contradictory or unclear instructions lackof examples etc) The software evolves faster than it takes to write or update the documentationContributors probably do not invest much time in good quality documentation because

1 these tools are intended for only a relatively small number of network administrators whocan figure out by themselves how to use them and how to debug them

2 they are open-source in the sense that it may not be always sufficiently founded

3

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 20: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

3 the focus is put on adding new features and fast growth instead of productization and properdocumentation

4 and finally a very hungry cloud-driven society in the making

There is less value in documenting and studying a deprecated software This is why we focus thisresearch on the very recent developments The early developments of the cloud while interestingwould have little impact on the outcome of this investigation

211 Keywords

The tool survey was mainly driven by searching the web but also based on our past experiencewith similar technologies (Openshift see [1]) Tables 21 and 22 provide the list of keywords usedto query the Google search engine

Table 21 Primary search Keywords

Keywords Notes

Cloud interoperability Often defines standard action of interfacing that can be reused acrossmultiple cloud implementations

Intercloud Defines interaction between multiple clouds Used in standard docu-mentation

Multicloud Yet another word for interaction between clouds not used very often

Cloud federation A communication between clouds with a mutually trusted authenti-cation and authorization mechanism Usually implies a homogeneousmulticloud environment

Hybrid cloud Used to define a public-private cloud interaction Usually implies aheterogeneous multicloud environment

k2k Keystone to keystone ndash Used to describe a cloud federation whichuses Keystone as an authentication mechanism

TripleO OpenStack On OpenStack ndash A project to deploy a production cloudover a provisioning cloud

SSO Single Sign On

OpenStack Open-source IaaS framework

4

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 21: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 2 Literature Review

Table 21 Primary search Keywords

RDO RDO is a community of people using and deploying OpenStack onCentOS Fedora and Red Hat Enterprise Linux A community sup-ported distribution of OpenStack based on the upstream source codefor Redhat Linux family

Public cloud Company that provides cloud services (IaaS PaaS SaaS) to the pub-lic usually for a fee

Private cloud Cloud infrastructure where the providers developers and users areall under the same company or organization

From the first set of articles found with the above keywords we discovered another set of technicalkeywords related to cloud technology that was useful in refining our research queries

Table 22 Secondary Search Keywords

Keywords Notes

CDMI Cloud Data Management Interface

CSCC Cloud Standards Customer Council

Devstack Development group for OpenStack

Packstack Semi-automated installation tool for OpenStack using Puppet

SAML Security Assertion Markup Language

IDP Identity Provider

SP Service Provider

Horizon OpenStack Dashboard

Keystone OpenStack identity service

Shibboleth Implementation of SAML2

Cloud bursting Mechanism by which a task is moved from one cloud to another (usually from aprivate to a public cloud) for load-balancing purpose

Tricircle Provide networking automation across the module Neutron in multiple OpenStackdeployments

5

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 22: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

212 Glossary

A glossary made from the definitions found on the web is presented to provide a better under-standing of the following sections of this study

For a longer and more complete cloud related glossary see reference [13]

Table 23 Glossary of cloud related definitions

Item Definition

OpenStack ldquoOpenStack is a free and open-source software platform forcloud computing mostly deployed as an infrastructure-as-a-service (IaaS)1rdquo

DevStack ldquoDevStack is a series of extensible scripts used to quicklybring up a complete OpenStack environment based on thelatest versions of everything from git master It is usedinteractively as a development environment and as the basisfor much of the OpenStack projectrsquos functional testing2rdquo

PackStack ldquoPackStack is a command line utility that uses Puppet3

modules to support rapid deployment of OpenStack on ex-isting servers over an SSH connection PackStack is suitablefor deploying both single node proof of concept installationsand more complex multi-node installations Deploymentoptions are provided either interactively via the commandline or via a text file containing preconfigured answers tothe questions PackStack asks4rdquo

Puppet ldquoPuppet provides a standard way of delivering and oper-ating software no matter where it runs With the Puppetapproach you define what you want your apps and infras-tructure to look like using a common easy-to-read languageFrom there you can share test and enforce the changes youwant to make across your data centre And at every stepof the way you have the visibility and reporting you needto make decisions and prove compliance The result youget a standard way of automating all of it at scale5rdquo

1 httpsenwikipediaorgwikiOpenStack2 httpsdocsopenstackorgdeveloperdevstack3 httpwwwpuppetlabscom4 httpsaccessredhatcomdocumentationen-USRed_Hat_Enterprise_Linux_OpenStack_Platform2

htmlGetting_Started_Guidepart-Deploying_OS_using_PackStackhtml5 httpspuppetcomproducthow-puppet-works

6

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 23: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

Horizon (Open-Stack)

ldquoHorizon is the canonical implementation of OpenstackrsquosDashboard which provides a web-based user interface toOpenStack services including Nova Swift Keystone etc6rdquo

ldquoHorizon ships with three central dashboards a User Dash-board a System Dashboard and a Settings dashboard Be-tween these three they cover the core OpenStack applica-tions and deliver on Core Support The Horizon applicationalso ships with a set of API abstractions for the core Open-Stack projects in order to provide a consistent stable set ofreusable methods for developers Using these abstractionsdevelopers working on Horizon donrsquot need to be intimatelyfamiliar with the APIs of each OpenStack project7rdquo

Federation (Open-Stack)

ldquoIn a standard OpenStack setup users authenticate againstthe identity APIs usually implemented by the Keystoneservice With federated Keystone the options for authenti-cation are greatly increased since it outsources that respon-sibility to an external Identity Provider (IdP) The ServiceProvider Keystone (SP) can instead just focus on manag-ing access to resources on the cloud where it resides As aresult a specific group of users in a trusted IdP service canaccess the cloud services without needing explicit accountson the SP8rdquo

Security AssertionMarkup Language(SAML)

ldquoSAML is an Extensible Markup Language (XML) -basedopen-standard data format for exchanging authenticationand authorization data between parties in particular be-tween an identity provider and a service provider9rdquo

6 httpswikiopenstackorgwikiHorizon7 httpsdocsopenstackorgdeveloperhorizonintrohtml8 httpsplatform9comblogopenstack-keystone-federation9 httpsenwikipediaorgwikiSecurity_Assertion_Markup_Language

7

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 24: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

Table 23 Glossary of cloud related definitions

Keystone (Open-Stack)

ldquoKeystone is an OpenStack identity service that managesuser databases and OpenStack service catalogs and theirAPI endpoints It integrates with existing backend direc-tory services like LDAP and supports multiple authentica-tion mechanisms such as username-and-password token-based systems and AWS-style logins10rdquo ldquoIt currently sup-ports token-based authN and user-service authorizationIt has recently been rearchitected to allow for expansionto support proxying external services and AuthNAuthZmechanisms such as oAuth SAML and openID in futureversions11rdquo

Single Sign On(SSO)

ldquoSingle sign-on (SSO) is a session and user authenticationservice that permits a user to use one set of login credentials(eg name and password) to access multiple applicationsThe service authenticates the end user for all the applica-tions the user has been given rights to and eliminates fur-ther prompts when the user switches applications duringthe same session On the back end SSO is helpful for log-ging user activities as well as monitoring user accounts12rdquo

Shibboleth ldquoShibboleth is an open-source project that provides Sin-gle Sign-On capabilities and allows sites to make informedauthorization decisions for individual access of protectedonline resources in a privacy-preserving manner13rdquo

ldquoThe Shibboleth Internet2 middleware initiative created anarchitecture and open-source implementation for identitymanagement and federated identity-based authenticationand authorization (or access control) infrastructure basedon Security Assertion Markup Language (SAML) Feder-ated identity allows the sharing of information about usersfrom one security domain to the other organizations in afederation This allows for cross-domain single sign-on andremoves the need for content providers to maintain usernames and passwords Identity providers (IdPs) supplyuser information while service providers (SPs) consumethis information and give access to secure content14rdquo

10 httpblogflux7comblogsopenstacktutorial-what-is-keystone-and-how-to-install-keystone-in-openstack11 httpswikiopenstackorgwikiKeystone12 httpsearchsecuritytechtargetcomdefinitionsingle-sign-on13 httpwwwpuppetlabscom14 httpsenwikipediaorgwikiShibboleth_(Internet2)

8

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 25: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 2 Literature Review

Table 23 Glossary of cloud related definitions

token ldquoA software token is a type of two-factor authenticationsecurity device that may be used to authorize the use ofcomputer services Software tokens are stored on a general-purpose electronic device such as a desktop computer lap-top PDA or mobile phone and can be duplicated15 rdquo

Identity Provider(IDP)

ldquoThe Identity Provider provides Single Sign-On servicesand extends reach into other organizations and new ser-vices through authentication of users and securely provid-ing appropriate data to requesting services In addition toa simple yesno response to an authentication request theIdentity Provider can provide a rich set of user-related datato the Service Provider This data can help the service pro-vide a more personalized user experience save the user fromhaving to manually enter data the service requires and re-fresh the data each time the user logs onto the service16rdquo

Service Provider(SP)

ldquoA cloud provider is a company that offers some compo-nent of cloud computing ndash typically Infrastructure as aService (IaaS) Software as a Service (SaaS) or Platformas a Service (PaaS) ndash to other businesses or individualsCloud providers are sometimes referred to as cloud serviceproviders or CSPs17 rdquo

Web Server Gate-way Interface(WSGI)

ldquoThe Web Server Gateway Interface (WSGI) is a specifica-tion for simple and universal interface between web serversand web applications or frameworks for the Python pro-gramming language18rdquo

ldquoIt is just an interface specification by which server andapplication communicate Both server and application in-terface sides are specified in the Python Enhancement Pro-posals (PEP) 3333 which is the Python Web Server Gate-way Interface If an application (or framework or toolkit)is written to the WSGI spec then it will run on any serverwritten to that spec19rdquo

15 httpsenwikipediaorgwikiSoftware_token16 httpsshibbolethnetproductsidentity-providerhtml17 httpsearchcloudprovidertechtargetcomdefinitioncloud-provider18 httpsenwikipediaorgwikiWeb_Server_Gateway_Interface19 httpwsgitutorialcodepointnet

9

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 26: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

22 Standards

There are many organizations involved in the cloud ecosystem These organizations gather togetherto define standards This section enumerates and describes existing standards relevant to cloudinteroperability and communications

221 Standard Inventory

The cloud computing ecosystem is wide and it offers a broad range of services and applicationsThe standards define the foundation of a cloud ecosystem by ensuring the compatibility andcommunication of every component toward each other The standards are required in order toachieve communication between multiple clouds The Cloud Standards Customer Council (CSCC)is an organism that helps define best practices patterns case studies and standards The CSCCcompiled a list of standards applicable to many components of the cloud and is shown in Table24

222 Relevant Standards

Two of the standards described in table 24 stand out from the others the Cloud Data ManagementInterface (CDMI) and the Open Cloud Computing Interface (OCCI)

2221 CDMI

The CDMI is a SNIA standard that specifies a protocol for self-provisioning administering andaccessing cloud storage This standard defines abstraction of complexity and promotes the sim-plicity of use as cloud resources It defines the Data Storage as a Service (DaaS) interface formanaging all kinds of data stored in the cloud The functionalities defined in this standard includethe operation and management of data objects containers accounts capabilities queues and datasystem metadata through a Representational State Transfer (REST)ful Hypertext Transfer Proto-col (HTTP) Application Programming Interface (API) The CDMI interface is designed to workalongside OpenStack storage interface as of release v11 in March 2015

2222 OCCI

The Open Cloud Computing Interface (OCCI) is a set of specifications delivered through theOGF for the standard services of cloud computing service development The OCCI goals areinteroperability portability integration and extensibility The interoperability goal is achievedby defining a single interface to interact with every cloud without having to manage multiple

10

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 27: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 2 Literature Review

Standard Description Authority

Open Visualization For-mat

A packaging standard that is designed to addressthe portability and deployment of virtual machines

Distributed Man-agement TaskForce (DMTF)

Cloud Data Manage-ment Interface (CDMI)

Defines the functional interface that applicationswill use to create retrieve update and delete dataelements from the cloud

Storage Network-ing Industry As-sociation (SNIA)

Open Cloud ComputingInterface

A set of open specifications that define a protocoland APIs for all kinds of cloud computing manage-ment tasks

Open Grid Forum(OGF)

Topology and Orches-tration Specificationfor Cloud Applications(TOSCA)

Description of applications and infrastructure cloudservices of the relationships between parts of theservice and of the operational behavior of theseservices (eg deploy patch shutdown)

Organization forthe Advancementof StructuredInformation Stan-dards (OASIS)

Cloud ApplicationManagement for Plat-forms (CAMP)

Defines an interoperability protocol that cloud de-signers can use to package and deploy their appli-cations

OASIS

Cloud Auditing DataFederation (CADF)

Define open standards for cloud auditing DMTF

LDAP OAuth OpenIDConnect and SAML

Standards that enable third party ID and AccessManagement functionality

Many implemen-tations

US FIPS 140-2

A standard that specifies the security requirementswhich need to be satisfied by a cryptographic mod-ule used within a security system protecting sensi-tive information

National Instituteof Standardsand Technol-ogy (NIST)

Table 24 Cloud standard list by CSCC

11

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 28: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

dependencies on multiple APIs The objective of portability is to avoid technical lock-in and toenable the client to switch providers with minimal technical costs The integration involves thedefinition of a stable interface in order to support the legacy and the latest infrastructure Finallythe extensibility goal is achieved by using the meta-model and capabilities discovery feature to beable to interact with any OCCI server using the providerrsquos extensions The OCCI client systeminteraction is illustrated in the Figure 21 There is an OpenStack implementation of the OCCIinterface that interacts directly with the OpenStack API20

Figure 21 OCCI Architecture Source httpocci-wgorgabout

23 Cloud environments

Cloud infrastructure is really complex there are a lot of terms and keywords that are often abusedor misinterpreted Levels of cloud services (IaaS PaaS and SaaS) are often confused because somecloud providers offer many levels of services

231 Cloud levels of services

The following subsections define levels of cloud services with clear definitions of cloud providersand cloud user roles Furthermore it explains how each level plays with other levels

bull The IaaS cloud provider owns hardware and network resources The IaaS is intended for aconsumer who is a developer or system administrator The cloud provider runs software to

20 httpsgithubcomopenstackooi

12

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 29: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 2 Literature Review

enable consumers to deploy and manage virtual machine The IaaS providerrsquos challenge isensuring constant performance and reliability to the customer The cloud consumer use theresources to install an operating system and software without bothering with purchasing thehardware or dealing with hardware failures Popular IaaS public providers are Amazon WebService (AWS) Microsoft Azure and Rackspace OpenStack is a solution to run a privateIaaS

bull The PaaS cloud provider offers ready-to-run infrastructure for developers The providerrsquosrole is to ensure standard running instances such as defined python Tomcat Docker containerruntime Postgresql instance MongoDB etc The PaaS consumers are application developersaiming for a controlled and stable environment to ease testing and deployment of applicationsThe PaaS provider can offer the hardware infrastructure and the PaaS software as a wholeor rely on a third party IaaS to host their PaaS software Popular PaaS providers areGoogle Appengine Heroku OpenShift Among the PaaS mentioned Heroku21 and OpenShiftOnline host their IaaS software on AWS OpenShift community is a PaaS platform for PaaSproviders

Figure 22 Types of cloud services Source httpotssiPrejsnje_konferenceOTS_2011

imagesfilestrust_v3Gabipdf

bull The SaaS cloud provider offers an application (often a web-based application) to the con-sumer The consumer of the cloud would be any cloud user The SaaS provider may run ona PaaS or on a IaaS The Software provided as a service can be a web application a web

21 httpsdevcenterherokucomarticlesregionsdata-center-locations

13

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 30: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

service or may use any protocol Some of the most popular IaaS are Gmail DropBox GoogleDocument etc Dropbox used to run on AWS22

Cloud services offer many levels of abstraction and target different consumers for each level Thelink between levels of services and consumer is shown in figure 22

232 Cloud software review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

fluid Op-erations eCloudManager

2009-03-01 Proprietary JavaGroovy

No Yes

AppScale 2009-03-07 Apache License[2] PythonRuby Go

Yes Yes

Cloud Foundry 2011-04-12 Apache License Ruby CJava Go

Yes Yes

CloudcomCloudStack

2010-05-04 Apache license Java C Yes Yes

Eucalyptus 2008-05-29 Proprietary GPLv3

Java C Yes Yes

Flexiant Lim-ited

2007-01-15 Proprietary soft-ware

Java C Yes Yes

Nimbus 2009-01-09 Apache License JavaPython

Yes Yes

OpenNebula 2008-03- Apache License C++ CRuby JavaShell scriptlex yacc

Yes Yes

22 httpsblogsdropboxcomtech201603magic-pocket-infrastructure

14

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 31: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 2 Literature Review

Table 25 Available cloud software Source httpsen

wikipediaorgwikiCloud_computing_comparison

Software Initialrelease

License(s) Written in As aservice

Localinstal-lation

OpenQRM 2008-03- GPL License C++ PHPShell script

Yes Yes

OpenShift 2011-05-04 Apache License Go Yes Yes

OpenStack 2010-10-21 Apache License Python Yes Yes

OnApp 2010-07-01 Proprietary Java RubyC++

Yes Yes

oVirt 2012-08-09 Apache License JavaPython

Yes

Jelastic 2011-01-27 GPL LicenseApache LicenseBSD License

JavaJavaScriptPerl Shellscript

Yes Yes

24 Tool recommendation

From the above results we conclude that DRDC choice to go with the cloud tool OpenStack forits private cloud deployment is sound and justified Putting aside the faulty documentation thereare many other factors demonstrating that OpenStack is the right choice Firstly there is a hugecommunity composed of the major players in IT companies that are contributing to the OpenStackSoftware23 Second many of the contributing companies offer support The fact that OpenStackis compatible with two of the most important standards is a important factor for interoperabilityAlso OpenStack has an important position in last yearrsquos trends survey for private cloud usage24

23 httpstackalyticscom24 httpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2016-state-cloud-survey

15

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 32: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

25 Other recommended books and articles

This section underlines some of the scarce but recent and interesting publications on the subjectof intercloud communications and mainly cloud federations Note that these references differ a lotfrom the online documentation of open-source cloud framework As mentioned before the latterwas often deprecated contradictory rarely updated and very technical On the other hand thefollowing references are in-depth documents on cloud technology and future developments

251 Books

Here are some of the books we found

1 Identity Authentication and Access Management in OpenStack Implementing and Deploy-ing Keystone [10] Some parts are accessible from GoogleBooks25 Easy to read Approachesbasic concepts It focuses on Keystone which is the portal between clouds in a federation

2 OpenStack Cloud Computing Cookbook - Third Edition [8] This book describes each Open-Stack module and describes the installation and configuration of each module This editioncovers the Juno and Kilo version of OpenStack but not the latest versions

3 Openstack Essentials [11] Mixed reviews Can be helpful for starting up but may not besufficient for more advanced features Only about single OpenStack implementation withnothing for federation of OpenStack

4 Intercloud Solving Interoperability and Communication in a Cloud of Clouds [3] The URLprovided in the reference shows the first fifty pages of the book The book that is the mostrelated to this study intercloud communication Easy to read but too much Cisco-Oriented

5 OpenStack cloud security [9] Better than OpenStack Essentials from the same series Atleast it mentions federation identity related information but it can become rapidly limitedfor onersquos need

6 OpenStack Operations Guide [13] Similar to Openstack Essentials but more detailed foreach OpenStack module Very little on federations Very large cloud-related glossary

Of the books listed above only one [3] addresses in depth the subject of intercloud communication

252 Articles

More updated and related information can be found in some articles

1 In Tartarini [14] we have a good tutorial of how a federation of cloud is set up at CERNAlthough the report is slightly old (2014) compared to other references we make an exception

25 httpsbooksgooglecabooksid=nZYpCwAAQBAJamppg=PT16ampsource=gbs_toc_rampcad=4v=onepageampqampf=false

16

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 33: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 2 Literature Review

for this reference as it provides nevertheless a very good introduction on different conceptsof intercloud technologies

2 In Assis and Bittencourt [2] they survey different types of cloud federation architectures

3 In Lee [5] the author describes how to approach the management of a federation

4 In [6] and [12] the articles describe setting up of a federated OpenStack system

The articles provide better understanding of OpenStack federations but these articles are stillpartially out-of-date In order to really understand the OpenStack framework we have to fall backon the software documentation (httpsdocsopenstackorg)

17

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 34: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

18

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 35: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 3

Multicloud Existing Applications

OpenStack was first released in 2010 Since then a lot of fans of this technology have deployed aprivate cloud in their infrastructure These users have established a baseline that enables Open-Stack community to evolve and improve This section refers to success stories and on-going projectsrelated to multicloud and interoperability

31 Success Stories

Three success stories that have helped improve OpenStack through their lessons and recommen-dations are described below

311 The Interop challenge

The Interop challenge1 has been submitted to the major players of the OpenStack community Themain goal was to achieve complete interoperability between different OpenStack cloud providersThe group of providers worked together to develop a common and standard operation to deploycloud resources They succeeded and developed orchestration tools that facilitate deployment ofcloud resources using templates

Interoperability is the ability of a system or a product to work with other systemsor products without special effort on the part of the customer2

312 European Grid Infrastructure (EGI)

EGI is an organization thatrsquos being developed to coordinate the European Grid Infrastructurebased on the federation of individual National Grid Infrastructures to support a multi-disciplinary

1 httpswikiopenstackorgwikiInterop_Challenge2 httpsearchmicroservicestechtargetcomdefinitioninteroperability

19

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 36: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

user community The EGI regroups the infrastructure into two realms The Open Standardrealm and the OpenStack realm The Open Standard Realm regroups heterogeneous IAAS cloudtechnology together using the OCCI standard The OpenStack realm while compatible with theOCCI standard uses the OpenStack native API The EGI cloud infrastructure provides servicesof identification monitoring a Virtual Machine image catalog a service registry and many more

313 NeCTAR

NeCTAR3 is similar to the EGI cloud It is an Australian government funded federated clouddistributed across Australia and made for research purposes

Each site runs its own computing nodes object storage and virtual machine image storage servicesThe whole infrastructure shares the same authentication services (keystone) a virtual machineimage registry and a dashboard When a site uses other sitesrsquo computing power they need to payback in computing power time

32 On-going experiences and projects

This section presents an overview of projects and of architectures of communicating clouds

321 Project Tricircle

This project aims to provide a networking automation across multi-region OpenStack deploymentThe project enables networking between two clouds by connecting local Neutron (network services)servers using a coordinator Neutron server The coordinating Neutron ensures the managementof the IP address pool of the IPMAC address allocation and of the network segment allocationwithout conflicts across clouds This type of configuration enables fast communication betweenclouds using low level Internet protocols (L2 and L3) The common architecture is illustrated inthe figure 31

From the control plane view (cloud management view ) Tricircle is to make Neutron(s)in multi-region OpenStack clouds working as one cluster and enable the creation ofglobal networkrouter etc abstract networking resources across multiple OpenStackclouds From the data plane view (end user resources view) all VMs (could alsobe bare metal servers or containers) are provisioned in different clouds but can beinterconnected via the global abstract networking resources of course with tenantlevel isolation4

3 httpswwwopenstackorguser-storiesnectar

httpsnectarorgaucloudpage

httpswwwopenstackorgsummitportland-2013session-videospresentation

13-months-of-operations-a-federated-cloud-serving-the-broader-research-community4 httpswikiopenstackorgwikiTricircle

20

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 37: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 3 Multicloud Existing Applications

Figure 31 Tricircle Architecture Source httpswikiopenstackorgwikiTricircle

322 Project Trio2o

This project aims to implement an API gateway between two OpenStack clouds and is derived fromthe Tricircle project Trio2o is based on a top-bottom architecture using a top Trio2o instance infront of multiple pods as seen in the Figure 31 Trio2o provides storage service gateway (novacinder) Trio2o proposes to use one instance of keystone to authenticate users

21

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 38: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

Figure 32 Trio2o Architecture Source httpswikiopenstackorgwikiTrio2o

323 Inter Cloud Resource Federation

The Inter Cloud Resource Federation project5 aims to provide an architecture of two independentclouds to establish a federation The figure 33 shows how a user would authenticate to anothercloud through his own keystone instance Once authenticated the user can request the resourcescatalog from any keystone Keystone the authentication module of OpenStack acts as an IdentityProvider and as a Service Provider The keystone federation project was integrated to the corekeystone API

5 httpswikiopenstackorgwikiInter_Cloud_Resource_Federation

22

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 39: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 3 Multicloud Existing Applications

Figure 33 Inter Cloud Resource Federation Architecture Source httpswikiopenstack

orgwikiInter_Cloud_Resource_Federation

23

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 40: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

24

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 41: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 4

OpenStack Components andFederation Architecture

OpenStack is a cloud system that controls large pools of computing storage and networkingresources through a data centre Everything can be managed through a dashboard that givescontrol to the administrators while empowering the users to provision resources through a webinterface OpenStack assembles many components operating together to provide a fully workingIaaS This section describes every core component of OpenStack and their role in a multicloudenvironment

41 Components roles

The componentsrsquo roles and names are displayed in the 41 figure

Figure 41 OpenStack component architecture Source httpsdocsopenstackorg

security-guideintroductionintroduction-to-openstackhtml

25

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 42: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

Nova ComputeManages the life cycle of computing instances in an OpenStack environment Features includespawning scheduling and decommissioning of machines on demand Nova support a wide varietyof computing technologies including libvirt variant Hyper V VMware and docker using a thirdparty plug in

Cinder Block StorageProvides persistent disk (block) storage for running instances Its pluggable driver architecturefacilitates the creation and management of block storage devices

Neutron NetworkingProvides various networking services for cloud instances such as IP address management DNSDHCP load balancing and firewall rules Neutron offers a fine-grained network configurationenabling users to manage guest network traffic isolation and network availability It has a pluggablearchitecture that supports many popular networking vendors and technologies

Glance Image ServiceProvides a catalog of predefined virtual machine Glance enables the administrator to add deleteand manage accessibility of virtual machine Users can directly use glancersquos proposed instance tostart VM

Swift Object StorageProvides a RESTful HTTP based API to store and retrieve unstructured data object Data objectis best used for storing static data such as media files virtual machine images and backup Swiftis highly fault tolerant with its data replication and scale out architecture

Keystone IdentityProvides authentication and authorization services for other OpenStack services Keystone is theentry point of the whole cloud It provides a catalog of endpoints for all OpenStack servicesKeystone also enables identification through federation

Horizon DashboardProvides a web-base interface for both cloud administrators and users Horizon enables the userto easily interface with all the OpenStack services

For a more detailed description of the above modules the reader is invited to visit the followingwebsites

1 httpsenwikipediaorgwikiOpenStack

2 httpsdocsopenstackorg

3 httpvmartinezdelacruzcomin-a-nutshell-how-openstack-works

Every component of OpenStack offers a documented REST API1 Furthermore every OpenStackcomponent is open source and the source can be consulted on the OpenStack GitHub2 OpenStack

1 httpsdeveloperopenstackorgapi-guidequick-start2 httpsgithubcomopenstack

26

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 43: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 4 OpenStack Components and Federation Architecture

services have very limited dependency upon each other the main dependency is by keystonemanaging the authentication Many services can be added or replaced with minimal impactOpenStack offers a software development kit3 to aid development of a new module

The interaction between the components is illustrated in the figure 42

Figure 42 OpenStack component interaction Source httpkenpeppleinfoopenstack

20120925openstack-folsom-architecture

Beside these core components other popular components can easily be added to OpenStack Open-Stack offers a list of optional components4 with varying levels of maturity The orchestration servicecalled Heat is one of the most interesting components in that list Heat allows the deployment ofa pre-defined infrastructure using templates

Each of the web services of OpenStack can be enableddisabled individually For example wecan deactivate the Nova service5 Because there are individual web services it is quite possibleto replace any of the OpenStack services by another custom made web service as long as theyobey the same API defined for that service Components interact with each other using theHTTP protocol following a standard API If the custom-made component follows the same APIit should be compatible with the other OpenStack services For example Nova is compatiblewith the OCCI standard interface and the reference of the API is defined in httpsdeveloper

3 httpsdeveloperopenstackorgsdkspythonopenstacksdkusersindexhtml4 httpswwwopenstackorgsoftwareproject-navigator5 httpsdocsopenstackorgadmin-guidecli-nova-manage-serviceshtml

27

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 44: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

openstackorgapi-refcompute Similarly for Neutron an architecture guide proposes to usethe old Nova networking capability The API is defined in httpsdeveloperopenstackorg

api-refnetworkingv2indexhtml

Another aspect of network management is resistance to network outages In this case there is nomiracle The cloud environment will provide tools to resync the system after an outage but theresponsibility for how the system or application will react is up to the developer choosing to useor not the tools provided by the cloud environment6

Besides the API constraint each web service could be modified to adopt additional constraintsNetwork control is much more sophisticated in clouds than organizationbusiness networks Forexample with Neutron you can still create proxies firewalls and other IP tables rules On topof that cloud networking allows you to completely isolate applications or create custom-madesub-nets on a need to know basis OpenStack is project-based (before that projects were calledtenants) A project can have a limitation Projects have their own resources of computing andnetworking and are isolated from other network projects by default Each project manages itsown network The network can be exposed or not The TriCircle project aims to manage Neutronacross multi-region cloud

Even if the components are independent and self containing OpenStack evolves very quickly andnew features may break legacy or outdated third party components

42 OpenStack Federation

A federated identity network represents the establishment of a trust relationship between cloudsIt enables users to securely access resources across multiple endpoints (in different clouds) by usingtheir credential only once Federated projects also centralize the identity management and allowthe usage of multiple clouds simultaneously This section defines the federation components andillustrates the architecture of a federation

421 Definition

The main components of an identity federation are the service provider and the identity providerThe role of the service provider is to provide cloud services to users The role of the identityprovider is to manage the authentication process for all the other clouds The service providermust be configured to trust the identity provider The configuration is often made through anexchange of metadata between them Thus the service provider doesnrsquot manage or store any usercredential The Identity Provider (IdP) stores and manages the credential of every cloud userKeystone can act as both service provider and identity provider for OpenStack However it needsan external software to act as a service provider and to use the federation protocol

The federated identity flow is defined by a federation protocol There are three main federation

6 httpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

what-s-going-on-behind-the-scenes-in-the-clusterhttpsdocsopenstackorgdeveloperswiftoverview_container_synchtml

28

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 45: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 4 OpenStack Components and Federation Architecture

protocols Oauth SAML2 and Openid Connect SAML2 is recommended in OpenStack despitethe complexity of the installation of its components

422 Architecture

In the case of a two-cloud federation a Keystone to Keystone (k2k) federation can be establishedOne keystone becomes the identity provider and the other the service provider The access canthen be symmetrical users of both clouds can authenticate and use the resources of the other cloudThe federation authentication is also called Single Sign On (SSO) and it is briefly illustrated inthe figure 43

Figure 43 Single sign-on using SAML in a Web browser

43 Federation architecture and mechanisms

There are multiple configuration possibilities to connect multiple clouds together The standardsdescribed in section 2 are mostly defined for IaaS clouds This section describes the mechanisms

29

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 46: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

of discovery management and monitoring when more than one cloud environment is available tofulfill a request

431 Discovery

In a single cloud environment keystone plays the role of identity management and cataloguingIn a multicloud environment multiple keystones can be connected together to form a federatedidentity network Once the federated identity is established both keystonesrsquo catalogs are availableto users The catalogs are the points of access to all the services offered by the cloud

432 Management and monitoring

The Horizon dashboard exploits the API of each module to offer a convenient web interface tointeract with all the OpenStack components The web interface is simple but powerful It providesa usage dashboard to help monitor the available resources OpenStack also offers a command-lineutility that interacts with components through the API Horizon can help manage multiple cloudssimultaneously

30

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 47: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 5

Installation Guide

This section describes the installation of a cloud identity federation OpenStack is a collection ofservices that can be deployed on one or more computers It can be used for several different usecases which make it very flexible However this flexibility makes it complex and hard to configureConsidering this it is not feasible to develop a quick installation guide

This guide provides information insight and trouble shooting regarding the deployment made inour experiment We have deployed the cloud federation several times on virtual hardware andonly two actual computers In order to be able to deploy multi-node clouds this guide will referto resources about multi-node clouds when needed

Warning The following instructions are meant to be executed by someone with deep understand-ing of large system installation and network administration and configuration These instructionsmay vary depending on the context such as special firewall or proxy settings We suggest thesystem administrator to revisit previous RISOMIA reports where instructions were given for thatcontext (MSARI administration documents and [1]) Also we strongly suggest not to deviatefrom the following instructions as a lot of lessons learned and debugging were included in theseinstructions The OpenStack version used in this section is the latest one at the time of writingthe ocata release If you use another version past or future some of the provided instructions maynot apply

A list of all the web resources used for debugging and producing this set of consistent instructionscan be found in Annex A

This chapter is divided in three sections

bull Section 51 Centos installation describes the installation steps of centos Operating systemfor OpenStack cloud deployment

bull Section 52 OpenStack installation explains how to install an OpenStack cloud

bull Section 53 Federation configuration details how to establish a cloud federation

31

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 48: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

51 Centos installation

This section enumerates steps to install Centos Centos is a free-Software operating system derivedfrom RedHat Enterprise Linux (RHEL) sources Itrsquos free of charge and itrsquos fully compatible withRHEL In the case of a multi-node clouds you should follow these instructions for every computer

511 Installation

1 Download centos minimal rdquohttpmirroresecuredatacomcentos7isosx86 64CentOS-7-x86 64-Minimal-1611isordquo and burn the ISO image on a CD Alternative mirrors can befound at httpswwwcentosorgdownloadmirrors

2 Proceed with the installation on a server (with Internet access) with sufficient disk space(500GB or higher) and memory (16GB or higher)

3 Define the network so we can enable NTP in the next step In the Network and hostnamesection define the parameters for your network In our case we used a manual (static IPaddress) configuration

4 Set date and time keyboard and language settings using the GUI interface

5 In the Installation source section use the automatic settings for installation from the mediaor choose another installation option (ie network) If access to Internet is required youmay have to provide proxy information (for DRDC httpwebproxydrdc-rddcgcca8080)See instructions in Call-up 13 3311

6 In the Software selection section select Minimal install

7 In the Installation destination section selectconfigure the disk partition that will containthe OS with the appropriate mount point If you use a disk partition which already has Grubinstalled uncheck the installation disk in the rdquoboot loaderrdquo section (bottom left)

8 When all the sections are configured click on Begin Installation

9 In the User settings (next page) create the root password and another user administrator(recommended) In the advanced section we set the oodaadmin home in the

10 Click on the Reboot button when the installation process is complete Remove the CD

11 For those who are using an already installed Grub you must run sudo update-grub2 on theOS that contains the first grub installation in order to detect the new Centos in the Grubboot section

12 Connect in the administrative account (root or oodaadmin)

13 Adjust the network parameter files if necessary (mostly etchosts to add other machinesetcsysconfignetwork-scriptsifcfg-XXXXX etcresolvconf etc) in order to reflect yournetwork configuration Reboot to activate any modification

14 Run

32

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 49: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 5 Installation Guide

yum -y update

15 Run

yum install -y net-tools wget bind-utils iptables-services bridges-utils

bash-completion policycoreutils-python

16 Configure your SSH

(a) Add the following line to the default etcsshsshd configmacr

PermitRootLogin yes

Note A similar configuration to callup 13 has been tried but was too restrictive andyielded problem at OpenStack installation

(b) Generate keys

ssh-keygen -t rsa -b 4096

Note You can also copy the full ssh directory from another machine

(c) Copy public keys from the two computers in the rdquoauthorized keysrdquo file

(d) Include also a local copy of the id rsapub in authorized keys to allow copy betweenadmin accounts

(e) Adjust the permission of ssh (700) and sshauthorized keys (600) is ok

semanage fcontext -at user_home_dir_t oodaadmin

semanage fcontext -at ssh_home_t oodaadminssh

semanage fcontext -at ssh_home_t oodaadminsshauthorized_keys

restorecon -Rv oodaadmin

17 Test the SSH connection

52 OpenStack installation

Since OpenStack is a free software licensed under the Apache license many big companies havedeveloped a ldquoflavorrdquo of OpenStack often customizing the dashboard interface Every single servicecan be installed manually but to facilitate and accelerate the installation a deployment solutionis preferable For this project we sought an open source and free openstack deployment solutionWe evaluated three solutions devstack packstack and openstack-ansible

The OpenStack development community has developed a tool called devstack to easily deployopenstack for development Devstack installs a selected component directly from github sourcecode The Devstack community advertises that this deployment solution is not stable and shouldnot be used for production

Packstack is a Redhat based community way ( RDOproject1 ) aiming to develop an easy andstable OpenStack development solution The developed tool uses a puppet2 tool to configure anddeploy every OpenStack component The project uses mature and stable OpenStack source andadvertises a production-ready deployment solution

1 httpswwwrdoprojectorg2 httpspuppetcomproductcapabilities

33

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 50: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

OpenStack Ansible is deployment solution similar to Packstack Ansible is an orchestration de-ployment and configuration manager

Packstack was used in this project because of its stability and simplicity and because itrsquos specificallydesigned for Red Hat-based operating systems

521 Architecture

The OpenStack services can be deployed in many different ways to fulfill every requirement andorrestriction of a project There are many architecture guidelines that can help design the deploy-ment3

For a simple deployment we suggest using one controller node and few compute nodes as illustratedin the figure 51

Figure 51 Cloud federation proposed architecture

522 Installation

The installation is divided in two sections the configuration to be made on all computers and theconfiguration of the controller node of each cloud We propose to install and configure each cloudseparately then configure the federation described in the section 53

5221 Every computer node

1 Configure network interface

3 httpsdocsopenstackorgarch-designhttpsaccessredhatcomdocumentationen-usred hat openstack platform10htmlarchitecture guidehttpsdocsoraclecomcdE36784 01htmlE54155archoverhtmlhttpswwwibmcomsupportknowledgecenterSST55W 430liacaliaca deploy overviewhtml

34

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 51: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 5 Installation Guide

OpenStack Networking does not work on systems that have the Network Manager serviceenabled Each server must disable NetworkManager and enable the standard network service

Disable NetworkManager

systemctl stop NetworkManagerservice

systemctl disable NetworkManagerservice

To enable the standard network service the interface network script must be modified toadd the following lines

etcsysconfignetwork-scriptsifcfg-enp3s0

NM_CONTROLLED=no

ONBOOT=yes

Finally the network service must be started and enabled at boot time

systemctl start networkservice

systemctl enable networkservice

2 Configure firewall

The last version of CentOS comes with firewalld instead of iptables We must replace firewalldwith iptables

Install iptables

yum install iptables-services

Disable firewalld and enable iptables

systemctl disable firewalldservice

systemctl stop firewalldservice

systemctl start iptablesservice

systemctl enable iptablesservice

systemctl start ip6tablesservice

systemctl enable ip6tablesservice

3 Disable SElinux

setenforce 0

getenforce

should output Permissive

4 Modify sshd Add the following line to etcsshsshd config

PermitRootLogin yes

5 Set hostname

The hostname should be different from the compute node ie controllercloud1drdc-rddclan compute01cloud1drdc-rddclan compute02cloud1drdc-rddclan Thehostname should be self-described and will be used on other steps of the installation Fullyqualified name is recommended

hostnamectl set-hostname controllercloud1drdc-rddclan

35

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 52: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

6 Install network time service

yum install chrony -y

systemctl enable chronyd

systemctl start chronyd

systemctl status chronyd

5222 On the controller node

In case of a multi-node deployment the controller node should be able to access every computenode using ssh without password

1 Install packstack

yum install openstack-packstack -y

2 Generate answer file

packstack --gen-answer-file=answerfiletxt

note This command will create an answer file to be used by the current user The answergenerated file contains information about the current user ie ssh keypair and default host

3 Generate ssl sertificate for every service

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivateselfkeykey

-out etcpkitlscertsselfcertcrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_vnckey

-out etcpkitlscertsssl_vnccrt -days 365 -nodes

openssl req -x509 -sha256 -newkey rsa2048 -keyout etcpkitlsprivatessl_dashboardkey

-out etcpkitlscertsssl_dashboardcrt -days 365 -nodes

chmod 600 etcpkitlsprivate

chmod go-rX etcpkitlsprivate

4 Modify the answer file

Open the answer file with your preferred editor and modify the line described above Fora multi-node deployment the ltCONTROLLER NODE IP gt and ltCOMPUTE NODEgt should be replaced with the cloud 1 controller hostname and compute node hostnamerespectively set at the section 51 In the case of a single node deployment the controllernode will also act as a compute node The ltRegionX gt should change according to thecloud ie RegionOne RegionTwo

CONFIG_SSH_KEY=rootsshid_rsapub

CONFIG_HORIZON_SSL=y

CONFIG_CONTROLLER_HOST=ltCONTROLLER NODE IPgt

36

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 53: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 5 Installation Guide

List the servers on which to install the Compute service (the control node may be in the list)

CONFIG_COMPUTE_HOSTS=ltCOMPUTE NODE 1gtltCOMPUTE NODE 2gt

IP address of the server on which to install the AMQP service (Usually the controller node)

CONFIG_AMQP_HOST=ltCONTROLLER NODE IPgt

Default region name to use when creating tenants in the Identity

Service

CONFIG_KEYSTONE_REGION=lt RegionX gt

Identity service API version string [v20 v3]

CONFIG_KEYSTONE_API_VERSION=v3

CONFIG_SSL_CACERT_FILE=etcpkitlscertsselfcertcrt

CONFIG_SSL_CACERT_KEY_FILE=etcpkitlsprivateselfkeykey

CONFIG_VNC_SSL_CERT=etcpkitlscertsssl_vnccrt

CONFIG_VNC_SSL_KEY=etcpkitlsprivatessl_vnckey

CONFIG_HORIZON_SSL_CERT=etcpkitlscertsssl_dashboardcrt

CONFIG_HORIZON_SSL_KEY=etcpkitlsprivatessl_dashboardkey

CONFIG_HORIZON_SSL_CACERT=etcpkitlscertsselfcertcrt

5 Install OpenStack

The installation downloads and installs all components In case of a multi-node environmentthe controller node will install the required software to every compute node The installationis a long process

packstack --answer-file=answerfiletxt --timeout=1000

6 Validate

At the end of the installation the script should output a file called keystonerc admin in thehome directory of the current user This file contains the login variables to be able to manageOpenStack

You can access the url httpltCONTROL_NODE_IPgtdashboard to validate the OpenStackinstallation The credential information should be in the keystonerc admin file

The OpenStack user guide4 offers pertinent information about how to use the dashboard andalso the OpenStack command line client The section OpenStack command-line interfacecheat sheet shows a broad list of examples of the OpenStack command line client

53 Federation configuration

An identity federation allows to rely on a trusted third partyrsquos authentication system in order touse a service A federation generally involves an identity provider and a service provider In a

4 httpsdocsopenstackorguser-guideUserGuidepdf

37

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 54: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

Keystone federation two keystones are involved (Identity Service for OpenStack) One keystonebecomes the Service provider while the other is the Identity Provider As describe in the figure51 the cloud 1 controller will act as the Identity Provider (IdP) and the cloud 2 will act as theService Provider (SP)

This documentation seeks to help you deploy a keystone to keystone federation using OpenStackOcata version (latest at this time)

531 Architecture

At this point the cloud 1 controller should be able to ping the cloud 2 controller by hostname andvice versa This means that either a Domain Name System (DNS) server resolves the hostnameor the etchosts file has been populated with the hostnameip of each cloud controller

532 Configuration Identity Provider (IdP)

The controllercloud1drdc-rddclan host is the controller of the cloud 1

Figure 52 Identity Provider Configuration

1 Install dependency

yum install xmlsec1 xmlsec1-openssl python-pysaml2 -y

2 Generate keystone saml signing key

mkdir etckeystonessl

mkdir etckeystonesslcerts etckeystonesslprivate

openssl req -x509 -newkey rsa2048 -keyout etckeystonesslprivatesigning_keypem

-out etckeystonesslcertssigning_certpem -days 9999 -nodes

3 Edit etckeystonekeystoneconf

38

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 55: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 5 Installation Guide

[saml]

certfile=etckeystonesslcertssigning_certpem

keyfile=etckeystonesslprivatesigning_keypem

idp_entity_id=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp

idp_sso_endpoint=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2sso

idp_metadata_path=etckeystonesaml2_idp_metadataxml

[auth]

methods = passwordtokenoauth1mapped

4 Generate IdP metadata

After populating keystoneconf you can run the following command to generate the IdPmetadata wich is based on the config file If you modify keystoneconf you must regeneratethe IdP metadata

keystone-manage saml_idp_metadata gt etckeystonesaml2_idp_metadataxml

5 Restart httpd

Keystone is a web service running under httpd restarting httpd restart keystone

sudo service httpd restart

At this point if you follow the following url you should be able to receive the IdP metadata

curl httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2metadata

6 Add the Service Provider to OpenStack

Every OpenStack service offers a HTTP API to communicate with it OpenStack communityhas developed a Python base client to interact with OpenStack services You can providecredentials to the python client by populating the environment variable using the followingcommand

cd ~

source keystone_admin

note The file keystone admin is generated at the end of the section 5222 and should be inthe home user directory

Once the file has been sourced use the OpenStack python client to create a Service providerreference

openstack service provider create --auth-url

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONidentity_providersmyidpprotocolsmappedauth

--service-provider-url httpscontrollercloud2drdc-rddclanShibbolethssoSAML2ECP mysp

note myidp and mysp is the ID of instance The ID should be consistent between The IdPand the Service Provider (SP)

39

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 56: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

533 Configuration Service Provider (SP)

The controllercloud2drdc-rddclan host is the controller of the cloud 2

Figure 53 Service Provider Configuration

1 Install Shibboleth

curl httpdownloadopensuseorgrepositoriessecurityshibbolethCentOS_7securityshibbolethrepo -o

etcyumreposdshibbolethrepo

yum update -y

yum install shibboleth -y

2 Generate Shibooleth keypair

cd etcshibboleth

sudo etcshibbolethkeygensh -f -u shibd -h centos2centoslan -y 3

-e httpscentos2centoslanshibboleth -o etcshibboleth

3 Enable shibboleth service

systemctl enable shibd

systemctl restart shibd

4 Configure httpd virtual host

Add the following lines at the end of the file etchttpdconfdshibconf

ltLocation Shibbolethssogt

AuthType None

Require all granted

SetHandler shib

ltLocationgt

ltLocationMatch v3OS-FEDERATIONidentity_providersprotocolsmappedauthgt

ShibRequestSetting requireSession 1

AuthType shibboleth

40

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 57: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 5 Installation Guide

ShibExportAssertion Off

Require valid-user

ltLocationMatchgt

5 Configure Shibboleth

The following files are all located in etcshibboleth

(a) attribute-mapxml

Add the following lines at the end of the file etcshibbolethattribute-mapxml

ltAttribute name=openstack_user id=openstack_usergt

ltAttribute name=openstack_roles id=openstack_rolesgt

ltAttribute name=openstack_project id=openstack_projectgt

ltAttribute name=openstack_user_domain id=openstack_user_domaingt

ltAttribute name=openstack_project_domain id=openstack_project_domaingt

(b) shibboleth2xml

ltSSO entityID=httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idpgt

SAML2 SAML1

ltSSOgt

ltMetadataProvider type=XML validate=true

file=etcshibbolethcontrollercloud1drdc-rddclan-metadataxmlgt

ltMetadataProvidergt

(c) Testing Shibboleth metadata

The following url should output the metadata corresponding with the information en-tered

curl httpscontrollercloud2drdc-rddclanShibbolethssoMetadata

6 Add the Identity Provider to openstack

You should use the command source keystone admin before executing the openstack com-mand

openstack identity provider create --remote-id

httpcontrollercloud1drdc-rddclan5000v3OS-FEDERATIONsaml2idp myidp

7 Create the domain project and group and assign roles

openstack --insecure domain create federated_domain

openstack --insecure project create federated_project --domain federated_domain

openstack --insecure group create federated_users

openstack --insecure role add --group federated_users --domain federated_domain _member_

openstack --insecure role add --group federated_users --project federated_project _member_

8 Create mapping

Create a json file containing the mapping to be loaded The

cat gt rulesjson ltltEOF

[

local [

41

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 58: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

user

name 0

group

domain

name Default

name federated_users

]

remote [

type openstack_user

]

]

EOF

Load the mapping to openstack

openstack --insecure mapping create --rules rulesjson myidp_mapping

openstack --insecure federation protocol create saml2 --mapping myidp_mapping --identity-provider myidp

9 Restart Keystone

sudo service httpd restart

The table 51 offer a reference of modification done on each cloud controller

Node Name controllercloud1drdc-rddclan controllercloud2drdc-rddclan

Role Identity Provider Service Provider

Installationxmlsec

xmlsec1-openssl

python-pysaml2

shibboleth

Files to createetckeystonesslprivatesigning keypem

etckeystonesslcertssigning certpem

etckeystonesaml2 idp metadataxml

etcshibbolethsp-keypem

etcshibbolethsp-certpem

rulesjson

Files to modify etckeystonekeystoneconfetchttpdconfdshibconf

etcshibbolethattribute-mapxml

etcshibbolethshibboleth2xml

Table 51 Configuration modification for each cloud controller

42

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 59: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 6

Applications of CommunicationBetween Federated OpenStack Cloud

This chapter describes the particular feature of OpenStack that can be exploited in a federationand present different examples of intercloud interaction at the level of IaaS PaaS and SaaS

61 Federated OpenStack Features

The OCCI API interface offers a standard way to interact with heterogeneous cloud When usinga homogeneous cloud federation both clouds can communicate with the standard defining theirinteraction In the case of OpenStack every component provides a web interface which eases theinteraction interclouds Federated OpenStack can augment the reliability and performance of asingle cloud

Some OpenStack components extend gracefully in a federated cloud network The keystone servicecan be configured to trust other cloud Identity services enabling federated users to access the cloudresources The Swiftrsquos storage RESTful HTTP API can be used easily in different clouds to storeor replicate data across the network The Novarsquos compute live migration features can be extendedto federated projects and offer a greater load balancing enhancing the performance for the wholecloud

62 IaaS application examples

Infrastructure as a service offers infrastructure to run and manage operating systems storage andnetworking The IaaS may also be the level that offers the most flexibility

43

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 60: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

621 Workload transfer

One of the main uses of IaaS intercommunication is workload transfers When several cloudsare connected together the pool of computing resources get augmented If a cloud resource usagepasses a defined threshold virtual machine can be migrated to a trusted cloud without interruptionThis feature is a clear example of performance and reliability enabling a constant workload on thecloud It also optimizes resource exploitation by using unused computing power

The name cloud bursting is used when a cloud migrates some of its workload to another cloud Itis used mostly in a hybrid cloud1 context where the originator is a private cloud and the recipientis a public cloud This process is designed to minimize the cost related to the constant usage oflarge public cloud service providers (eg Amazon AWS and EC2)

In the case of a homogeneous multicloud environment such as a federation of OpenStack the termcloud bursting is less used but the same process can be applied2

Currently OpenStack did not yet implement an automatic scaling capability for doing cloud burst-ing within a OpenStack federation but it is under study

63 PaaS application examples

Platform as a Service offers the whole infrastructure to run an application It abstracts theinfrastructure and the operating system but provide a runtime environment such as Java PythonPHP etc PaaS is often targeted to software developers enabling them to concentrate their workon coding instead of installing infrastructure However some cloud interoperability still can beconsidered

631 Database synchronization

PaaS providers often offer database instance The databases offered are well known stable andmature databases such as mysql Postgresql Mongodb etc These databases offer some replicationand synchronization services that could be used across multiple cloud

632 Application provisioning

Applications running on PaaS rarely own a storage block Applications can be easily replicatedfor load balancing a critical workload or even migrated to another cloud as instances in section621 Application images can also be copied to other cloud services

1 httpwwwservice-architecturecomarticlescloud-computingcloud_burstinghtml2 httpswwwopenstackorgsummitopenstack-summit-atlanta-2014session-videospresentation

extending-openstack-to-allow-federation-between-clouds

44

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 61: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 6 Applications of Communication Between Federated OpenStack Cloud

64 SaaS application examples

Software as a service is the simplest architecture for intercloud communication In a SaaS platformonly application endpoint is exposed The applications presented are usually web base servicesWhen two clouds are connected through a federation users can access available services

641 Exchange of services

At the SaaS level services can be exploited from different clouds If a vessel owns a uniquedetection hardware an interface (ie REST) could be implemented to use it The access to thecloud services could be controlled by the cloud federation authentication

45

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 62: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

46

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 63: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 7

At-sea Environment Limitation

This section describes the at-sea environment in terms of interoperability as well as informationstorage discovery distribution and access that may impact an at-sea cloud implementation

The section 41 of RISOMIA call-up 11 report summarized the conclusions of the analysis per-formed regarding the current state-of-the-art for at-sea data exchange environment as follows

1 There are a number of separate networks that support the exchange of information fordifferent purposes and between different geopolitical subgroups

2 There is information exchange at different levels of security

3 There are satellites as well as other radio communication systems that support the connec-tivity needs of at-sea vessels using various frequency channels

4 There are US Military NATO standards that enforce how often and what type of informationis exchanged

Each one of these observations imposes limitations in terms of interoperability and informationdistribution and access Observations 1 and 2 may impose restrictions for access of information bydifferent members of a task group Hence independently of any other technological limitationsthere will likely be a need for a multitude of cloud categories and for a multitude of access andprocessing purposes Observations 3 and 4 impact interoperability and data distribution whichmay become less limiting with technological enhancements over time However in cloud imple-mentations it is natural to assume that there will be less advanced task group members Theimplementation designs should be able to accommodate various levels of technological maturityfor interoperability and data distribution (standards for data exchange)

Section 42 of RISOMIA call-up 11 report provided an analysis of state-of-the-art informationstorage discovery distribution and access for on-board information sources It sought to sharethe information between vessels and to establish the task force situational awareness throughcombination of on-board and off-board data In the current state-of-the-art

47

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 64: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

1 The information discovery services for on-board sensors are part of each platform Commandand Control (C2) except for some US vessels that include the Cooperative EngagementCapability (CEC) which is a real-time sensor netting system that enables sharing and inte-grating the sensor raw reports of all ships enabled with CEC together

2 Each vessel also includes services for combination and discovery of on-board and remotevessels

3 Vessels may also have access to some capabilities and information products from the GlobalCommand and Control System Maritime (GCCS-M) which includes its own informationdiscovery capabilities supporting all of US maritime situational awareness

Figure 71 DoD Enterprise Cloud Environment Source [15]

Figure 71 presents the US vision of Enterprise Cloud Environment where there is one centralcloud-based C2 It contains all the necessary services that provide information storage discoverydistribution and access capabilities It also collects information from participating vessels and avariety of service providers via a secure communication link and it provides the required situationalawareness to all the vessels and other decision centres Ultimately with technological advancementssuch a vision may be achieved within one nation in the future Yet it is hard to envisage thisvision becoming a reality even within one nation very soon This vision as presented also does notprovide the necessary services to support multinational task force operations which would need anumber of separate networks that support the exchange of information for different purposes andbetween different geopolitical subgroups (data exchange observation 1 above)

While a fully cloud-based multinational operation can be envisaged into far enough future eachone of the at-sea data exchange environment observations impose constraints on the architecture

48

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 65: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 7 At-sea Environment Limitation

design decisions of cloud implementations of the information storage and of the discovery servicesFor example

1 It is a recognized reality that Navies of different nations and even various vessel types withinthe same nation do not evolve to the same technological maturity at the same time Thereforethe cloud architecture of any vessel will need to be designed to be able to accommodateinformation sharing with both cloud enabled vessels and cloud non-enabled vessels It willnecessitate the use of standards (observations 3 and 4) of various levels of technologicalmaturity and to ensure maximal feasible situational awareness in every participating vessel

2 Even in the case of most advanced connectivity and cloud capability environment (assumingall vessels having same technological maturity) as a result of limitations 1 and 2 there willbe restrictions for sharing of information of certain types with certain vessels and restrictionson certain vessels from using some external cloud-based information discovery and storageservices Consequently the architecture for cloud implementation in each vessel (based oncountry roles missions etc) will need to be designed to distribute information discoveryand storage services between their own cloud and external clouds (on other vessels and globalC2) to maximize situational awareness for each vessel

The discussions above mention technological maturity for supporting cloud enabled naval vesselsincluding bandwidth communication protocol physical space computer power decoupled instal-lation (ability to install as many clouds as necessary) security (encryption) Section 424 of the ofRISOMIA call-up 11 report presents the details of the US defence messaging and communicationtechnologies state-of-the-art of 2008 It also refers to the US efforts to evolve this to support thecurrent connectivity needs1 It is realistic to expect that the technological advancements in sup-porting the connectivity needs will occur at a quicker pace than the efforts to upgrade the navalfleets of even most advanced nations

In the near term it would be valuable for the RampD efforts to focus on analyzing and evaluatingthe cloud implementations in terms of

bull Trade-off between having dedicated clouds hosting specific information discovery servicesthat receive information of a specific type versus putting all information discovery servicesin each cloud with each cloud processing information only pertinent to that vessel and thensharing the results

In the terminology of data fusion having a central level cloud hosting the C2 which collectsand processes all information and provides to each vessel whatever information they require forsituational awareness is optimal in terms of information processing performance It is howevertoo costly in terms of data transfer and connectivity A distributed C2 sharing post-processedinformation products in each cloud could lead to a somewhat lower quality information processingperformance but it has much lower data transfer costs Experimentation with such alternativearchitectures will be valuable to help design the future naval vessel cloud architectures

1 US Navy C2oix the Navyrsquos Newest Messaging System ndash Cost Efficient and Simpler November 2013 http

wwwnavymilsubmitdisplayaspstory_id=74740

49

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

OODA Technologies Inc

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 66: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

This page is intentionally left blank

50

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 67: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Part 8

Conclusion and Future Work

In this study mechanisms and standards were found that allowed intercloud communication Afederation of two OpenStack implementations provided a concrete example of these communicationfunctionalities Intercloud interactions can be realized at different levels IaaS PaaS and SaasUsage and applicability will differ in each of these levels

If we answer the questions listed in the introduction

bull What is the state of these cloud framework environments Answer Evolving rapidlyUpdated documentation sparse and erratic Very little in-depth blogs or tutorials for helpingthe novice

bull Have they achieved a certain level of maturity Are they stable Answer Not yet

bull Are they easy to use Answer Yes and no Installation is still excruciatingly difficultOnce in place feature usage can be automatic or easy manageable through the OpenStackDashboard or other means depending on the context

bull What can we do with it And cannot do Answer The number of functionalities isincreasing rapidly as new releases are coming out Restrictions may apply depending on thelevel (IaaS PaaS SaaS) security and cloud standard protocols

bull Are they secured Answer Yes and it is partially part of the problem Many things cango wrong during the installation and the level of security you want to achieve

Some challenges were encountered during this project notably

bull Very steep learning curve It is no secret that learning about cloud technology requiresa lot of effort It is related to many aspects of system administration security optimizationfrom hardware farm management container technology and web services

bull Absence of a stable version Because no stable release existed we selected the latestversion of OpenStack (named ocata) in order to minimize the number of deprecated featuresand maximize the number of new features

51

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 68: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

bull Lack of updated documentation On many occasions we found different user guideswith contradictory instructions mainly because they describe OpenStack at different stagesof its development It took time to make sense of the information and to sort them out

Based on the lessons learned during the development cycle possible improvements are suggestedfor the next iteration of a intercloud application

bull Improve and update the findings of this study As development continues on Open-Stack new features will be implemented in future release and more examples of intercloudcommunication will be available An update of the findings of this study may be useful in 6months or more

bull Navy example An example based on real-life restrictions of what would be a private cloudon a vessel could be interesting to investigate in order to identify the strengths and theweaknesses of such of construct and how it would apply in cloud communication betweenships

To summarize although there were many hurdles and challenges in this investigation this studysuccessfully demonstrated the existence and the application of intercloud communication and itspotential for maritime applications

52

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 69: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Bibliography

[1] Yannick Allard Michel Mayrand Dan Radulescu and Mathieu Lavallee Command andControl Application Development VOIR Server Design Details and Installation guideTechnical Report Contract W7707-145677001HAL Call-up 13 final report DefenceResearch and Development - Atlantic 2016

[2] MRM Assis and LF Bittencourt A survey on cloud federation architectures Identifyingfunctional and non-functional properties Journal of Network and Computer Applications7251ndash71 2016

[3] J Frahim V Josyula M Morrow and K Owens Intercloud Solving Interoperability andCommunication in a Cloud of Clouds 2016 ISBN 9780134189239URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[4] Jazib Frahim Venkata Josyula Monique J Morrow and Kenneth Owens IntercloudSolving interoperability and communication in a cloud of clouds Technical report Cisco2016URL http

ptgmediapearsoncmgcomimages9781587144455samplepages9781587144455pdf

[5] Craig A Lee A demonstration of federation management using the keyvoms prototype2016URL httpgsaworgwp-contentuploads2016032016s11d_leepdf

[6] Geant Network Services People Federated access to openstack 2016URL httpswikigeantorgdownloadattachments531173732016012220-

20Federated20access20to20Openstackpdf

[7] F Pop and M Potop-Butucaru Adaptive Resource Management and Scheduling for CloudComputing Second International Workshop ARMS-CC 2015 Held in Conjunction withACM Symposium on Principles of Distributed Computing PODC 2015 Donostia-SanSebastian Spain July 20 2015 Revised Selected Papers Lecture Notes in ComputerScience Springer International Publishing 2016 ISBN 9783319284484URL httpsbooksgooglecabooksid=ERxaCwAAQBAJ

[8] K Jackson C Bunch and E Sigler OpenStack Cloud Computing Cookbook - Third EditionQuick answers to common problems Packt Publishing 2015 ISBN 9781782174783URL httpsbooksgooglecabooksid=850hswEACAAJ

53

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 70: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

[9] Fabio Alessandro Locati OpenStack cloud security Packt Publ Birmingham 2015URL httpcdscernchrecord2050420

[10] Steve Martinelli Henry Nash and Brad Topol Identity Authentication and AccessManagement in OpenStack Implementing and Deploying Keystone OrsquoReilly Media Inc1st edition 2015 ISBN 1491941200 9781491941201

[11] D Radez Openstack Essentials Community experience distilled Packt Publishing 2015ISBN 9781783987085URL httpebookkonfigurasinetOpenstackOpenStack20Essentialspdf

[12] David W Chadwick Kristy W S Siu Craig Lee Yann Fouillat and Damien GermonvilleAdding federated identity management to openstack J Grid Comput 12(1)3ndash27 2014doi101007s10723-013-9283-2URL httpdxdoiorg101007s10723-013-9283-2

[13] Tom Fifield Diane Fleming Anne Gentle Lorin Hochstein Jonathan Proulx EverettToews and Joe Topjian OpenStack Operations Guide OrsquoReilly Media Inc 1st edition2014 ISBN 1491946954 9781491946954URL httpwwwstilsonnetdocumentationOpenStack20Operations20Guidepdf

[14] Luca Tartarini and Marek Denis Federation of openstack clouds 2014doi105281zenodo11982URL httpszenodoorgrecord11982filesCERN_openlab_Luca_Tartarinipdf

[15] DoD Cloud computing strategy 2012URL httpdodciodefensegovPortals0DocumentsCloudDoD20Cloud

20Computing20Strategy20Final20with20Memo20-20July205202012pdf

54

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 71: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Appendix A

Installation Internet references

We present here a list of the web links that were helpful for the installation of OpenStack and theconfiguration of a federation

A1 OpenStack

bull httpswwwrdoprojectorginstallquickstart

bull httpsdocsopenstackorg

bull See also section A5

A2 OpenStack Federation

bull httpsdocsopenstackorgdeveloperkeystonefederationfederated_identityhtml

bull httpsdocsopenstackorgocataconfig-referenceidentityfederated-identityhtml

bull httpsdocsopenstackorgdeveloperkeystonemitakaconfigure_federationhtml

bull httpsdocsopenstackorgdeveloperkeystonefederationconfigure_federationhtml

bull httpsdeveloperopenstackorgapi-refidentityv3-extindexhtml

bull httpsdocsopenstackorgdeveloperopenstack-ansiblemitakainstall-guideconfigure-federation-use-case

html

bull httpsdocsopenstackorgdeveloperkeystonefederationwebssohtml

bull httpsdocsopenstackorgdeveloperkeystonedevrefapi_curl_exampleshtml

bull httpsplatform9comblogopenstack-keystone-federation

bull httpsplatform9comsupportusing-openstack-cli-saml-authentication

bull httpsgithubcomopenstackkeystone-specsblobmasteratticv3identity-api-v3-os-federation-ext

rst

bull See also section A5

A-1

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs
Page 72: Coordination of Cooperative Cloud Computing Platformscradpdf.drdc-rddc.gc.ca/PDFS/unc274/p805346_A1b.pdf · Coordination of Cooperative Cloud Computing Platforms . ... Coordination

Final Report for RISOMIA Call-up 17

A3 Shibboleth

bull httpswwwswitchchaaiguidesspconfiguration

bull httpswwwswitchchaaisupportpresentationsshibboleth-training-2015Shibboleth-Training-201509

pdf

bull httpsdoitmissourieduwp-contentuploads201409LinuxGuidepdf

bull httpiamharvardeduresourcessaml-shibboleth-integration

A4 Keystone

bull httpswwwibmcomsupportknowledgecenterenSST55W_430liacaliaca_configuring_keystone_

service_providerhtml

bull httpibm-blue-box-helpgithubiohelp-documentationkeystonek2k-federation

A5 Wiki and Blogs

bull httpblogrodrigodscomit-is-time-to-play-with-keystone-to-keystone-federation-in-kilo

bull httpblogrodrigodscomplaying-with-keystone-to-keystone-federation

bull httpshuquangithubiosetting-up-keystone-to-keystone-federation

bull httpsgithubcomshuquank2k-fed

bull httpshuquangithubio

bull httpxuctarineblogspotca201602how-to-setup-keystone-with-shibbolethhtml

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneintegration_in_openstack_

kilo

bull httpswikiinfnitprogetticloud-areapdaai_integration_with_keystoneaai_integrations_in_

openstack_ocata

A-2

The use or disclosure of the information on this sheet is subject to the restrictions on the title page of this document

  • Executive Summary
  • Contents
  • List of Figures
  • List of Tables
  • List of Acronyms
  • Introduction
    • Background
    • Document Overview
      • Literature Review
        • Research Methodology
          • Keywords
          • Glossary
            • Standards
              • Standard Inventory
              • Relevant Standards
                • CDMI
                • OCCI
                    • Cloud environments
                      • Cloud levels of services
                      • Cloud software review
                        • Tool recommendation
                        • Other recommended books and articles
                          • Books
                          • Articles
                              • Multicloud Existing Applications
                                • Success Stories
                                  • The Interop challenge
                                  • EGI
                                  • NeCTAR
                                    • On-going experiences and projects
                                      • Project Tricircle
                                      • Project Trio2o
                                      • Inter Cloud Resource Federation
                                          • OpenStack Components and Federation Architecture
                                            • Components roles
                                            • OpenStack Federation
                                              • Definition
                                              • Architecture
                                                • Federation architecture and mechanisms
                                                  • Discovery
                                                  • Management and monitoring
                                                      • Installation Guide
                                                        • Centos installation
                                                          • Installation
                                                            • OpenStack installation
                                                              • Architecture
                                                              • Installation
                                                                • Every computer node
                                                                • On the controller node
                                                                    • Federation configuration
                                                                      • Architecture
                                                                      • Configuration Identity Provider (IdP)
                                                                      • Configuration Service Provider (SP)
                                                                          • Applications of Communication Between Federated OpenStack Cloud
                                                                            • Federated OpenStack Features
                                                                            • IaaS application examples
                                                                              • Workload transfer
                                                                                • PaaS application examples
                                                                                  • Database synchronization
                                                                                  • Application provisioning
                                                                                    • SaaS application examples
                                                                                      • Exchange of services
                                                                                          • At-sea Environment Limitation
                                                                                          • Conclusion and Future Work
                                                                                          • Bibliography
                                                                                          • Appendix Installation Internet references
                                                                                            • OpenStack
                                                                                            • OpenStack Federation
                                                                                            • Shibboleth
                                                                                            • Keystone
                                                                                            • Wiki and Blogs