Fog Computing for Robotics and Industrial Automation

Paul Pop, Professor of Cyber-Physical SystemsDTU Compute, Technical University of Denmark

Ph.D. inComputer Systems

Linköping UniversitySweden

Associate professorDTU Compute,Technical Universityof Denmark (DTU)

Assistant professorLinköping University Sweden

ProfessorDTU Compute,

Technical Universityof Denmark (DTU)

Director

2

Industry 4.0

Digitization of the manufacturing sector, with embedded sensors in virtually all product components and manufacturing equipment, ubiquitous cyberphysical systems, and analysis of all relevant data.

Definition

Benefits $15Trillionincrease in global GDPwith Industrial internetover the next 20 years

Source: McKinsey

3

“Software is eating the world”

IT has transformed our society. Everything is now software in the Cloud.

What do all of these have in common?

Will the factorymove the the Cloud?

4

The automation pyramid and OT

The Automation Pyramid:

Soon to be Ancient History?

22

Rigid infrastructure with

separation between levels

of functionality

Levels connected by dedicated,

specialist networks

Data exchange only via gateways

or proprietary systems

Difficulties to transparantly

access data at the cyber

pyramid (machine) level

Operationstechnology (OT)

The hardware and software dedicated to detecting or causing changes in physical processes through direct monitoring and/or control of physical devices such as valves, pumps, etc.

Source: TTTech

5

We cannot move the factory to the Cloud because of OT

High dependability, long lifetime, safety-criticalHard real-time

Tough requirements

Reboot? Retry? Restart?

6

What is Cloud that comes close to the ground (edge of the network)?

System-level horizontal architecture that distributes resources and services of computing, storage, control and networking anywhere along the continuum from Cloud to Things.Similarities: (Dependable Real-Time) Edge Computing

© Nebbiolo Technologies 6

The IoT Infrastructure and Fog Computing

Public'Clouds' Private'Clouds'

Enterprise)Data)Center)

Machines,'devices,'sensors,'actuators,'things'

Tradi7onal'ICT'end9points'

Public'and'Private'Communica7ons'Infrastructure''

(Internet,'Enterprise'Ethernet,'WiFi)'

Compu7ng'and'Storage'Today:'

1. Clouds'2. Enterprise'Datacenters'3. Tradi7 onal'and'Embedded'

Endpoints'

IoT'requires'more'virtualized,'scalable,'reliable,'secure,'real97me'capable'Compu7ng'

and'Storage'at'the'Edge:'

Fog'Compu7ng!''

What is Fog Computing?

Source: Flavio Bonomi

7

Industry 4.0 = Convergence of IT and OT

Information Technologies:

Virtualization, Big Data, Automation Analytics, Scalability, SDN, Security and Privacy

Operations Technologies:

Real-time, Safety, Reliability, Control, Machine Connectivity and Data Acquisition, Human Machine Interface

IT

OT

Cloud

Fog

IoT Endpoints

8

Vision: OT becomes virtualized

9

A history of Fog

FlavioBonomi

Sept 2010

Bay area

Source: Flavio Bonomi

April 2016 Hannover Messe

KUKA calls it Real-Time Edge Computing

10

Fog: Converged Functionality for Industrial Automation

Cell 1 Cell N

Machine Vision and Bar Code Reader

PLC System

Robotics ControllerAnd Visualization

Cell Controller

Manufacturing Execution System

ERP/MRP System

Adding Functionality

Nebbiolo Fog Node

Data Center/Clouds

Converging Functionality

Source: Flavio Bonomi

11

Fog Computing: a new Infrastructure layer

1. Communications, gateway networking convergence

2. Edge data management, analytics

3. Distributed application hosting

4. Virtualization of all resources, multi-tenancy

5. Security and Privacy

6. Real-time, local control

7. Scalability

8. Reliability

Cloud

Fog

IoT Endpoints

12

Fog: Hardware convergence

47

Industrial Micro-controller

PLC

Industrial PC

Real-time, High-availability, Industrial Fog Node

Virtualizing and consolidating , multiple PLCs and Industrial PCs

(SBCs)

3 Slot System:=

20-30 Industrial PCs

3 Slot System:=

~15-20 IoT Gateways

IoT Gateway

Data center Servers or

Micro-Servers

13

Fog: Virtualization and Security

A combination of physical separation (multicore), hard, RT-NRT Virtual Board/Machine based virtualization and more lightweight Linux Container based virtualization

GuestOS+Hypervisor (KVM) or Thin Hypervisor (Wind River, Lynx,..)

Linux RT-OS 2Windows RT-OS 1…

RT-AppsRT-AppsWin-Apps

NRT-VM 2 RT-VM 2RT-VM 1

LXC

1

LXC

N…

AAA

LXC: Linux Container

Virtualization

Decentralized model, including distributed authentication, trusted booting, secure software management, SDN based connectivity control, OpenVPN, NFV Security…

Security

14

Network convergence: IEEE Time-Sensitive Networking

IntegrationMultiple traffic classes share the network, supporting applications with mixed-criticality requirements

Separation: Virtual links separate different criticalities

Principles

Fog

IoTEndpoints

DeterministicNetworking

Standardization as IEEE TSN

19

A number of IEEE 802.1 standards due for release early in 2017

15

Open standards will win: IEEE 802.1 TSN and OPC UA over TSN

Creating a Flexible IoT Infrastructure

with Deterministic Ethernet

24

OPC UA over TSN: one unified

architecture for all communication

infrastructure elements

Logical machine/control

boundaries are dissolved

Direct access to machine

data from ERP/MES

Real-time and non-real-time

domains integrated

OPC UA provides leading

technology of proven security

concepts

Learning from the Internet

Experience: Digital Platforms

and Open Innovation win. Source: TTTech

16

Software convergence: an app store for the factory

17

Safety: Certification is very expensive

For safety-critical systems, decision-makers require pre-release safety assurance evidence that it manages risk acceptably.

Highly critical systems require certificationby a regulatory authority.

Verification

System

System and Safety Requirements

Argumentation Environment

Evidence

Safety standard

Documents

Company Independent safety assessor

Certificate

Source: Hans Hansson

18

The industry is not the new “space customer”

Components are produced for computers and consumer electronics; they have 3-6 year production lives and narrow operating temperature ranges

The needs of the “space customer” are not addressed

Industry is not prepared to pay ”space customer” prices

Source: Boeing

Design

Production

Airplanes:

Service

Computers: Design

Production

Service

System archictecture

Software

Mfg. processes

Piece parts

0 10 20 30 40

Time, years

Technology element:

19

Security

Security Framework 2: Motivation

IIC:PUB:G4:V1.0:PB:20160926 - 14 -

critical infrastructure. With a geographically distributed IIoT system, care must be taken to ensure that disruption of an isolated system does not cascade to have global effects.

Organizations must take these risks seriously; they must use their expertise to make their IIoT systems trustworthy. The use of sensors and actuators in an industrial environment is not the typical Information Technology (IT) experience, nor are systems that span many organizations and organizational systems. IT and OT prioritize system characteristics differently. For example, resilience in IT is less important than in OT, and security is less important in OT than in IT, as illustrated in Figure 2-1. These characteristics interact with each other, and can conflict. In IIoT systems, these system characteristics must converge and be reconciled with each other into overall system trustworthiness.

Figure 2-1: Convergence of IT and OT Trustworthiness

IIoT organizations must place increased importance on safety and resilience beyond the levels expected in many traditional IT environments. IIoT systems may also have data flows that include intermediaries and involve multiple organizations, requiring more sophisticated security approaches than, for example, link encryption. Unfortunately, IT departments rarely speak the same language as those concerned with control systems and OT. The two perceive risk differently, and they cannot be combined for positive gain without a balanced consideration of their differing motivations.

The highest priority of many OT systems is safety: do not cause injury or death, do not put public at risk and protect the environment from harm. The second and third priorities are often quality of production and meeting production targets, which depend on the reliability and resilience of the system. Reliability and resilience are required to prevent the interruption of society-critical processes such as the electric grid, and to avoid idling machinery that represents large investments in physical infrastructure. Security aspects are considered in OT, but given that most systems are not connected it is mostly physical security. (Some industries, such as healthcare, must protect patient data.) Security concepts such as user-based access control applies less often in OT systems than they do in IT.

20

The Cloud will come to the factory

44 45Hello Industrie 4.0 we go digital

Economic, social, technological and ecological megatrends such

as globalization, urbanization, digitization and sustainability are

radically changing the industrial landscape of the future. This poses

new challenges for manufacturing industry. Industry is responding

to these processes of change and to the increasing competitive

pressure with shorter product cycles and a more differentiated

product portfolio.

Versatile solutions are required to cushion workload peaks and

resource bottlenecks in the age of Industrie 4.0. Matrix production

may become a decisive competitive factor through configurable

production cells, the transfer of parts and tools using automated

guided vehicles (AGVs) and the separation of logistics from

production.

Matrix productionHighly flexible production is becoming reality.

The production cell for easy conversion.

Four robots handle and join the parts. Tool maga-

zines in the cells enable provision of the required

tools. This enables adaptation of the cell during

the cycle time.

Cloud

Fog

High risk of change, but!

High impact, and new equipment needed only for about about 40 to 50% of installed base

Source: McKinsey

21

Industry 4.0 vision: Research at DTU Compute

ConfigurationtoolsResource

management

FogMiddleware

OPCUA/DDS

HypervisorpartitionedOS

FogApp

VirtualizedControl

Pla

tfo

rm

FOGNODE

Se

curi

ty,

Saf

ety

and

De

pe

nd

abil

ity

Computation

ESR14ESR13

ESR

12ES

R11

ESR8

ESR7

ESR2

ESR1

ESR3

ESR4

ESR5

Storage Communication

FogApp

FogApp

API

API

API

ESR6

ESR10 ESR9

Dataanalytics

WP1FogComputingPlatform

WP2

Res

ourc

eM

anag

em

ent

and

Mid

dlew

are

WP3DependabilityServicesandApplicationModeling

AppAppApp

ESR15

FORA proposal: 15 PhDsFog Computing for Robotics and Industrial Automation

DTU’s IoT Center, iotcenter.dk

We are open to researchers from other departments at DTU, other Danish universities, and we’re welcoming company members.

Prof. Paul Pop. Industrial IoTProf. Nicola Dragoni. IoT Security and Privacy Prof. Jens Sparsø and Assoc. Prof. Martin Schoeberl. IoT Platforms Assoc. Prof. Michael S. Berger. IoT Communications Infrastructure Prof. Henrik Madsen. IoT Cyber-Physical Modeling and Smart Energy IoTProf. Lars Kai Hansen’ group. IoT Data Analytics Prof. Jan Madsen. Health IoTAssoc. Prof. Sven Karlsson. IoT System Software

Mission: Performing research on IoT technologies and facilitating collaboration among researchers and practitioners in Denmark.

Vision: Dependable and secure IoT enables efficient solutions to societal challenges.

23

Thank you!

1.Machine

4.Enterprise

3.Manufacturing

2.Control

EnterpriseManu-

facturing

Machine Control

IT+OTIT

OT

IT-OTgap crossedwithmulti-disciplinary,inter-sectoral,internationalResearch