microservices & containerscc.ee.ntu.edu.tw/~rbwu/rapid_content/course/iot_intro/... · 2019. 9....
TRANSCRIPT
-
National Taiwan UniversityDepartment of Electrical Engineering
Alex Lai, Ph.D.
Adjunct Professor
Cyber III:
IoT, Microservices, &
Software Containers
-
National Taiwan UniversityDepartment of Electrical Engineering
Microservices & Software Containers
Introduction of Docker and Kubernetes
Hands-on
Agenda
2
-
National Taiwan UniversityDepartment of Electrical Engineering
Virtualization is one of the most well-known
mechanisms to implement Cloud Computing.
Recap: A Note on Virtualization
3
-
National Taiwan UniversityDepartment of Electrical Engineering
Formally OS-level virtualization (“Containerization”)■ = The OS kernel allows the existence of multiple isolated user-space
contexts.
Good at■ Fast deployment of microservices. (快進快出)
Compared to VM Lower overhead
Faster deployment
No heterogeneity
Lower protection
相輔相成而非取代
Recap: A Note on Container (vs. VM)
*Source: Adapted from Wikipedia and InwinStack presentation. 4
-
National Taiwan UniversityDepartment of Electrical Engineering
5*Source: Adapted from Microservices: From Design to Deployment by Chris Richardson with Floyd Smith, an NGINX eBook, 2016. .
“Control” “Coordination”
Microservices – A variant of the service-oriented architecture (SOA)
organizing an application as a collection of fine-grain, loosely coupled,
autonomous primitives, with clear scope.
Compared to the old-school “monolithic” architecture:
IoT Realization via Microservices*
-
National Taiwan UniversityDepartment of Electrical Engineering
A Software Container Premier
6
Containerization, also known as OS-level virtualization, refers to an
OS feature in which the kernel allows the existence of multiple isolated
user-space instances (software containers).
Why Containerization: Good at implementing Microservices.
Example of Container Technologies:
LXC (Linux Containers)
*Source: Adapted from Wikipedia
-
National Taiwan UniversityDepartment of Electrical Engineering
Linux Containers (LXC) An operating system–level virtualization method for running multiple
isolated Linux systems (containers) on a single control host.
7
-
National Taiwan UniversityDepartment of Electrical Engineering
Linux Kernel Features used by LXC
Namespaces
(mnt, pid, net, ipc, uts/hostname, user ids)
cgroups
(cpu, memory, disk, i/o - resource management)
AppArmor, SELinux (security/access control)
seccomp (computation isolation)
chroot
(file system isolation)
8
-
National Taiwan UniversityDepartment of Electrical Engineering
More about Linux Containers
9*Source: Adapted from linuxcontainers.org.
-
National Taiwan UniversityDepartment of Electrical Engineering
Docker, started as an open source
project in 2013, is an open-
source project that automates the
deployment
of applications inside software
containers
Docker uses the resource isolation
features of the Linux kernel such
as cgroups and kernel namespaces,
and a union-capable file
system such as OverlayFS to allow
independent "containers" to run
within a single Linux instance.
Container Example (2): Docker
10
https://en.wikipedia.org/wiki/Open-sourcehttps://en.wikipedia.org/wiki/Application_softwarehttps://en.wikipedia.org/wiki/Software_containerhttps://en.wikipedia.org/wiki/Linux_kernelhttps://en.wikipedia.org/wiki/Cgroupshttps://en.wikipedia.org/wiki/Linux_namespaceshttps://en.wikipedia.org/wiki/Union_mounthttps://en.wikipedia.org/wiki/OverlayFS
-
National Taiwan UniversityDepartment of Electrical Engineering
Docker Container
Docker Image
Docker Registry
Dockerfile
A Simplified Docker Overview
11
-
National Taiwan UniversityDepartment of Electrical Engineering
Docker is built upon
cgroup and namespacing capabilities of the
Linux kernel
Go programming language (written in Go)
Docker Image Specification (for container image
management)
LibcontainerSpecification (namespaces,
filesystem, resources, security, etc)
12
-
National Taiwan UniversityDepartment of Electrical Engineering
Virtual Machines Vs Docker Containers
13
-
National Taiwan UniversityDepartment of Electrical Engineering
Docker Architecture
14
Docker Architecture
-
National Taiwan UniversityDepartment of Electrical Engineering
Docker Image Structure
15
-
National Taiwan UniversityDepartment of Electrical Engineering
Limitations with standalone Docker
16
-
National Taiwan UniversityDepartment of Electrical Engineering
Introduction to Kubernetes (k8s)
17
-
National Taiwan UniversityDepartment of Electrical Engineering
Kubernetes (known as k8s) is an open-source system for automating deployment, scaling, and management of containerized applications.
It groups containers that make up an application into logical units for easy
management and discovery. Kubernetes builds upon 15 years of
experience of running production workloads at Google, combined with
best-of-breed ideas and practices from the community.
With Kubernetes, users are able to create and manage microservices:
■ Deploy applications quickly and predictably.
■ Scale applications on the fly.
■ Roll out new features seamlessly.
■ Limit hardware usage to required resources only.
■ Supports GCE, CoreOS, Azure
■ Can manage Docker containers as a default implementation
Kubernetes – What it is
18
-
National Taiwan UniversityDepartment of Electrical Engineering
⧫ Automated arrangement, coordination, and management of containers
Why Kubernetes:Container Orchestration
Kubernetes Worker 3
Running Environment 1
is the best place for you
Create a copy to make a HA
Kubernetes Worker 2
Kubernetes Worker 1
Kubernetes Master
19
-
National Taiwan UniversityDepartment of Electrical Engineering
What can k8s do?
Kubernetes provides a container-centric management
environment. It orchestrates computing, networking, and
storage infrastructure on behalf of user workloads.
This provides much of the simplicity of Platform as a
Service (PaaS) with the flexibility of Infrastructure as a
Service (IaaS), and enables portability across
infrastructure providers.
20
-
National Taiwan UniversityDepartment of Electrical Engineering
High-level k8s Architecture
21
-
National Taiwan UniversityDepartment of Electrical Engineering
High-level k8s Overview
22
22
Master
Node 1 Node 2 Node 3
Service
Service A Service B Service C Service D
MasterMaster
Pod A-1
Container
Container
Container
Pod A-1
Container
Container
Container
Pod A-1
Container
Container
Container
Pod A-2
Container
Container
Pod A-2
Container
Container
Pod B-1
Container
Container
Container
Pod B-1
Container
Container
Container
Pod B-2
Container
Pod C-1
Container
Container
Container
Pod C-1
Container
Container
Container
Pod D-1
Container
Container
Container
Pod D-2
Container
Container
Container
Container
Pod D-2
Container
Container
Container
Container
-
National Taiwan UniversityDepartment of Electrical Engineering
High Level Components of k8sMaster
API Server
Controller Manager
Etcd
Service and Labels
Node (VM/PM)
Kubelet
Kube-proxy
Pod
Container
23
-
National Taiwan UniversityDepartment of Electrical Engineering
Master Componentskube-apiserver
Component on the master that exposes the Kubernetes API. It is the front-
end for the Kubernetes control plane.
etcd
Consistent and highly-available key value store used as Kubernetes’
backing store for all cluster data.
kube-scheduler
Component on the master that watches newly created pods that have no
node assigned, and selects a node for them to run on.
kube-controller-manager
Node Controller, Replication Controller, Endpoints Controller, Service
Account & Token Controllerscontroller-manager.
https://kubernetes.io/docs/concepts/overview/components/
24
-
National Taiwan UniversityDepartment of Electrical Engineering
Minion Node (“Node”) Overview
25
-
National Taiwan UniversityDepartment of Electrical Engineering
kubelet
An agent that runs on each node in the cluster. It makes sure that
containers are running in a pod.
kube-proxy
Enables the Kubernetes service abstraction by maintaining network rules on
the host and performing connection forwarding.
Container Runtime
The software that is responsible for running containers. Kubernetes
supports several runtimes: Docker, rkt, runc and any OCI runtime-spec
implementation.
*Source: https://kubernetes.io/docs/concepts/overview/components/ 26
Minion Node (“Node”) Components
-
National Taiwan UniversityDepartment of Electrical Engineering
Cluster NetworkingNeed to achieve:
Highly-coupled container-to-container communications
Pod-to-Pod communications
Pod-to-Service communications
External-to-Service communications
Additionally required ingredients
Flannel/Calico, OpenVSwitch, NSX-T…
*Source: https://kubernetes.io/docs/concepts/cluster-administration/networking/ 27
-
National Taiwan UniversityDepartment of Electrical Engineering
LoadBalancer - Haproxy Server(Docker on a VM)
node1 node2 node3
Master 1, 2
Link up by
Networking
PodService: Dashboard, SkyDNS, Monitoring, EFK-Stack
Architecture – Revisited & Revised
28
-
National Taiwan UniversityDepartment of Electrical Engineering
Namespace, Services, Pod, Expose
Namespace BNamespace A
Service X
Select “X”
e.g.,
http://service-X
Service Y
Select “Y”
e.g.,
http://service-Y
Service Z
Select “Z”
e.g.,
http://service-Z
Expose: NodePort, LoadBalancer, Ingress
29
Kubernetes Runtime
-
National Taiwan UniversityDepartment of Electrical Engineering
Link up by Flannel Networking
30*Source: Adapted from https://github.com/coreos/flannel
-
National Taiwan UniversityDepartment of Electrical Engineering
Link up by Calico Networking
31
*Source: Adapted from Project Calico Web page (https://www.projectcalico.org/)
-
National Taiwan UniversityDepartment of Electrical Engineering
Pods ControllersReplicationController
ReplicaSet
Deployments
DaemonSet
Jobs - Run to Completion
*Source: https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/#how-to-use-a-replicaset 32
-
National Taiwan UniversityDepartment of Electrical Engineering
Working with kubectl
33
-
National Taiwan UniversityDepartment of Electrical Engineering
Scripting the Pod Controller ActionsAn Example of kubectl YAML file
Deployment
Replica Set
Pod Spec
Container
Spec
Container
Spec
How are updates handled?
Rolling/recreation
How many Pods should run?
Node selector Service labels
Docker image Environment
variables Storage Claims
34
-
National Taiwan UniversityDepartment of Electrical Engineering
Exposing Services (1) - NodePort
35
-
National Taiwan UniversityDepartment of Electrical Engineering
Exposing Services (2) - LoadBalancer
36
-
National Taiwan UniversityDepartment of Electrical Engineering
Exposing Services (3) - Ingress
37
-
National Taiwan UniversityDepartment of Electrical Engineering
Additional Services in k8s PodsDashboard
SkyDNS
Monitoring
InfluxDB
Heapster
Grafana
EFK-Stack(log)
Elasticsearch
Fluent
Kibana
38
-
National Taiwan UniversityDepartment of Electrical Engineering
k8s Environment in NTUEE
NTUEE Kubernetes Development Environment for IoT
Master : 140.112.18.229
• SSH Login(ID: student1 / Password: student1)
Node1 : 140.112.18.230
Node2 : 140.112.18.231
A “Hello World” Example
嘗試建立與刪除一K8s service, 此service包含由Apache Pod與其他所組成
39
-
National Taiwan UniversityDepartment of Electrical Engineering
DashBoard
https://140.112.18.229:6443/api/v1/namespaces/kube-
system/services/https:kubernetes-dashboard:/proxy/#!/login
Token Login
• https://hackmd.io/s/H1yzV-sjz
Login
40
40
https://140.112.18.229:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/loginhttps://hackmd.io/s/H1yzV-sjz
-
National Taiwan UniversityDepartment of Electrical Engineering
首次登入畫面
41
• 修改URL中的namespacedefault改成各組名稱(例:team1)
• 提示訊息:權限不足
41
-
National Taiwan UniversityDepartment of Electrical Engineering
42
正確進入所屬Namespace
-
National Taiwan UniversityDepartment of Electrical Engineering
建立一個Service,一個Pod包含apache, grafana 兩個container
YAML Example:https://hackmd.io/s/ByU1rZioz
①
②建立方式• 上傳已編輯的.yaml檔案• 透過dashboard UI
43
新建一個Service
https://hackmd.io/s/ByU1rZioz
-
National Taiwan UniversityDepartment of Electrical Engineering
使用空白字符(不能使用Tab)分層,同層元素左側對齊
用“-”來表示一些序列的項(Sequence)
用“:”來表示一對項目(Map)里的欄目(Key)和其相應的值(Value)
單行註解由“#”字號開始,可以出現在行中任何位置
字符串一般不使用引號,但必要的時候可以用引號框住
YAML也有用來描述好幾行相同結構的數據縮寫語法,數組用“[]”包括起來,hash用“{}”來包括
44
A Note on YAML Fundamentals
-
National Taiwan UniversityDepartment of Electrical Engineering
• Apache container 內部port 80, 外部 32391• Grafana container 內部 port 3000, 外部 32260
45
新建完成
-
National Taiwan UniversityDepartment of Electrical Engineering
Master, Node1, Node2 IP:外部PORT
Master IP Node1 IP Node2 IP
46
Apache 測試
-
National Taiwan UniversityDepartment of Electrical Engineering
①
②
47
刪除-Pod
-
National Taiwan UniversityDepartment of Electrical Engineering
①
②
48
刪除-Service
-
National Taiwan UniversityDepartment of Electrical Engineering
Kubeadm (官方推出)
Minikube (官方推出)
Ansible
Rancher(Web UI) https://rancher.com/kubernetes/
Kubernetes快速部署工具
49
49
-
National Taiwan UniversityDepartment of Electrical Engineering
Hands-on
50
-
National Taiwan UniversityDepartment of Electrical Engineering
Before we start: Preparation
1. Set up & Test-drive Docker Environment
2. Set up & Test-drive k8s by employing Rancher admin console
3. Link a Physical Connected Thing (your RPi) to the k8s service
51
Objectives
-
National Taiwan UniversityDepartment of Electrical Engineering
Download Lab Container VM Images w/ Ubuntu 16.04 64-bitEither VMWare Workstation & VirtualBox version
Use Bridge mode (instead of NAT) in Networking configuration
52
Before Starting…
Bridge Mode in VMWare Workstation
Bridge Mode in VirtualBox
-
National Taiwan UniversityDepartment of Electrical Engineering
Configuring the Remaining Essential SettingsSSH (w/ Root Login permission)
Set Proper Firewalling Rules
53
Before Starting (cont.)
#sudosudo -i
#updateapt-get update
#SSH Serviceapt-get -y install openssh-server
#如出現「E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it」,執行:rm -rf /var/lib/dpkg/lock
#install vimapt-get -y install vim
#install curlapt-get -y install curl
#修改root密碼passwd root
#讓Root登入vi /etc/ssh/sshd_configPermitRootLogin prohibit-password 改 PermitRootLogin Yes
#restart sshd servicesystemctl restart sshd
#停用防火牆 BAD Practice but just works for testing nowsystemctl stop ufwsudo ufw disable
-
National Taiwan UniversityDepartment of Electrical Engineering
安裝指令: docker version
官網安裝說明: https://docs.docker.com/install/
GitHub : https://github.com/rancher/install-docker
54
Set up Docker Service in your VM
• apt-get install -y apt-transport-https ca-certificates curl software-properties-common
• curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
• add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
• apt-get update
• apt-cache madison docker-ce
• apt-get install -f -y docker-ce=17.03.2~ce-0~ubuntu-xenial
#安裝完後確認及服務是否正常# 版本確認:• docker version
服務確認• systemctl status docker
-
National Taiwan UniversityDepartment of Electrical Engineering
https://hub.docker.com/
55
The Docker Hub
https://hub.docker.com/
-
National Taiwan UniversityDepartment of Electrical Engineering
docker pull [image name]
docker images
此處以 Ubuntu Image 為例
56
Pull a Docker Image from Repository
-
National Taiwan UniversityDepartment of Electrical Engineering
docker run [OPTIONS] IMAGE[:TAG|@DIGEST]
[COMMAND] [ARG...]
docker exec -it [name or container id] bash
apt-get update
apt-get install [service name]
docker stop [name or container id]
docker start [name or container id]
docker restart [name or container id]
57
Run, Operate & Configure Containers
-
National Taiwan UniversityDepartment of Electrical Engineering
docker run -d -p 5000:5000 -v
/mnt/data/registry:/var/lib/registry --name
registry registry
docker run -d -p 8080:8080 --name registry-web
--link registry -e
REGISTRY_URL=http://10.1.1.196:5000/v2
hyper/docker-registry-web
Commit 改過的Imagedocker commit [OPTIONS] CONTAINER
[REPOSITORY[:TAG]]
58
Set up & Commit into a
Private Docker Repository
-
National Taiwan UniversityDepartment of Electrical Engineering
A ‘Hello World’ Example of Docker
Get one base Docker image
https://hub.docker.com/
List Docker images
Run “hello world” and List Container
59
-
National Taiwan UniversityDepartment of Electrical Engineering
‘Hello World’ (cont.): Detached mode
Run hello world in detached mode(-d)
Exec to the container
stop/ start /restart the container
#docker stop
60
-
National Taiwan UniversityDepartment of Electrical Engineering
Rancher 2.0 已經包裝好為 container 形式使用putty ssh 登入VM,並輸入下列指令docker run -d --restart=unless-stopped -p
80:80 -p 443:443 rancher/rancher
61
k8s: Install Rancher Admin Console
-
National Taiwan UniversityDepartment of Electrical Engineering
開啟瀏覽器,輸入VM的IP,即會出現登入畫面,第一次登入請先設定admin密碼:
62
k8s: Start Rancher
-
National Taiwan UniversityDepartment of Electrical Engineering
設定Server URL 並儲存
63
k8s: Configure Rancher Server URL
-
National Taiwan UniversityDepartment of Electrical Engineering
Add Cluster->點選「CUSTOM」,並「Cluster Nmae」中輸入自定名稱,完成後點選「NEXT」
64
k8s: Add a Cluster from Rancher
-
National Taiwan UniversityDepartment of Electrical Engineering
勾選「etcd, Control Plane, Worker」將docker指令複製,完後點選「Done」
65
k8s: Configure Nodes from Rancher
-
National Taiwan UniversityDepartment of Electrical Engineering
回到putty畫面,貼上剛剛複製的docker指令即開始安裝, 可用Web browser看到各services安裝過程:
66
k8s: Configure Nodes (cont.)
-
National Taiwan UniversityDepartment of Electrical Engineering
點選「Default」Namespace,再點選「Import YAML」
67
k8s: Testing your 1st Cluster
-
National Taiwan UniversityDepartment of Electrical Engineering
點選「Default」Namespace,再點選「Import YAML」What is the Service described in the YAML file?
68
k8s: Import YAML Test File
-
National Taiwan UniversityDepartment of Electrical Engineering
The YAML File to be used hereapiVersion: v1kind: Servicemetadata:
namespace: defaultname: nginx-svclabels:
service: nginx-servicespec:
selector:service: nginx-pod
type: NodePortports:- name: http
port: 80 ---apiVersion: extensions/v1beta1kind: Deploymentmetadata:
name: nginx-podnamespace: default
spec:replicas: 1template:
metadata:labels:
service: nginx-podspec:
containers:- name: nginx-pod
image: nginximagePullPolicy: IfNotPresentports:- containerPort: 80
69
k8s: Import YAML Test File
-
National Taiwan UniversityDepartment of Electrical Engineering
Service
Pod
70
k8s: Your 1st Service & Pod
-
National Taiwan UniversityDepartment of Electrical Engineering
71
Session Takeaways
Microservice (微服務) 是一種軟體架構,以協調多數小型功能區塊為基礎,利用模組化的方式組合出複雜的大型應用程式。相較於注重控制流程的單體式(Monolithic) 應用,Microservice輕量化與快進快出的特性,適用於現代小型但量多之服務,包括IoT在內。
Software Container (軟體容器)是藉由OS層虛擬化(OS Level Virtualization) 防止各執行程式之資料與命名空間互相干擾,因此特別適合實作微服務。
由當代標準 Kubernetes (k8s) 與 Docker 實務演練中,瞭解如何使用Container實作Micro Cloud Service for IoT.
-
National Taiwan UniversityDepartment of Electrical Engineering
Docker 官網: https://docs.docker.com/
互動操作學習: https://www.katacoda.com/courses/docker
Docker-從入門到實踐:https://philipzheng.gitbooks.io/docker_practice/content/introduction/what.html
K8S部署: https://kairen.github.io/
官網: https://kubernetes.io/
互動操作學習: https://www.katacoda.com/courses/kubernetes
72
參考資料與推薦閱讀
https:https:https:https:https:
-
National Taiwan UniversityDepartment of Electrical Engineering
Q&A
73