gluster and kubernetes
TRANSCRIPT
![Page 1: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/1.jpg)
GlusterFS & Kubernetes
The State of GlusterFS Support in Kubernetes
![Page 2: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/2.jpg)
Joe Julian
● Long time community member.● Senior Staff Engineer with Samsung SDS/CNCT.● Certified Kubernetes Administrator (my first cert ever, hope it’s worth the
paper it’s printed on)(I haven’t printed it).● Insufferable know-it-all with an opinion about everything.
![Page 3: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/3.jpg)
What is Kubernetes
Kubernetes is ancient Greek for "Helmsman". Root of the word "Governor", "Cybernetics".
Kubernetes is a "Container Orchestrator" or "Cluster Manager".
● Places containers on nodes● Recovers automatically from failure● Basic monitoring, logging, health checking● Enables containers to find each other.
![Page 4: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/4.jpg)
How can gluster be used with Kubernetes
● Persistent state Containers can be restarted at any time due to failed liveness check or deployment.
![Page 5: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/5.jpg)
How can gluster be used with Kubernetes
● Persistent state● Shared Storage
Multiple containers may need to operate on the same data set.
![Page 6: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/6.jpg)
How can gluster be used with Kubernetes
● Persistent state● Shared Storage● Cats
![Page 7: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/7.jpg)
How can gluster be used with Kubernetes
● Persistent state● Shared Storage● Cats
![Page 8: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/8.jpg)
GlusterFS volume driver
![Page 9: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/9.jpg)
GlusterFS Volume Driver
volumes: - name: glusterfsvol glusterfs: endpoints: glusterfs-cluster path: kube_vol readOnly: true
kind: Endpointsmetadata: name: glusterfs-clustersubsets:- addresses: - ip: 10.0.0.1- addresses: - ip: 10.0.0.1
![Page 10: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/10.jpg)
GlusterFS Volume Driver
containers: - name: glusterfs image: nginx volumeMounts: - mountPath: "/mnt/glusterfs" name: glusterfsvol
Mounts the volume to /mnt/glusterfs in the container’s chroot.
![Page 11: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/11.jpg)
GlusterFS Volume Driver
Pros:
● Works with existing volumes● Shared data
Cons:
● Not dynamic● Must be managed externally● Only shared data
![Page 12: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/12.jpg)
Local-volume storage class
![Page 13: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/13.jpg)
Local-volume storage class
kind: StorageClassapiVersion: storage.k8s.io/v1metadata: name: myglustervol namespace: kube-systemprovisioner: kubernetes.io/host-path
The code says:// This Provisioner is meant for// development and testing only and // WILL NOT WORK in a multi-node// cluster.
![Page 14: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/14.jpg)
Local-volume storage class
kind: StorageClassapiVersion: storage.k8s.io/v1metadata: name: myglustervol namespace: kube-systemprovisioner: kubernetes.io/host-path
The code says:// This Provisioner is meant for// development and testing only and // WILL NOT WORK in a multi-node// cluster.
![Page 15: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/15.jpg)
Local-volume storage class
● Can only use /tmp/hostpath_pv● Awkward mount● No actual allocation restriction● Works great!
/tmp/hostpath_pv glusterfs gluster:/myvol ...
/tmp/hostpath_pv/{{ uuid }}
kind: PersistentVolumeClaimapiVersion: v1metadata: name: deluge-configspec: accessModes: - ReadWriteMany resources: requests: storage: 50Mi
![Page 16: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/16.jpg)
HeketiGlusterFS storage class
![Page 17: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/17.jpg)
GlusterFS+Heketi
Heketi provides a REST api for the configuration of GlusterFS which can be (and is) used by several infrastructure tools including Kubernetes. Support for GlusterFS using Heketi is part of the main kubernetes tree and has strong downstream acceptance.
Heketi is the Taíno (a Caribbian aboriginal) word for “One”.
![Page 18: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/18.jpg)
Heketi
● GlusterFS in pods● Strongly opinionated cluster design● Allows metered private PVCs● Provides an excellent api interface to GlusterFS
![Page 19: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/19.jpg)
Heketi
● GlusterFS in pods (privileged)● Strongly opinionated cluster design
○ Whole disk○ volume-per-PVC
● Requires prior knowledge● Doesn’t follow best practices wrt hostnames vs IP addresses
![Page 20: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/20.jpg)
Rook
![Page 21: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/21.jpg)
Rook
Rook is a completely cloud-native toolset to provide turnkey storage to kubernetes. It currently only does so with Ceph, but they want to add GlusterFS support and are seeking help.
![Page 22: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/22.jpg)
Rook
What I like about rook:
$ helm repo add rook-alpha https://charts.rook.io/alpha
$ helm install rook-alpha/rook
![Page 23: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/23.jpg)
Rook
● Established provider of ceph storage to kubernetes● Downstream user = community contributions!● Enhance the user experience
![Page 24: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/24.jpg)
Rook
● Automated decision making● Too easy?● Small team, needs contributions
![Page 25: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/25.jpg)
Glusterd V2?
![Page 26: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/26.jpg)
Questions?
![Page 27: Gluster and Kubernetes](https://reader034.vdocuments.site/reader034/viewer/2022051304/5aacf6fa7f8b9a003b8b460f/html5/thumbnails/27.jpg)
Joe Julian, Samsung SDS
Think about where we’re going with respect to kubernetes.Contribute to Kubernetes, Rook, Kraken.
Be part of the community!
● Join the sig-storage meetings● Hang out in #sig-storage on the
kubernetes slack● Join the rook slack● Hang out with me in #gluster on IRC
github.com/samsung-cnct/krakengithub.com/kubernetes/kubernetesgithub.com/rook/rook
Comment on the Rook + GlusterFS design document! https://goo.gl/L62Bi2