AgendaIntroduction to MayaData - MDAP technology:
● Where are we today? ○ OpenEBS Community / Adoption
● Technology Introduction○ Control plane & Data plane(s) (OpenEBS, Jiva, cStor)○ Node Disk Manager (NDM) - Upstream in Kubernetes○ What is MDAP?
Demo
MayaData and OpenEBSData Agility. Delivered● Proven global team with depth in Storage, DevOps, SaaS and managed services.
Funding by Fidelity / Eight Roads and other long term VCs.● 73 employees. Sponsor of leading container attached storage project OpenEBS and leading
chaos engineering project Litmus.● Committed to our Data Agility mission and our PLOW values.
More at: https://mayadata.io/aboutus
● OpenEBS:○○ 2.5M+ Docker pulls○ 5500+ GitHub stars○ 8k+ users○ 1.3k+ Slack Community users
See OpenEBS vs all CNCF projects: devstats.openEBS.io
Nodes run on physical machines
Pod group containers, share an IP address, and each include a Kubelet agent
K8s Master services include: etcd, APIs, the scheduler, the control manager & others
Node
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Node
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Node
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
K8s MasteretcdAPIs
CntrlSchld
Architecture: Kubernetes
Data Containers run in Pods on physical machines
Data Containers group physical disks local or remote, manage replicas, provide QoS control and more
OpenEBS runs on the Master; delivers services such as: APIs, the storage scheduler, analytics & others
Node
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Node
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Node
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
K8s MasteretcdAPIs
CntrlSchld
OpenEBS SchldAPIs
Converged : Kubernetes + OpenEBS
Data Containers run in Pods on physical machines
Data Containers group physical disks local or remote, manage replicas, provide QoS control and more
OpenEBS runs on the Master; delivers services such as: APIs, the storage scheduler, analytics & others
Maya delivered as SaaS solution to fully automate storage operations
Node
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Node
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Node
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
Container
Kubelet
POD
Container
Container
K8s MasteretcdAPIs
CntrlSchld
OpenEBS SchldAPIs
MayaVisualization
Cross DC controlRBAC
ML coaching
Cloud Converged: K8S + OpenEBS + Maya
K8s NodeDisks
Storage Pool
PVs
Pool or Storage Pod
Pool Types{
Local ext4 ( hostDir) OpenEBS JivaLinux LVMOpenZFSPortworxGlusterPoolCePHPool
}
Local disks
iSCSI targets
GPD
EBS
FC targets
Controller (IO) Pod
Controller (IO) Pod
Critical IO Path
Long Running
Resilient to Disk Errors
Replace/Attach Disks
“CAS” - A Storage Implementation Pattern
Node
Control PlaneMonitoring/Alert
Manager (Prometheus)
Node Disk Manager
eBPF, NPD
Kube API Server
Storage Control Plane (Local PV/ OpenEBS/GlusterFS)
Storage Pods(OpenEBS/GlusterFS
External Storage Provisioners
node-disk-manager(ndm) runs as a DaemonSet detects the disks attached to the node and creates corresponding Disk objects.
ndm monitors and detects the faults on the disks and can be configured to push events to Storage Control Plane or Storage Pods running on the local node. It can also consume disk related events from Node Problem Detector (NPD), registering to disk events via eBPF.
ndm will expose metrics that can be used by Prometheus
ndm can be configured to provision additional disks by interacting with external storage provisioners.
Node-disk-manager (NDM)
Jiva storage engine is developed with Rancher's LongHorn and gotgt as the base. Written in GO language and runs in the user space. LongHorn controller synchronously replicates the incoming IO to the LongHorn replicas. The replica considers a Linux sparse file as the foundation. It supports thin provisioning, snapshotting, cloning of storage volumes.
Pluggable OpenEBS Storage EnginescStor is a high performing storage engine built with proven building blocks of storage components. Access protocol iSCSI stack is a linux ported BSD based Multi-threaded iSCSI protocol stack. Field tested at thousands of installations for many years". The storage block layer is the DMU layer of user space ZFS inherited from the proven OpenSolaris stack. Highly reliable for storing and protecting enterprise data.
MayaData Confidential
+ OpenEBS: Leading OSS containerized storage
+ Litmus:Leading OSS chaos engineering for stateful workloads
+ MayaOnline:Control and visibility
+ Services such as DMaaS: Data migration as a service
+ Proactive support
MDAP
16
MDAP commercial solution
○ Leading OSS containerized storage
○ Trivial to set up and run
○ Accelerates & empowers DevOps thanks to snaps and automation and much more
○ Cost savings and independence vs. proprietary cloud or hardware storage
○ Leading OSS chaos engineering for stateful workloads on Kubernetes
○ Increases confidence and resilience
○ User stories directly translated into automated scenarios
○ Integrated into MayaOnline
○ Unequaled visibility into stateful workloads on Kubernetes
○ Reveals hot spots, inconsistencies and more
○ Free version for initial usage
○ Customers receive RBAC enabled version with other extended capabilities