hackfmi talk #3 - interesting times: 6 current paradigm shifts in it - teodor tonchev
DESCRIPTION
Teodor Tonchev, from VMWare Bulgaria, with education from MIT and 18 years of experience in the IT industry, talked about the radical changes in the industry, that will change the way we think about and create Software. From Virtualization, Cloud, App Development to Polyglot Persistence Layers and Software Defined DatacentersTRANSCRIPT
![Page 1: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/1.jpg)
© 2013 VMware Inc. All rights reserved
Interesting Times: Paradigm Shifts in IT
Teodor Tonchev, Staff Engineer, VMware
![Page 2: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/2.jpg)
2
Definition
Paradigm Shift:
A change from one way of thinking to another
![Page 3: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/3.jpg)
3
Virtualization
![Page 4: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/4.jpg)
4
Create Virtual Machines
Run different Operating
Systems and Applications
within these VMs
Clone/Move/Backup VMs
Snapshot/Restore VMs
VMware Workstation™
Virtual Machines
Chang
ed
script
and
slide
OS
APP
OS
APP
OS
APP
OS
![Page 5: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/5.jpg)
5
Hypervisor – an Operating
System for running VMs
Many servers now come
with ROM-based hypervisor
VMware ESX™
Server Virtualization
Chang
ed
script
and
slide
OS
APP
OS
APP
OS
APP
![Page 6: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/6.jpg)
6
OS
APP
vMotion allows running VMs
to be moved among servers
seamlessly
Clients connected to the
VMs do not observe any
changes VMware vSphere™
VMware vMotion
Chang
ed
script
and
slide
![Page 7: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/7.jpg)
7
OS
APP
OS
APP
OS
APP
DPM consolidates
workloads onto fewer
servers when the cluster
needs fewer resources
Places unneeded
servers in standby
mode
Brings servers back
online as workload
needs increase
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
VMware vSphere™
VMware DPM (Distributed Power Mgmt)
DPM powers off
server when
requirements
are lower
DPM brings
servers back
online when
load increases
Chang
ed
script
and
slide
![Page 8: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/8.jpg)
8
Distributed Resource Management (DRS)
Shrink and grow of
applications based on
demand and priority
Dynamic and responsive
load balancing VMware vSphere™
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
![Page 9: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/9.jpg)
9
Current Virtualization Use
2012 Survey: 51% of x86 Servers Now Virtualized
Nearly 92 percent of all enterprises use at least some form of
virtualization.
Majority of servers on the market support integrated hypervisor
![Page 10: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/10.jpg)
10
Cloud
![Page 11: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/11.jpg)
11
What is Cloud Computing?
Where at home do you keep your Electric Power Station?
Cloud: Computing resources provided on demand, as a service,
remotely (Internet or Intranet)
• Utility-Based Computing is another term for cloud computing
![Page 12: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/12.jpg)
12
Cloud Categories
![Page 13: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/13.jpg)
13
Starting a software service is much easier now
Application Server
OS
App Libs
DB
With IaaS there is no need to buy hardware.
With PaaS one can focus only on developing the App.
![Page 14: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/14.jpg)
14
Application Development
![Page 15: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/15.jpg)
15
Pets vs. Cattle
![Page 16: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/16.jpg)
16
Multicore Future
Chips are reaching the physical limits
Moore’s law still holds, but due to increasing the number of cores
Now: tens, hundreds of cores
Soon: 1000s of cores
2020s: Millions of cores
![Page 17: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/17.jpg)
17
Parallel programming
Approach 1: Shared Memory
• Threads
• Locks/Mutexes
DOES NOT SCALE. COMPLEX (e.g. error handling).
Approach 2: Message Passing
• Actors
• Messages
SCALES WELL. RELATIVELY SIMPLE (once you learn it).
![Page 18: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/18.jpg)
18
Actor Model
![Page 19: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/19.jpg)
19
Actor Model
![Page 20: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/20.jpg)
20
Actor Model
Languages
• Erlang
• Scala
Frameworks
• Akka (Java/Scala)
• Kilim (Java)
• Pulsar (Clojure, Python)
• …
Erlang Hall of Fame:
• Availability: 99.9999999%
![Page 21: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/21.jpg)
21
Databases
![Page 22: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/22.jpg)
22
Background
Limited data
Localized data
Homogeneous requirements
RDBMS have matched these requirements
![Page 23: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/23.jpg)
23
The Rise of NoSQL
ACID
• Atomic
• Consistent
• Isolated
• Durable
vs
BASE
• Basic Availability
• Soft-state
• Eventual Consistency
![Page 24: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/24.jpg)
24
Requirements Have Changed
Mass scale (Big Data)
Distributed Data Storage
Different varieties of data
Incompatible requirements
RDBMS are no longer suitable to handle all use cases
![Page 25: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/25.jpg)
25
Many Databases
![Page 26: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/26.jpg)
26
NewSQL strikes back
Eventual Consistency complicates life a lot
"We also have a lot of experience with eventual consistency systems at
Google. In all such systems, we find developers spend a significant
fraction of their time building extremely complex and error-prone
mechanisms to cope with eventual consistency and handle data that
may be out of date.“
MIT Prof. Michael Stonebraker: “The Traditional RDBMS Wisdom is
All Wrong”
• Most OLTP systems can afford to buy the amount of memory needed to keep
data off the disk.
• Single-threading gets rid of the latching overhead.
• OLAP: Column Stores are 50-100 times faster
• …
![Page 27: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/27.jpg)
27
Memory/Network/Disk speed
![Page 28: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/28.jpg)
28
Separate CPU/Memory/Storage Racks
140 TB
![Page 29: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/29.jpg)
29
So Who is Right? NoSQL or NewSQL?
Well… Both!
Polyglot Persistence
![Page 30: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/30.jpg)
30
“Software Defined Datacenter”
20.11.2013 г. 30
![Page 31: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/31.jpg)
31
Storage
![Page 32: HackFMI Talk #3 - Interesting Times: 6 Current Paradigm Shifts in IT - Teodor Tonchev](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a5ddfb4c90531228b538b/html5/thumbnails/32.jpg)
32
Interesting Times
Application/OS Deployment
• Deploy directly on the hardware
• New business -- buy hardware
Application Development
• Pet Apps (Fixed Scale)
• Sequential Programming + Parallel
Programming with Shared Memory
Databases
• Local fast disks
• RDBMS or NoSQL
Data centers
• Specialized hardware
• Virtualize the hardware
• Use a service
• Cattle Apps (Dynamic Scale)
• Parallel Programming with
Message Passing
• In-memory, distributed
• Polyglot Persistence
(NoSQL,NewSQL,etc)
• Cheap hardware + software