vmugit uc 2013 - 05a vmware vfabric
TRANSCRIPT
© 2009 VMware Inc. All rights reserved
VMware Cloud Application Platform
Filippo Lambiente,
Sr. Systems Engineer, vFabric, Cloud Application Platform
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
2
Traditional Applications – a sample
VMware Confidential
Java EE My-Application.war
RDBMS
Front-end
Order
Inventory
Shipping
§ Lack of scalability
• Scale through replication
• Can’t scale different parts of the application differently
§ Lack of deployability
• Deploy it all in one go, lots of communication & coordination
• Increased risk of something breaking
§ Applications are brittle
• Store can’t accept orders unless all services are available
• Failure (e.g. memory leak) in one component can take down every other
§ Vendor Lock-in
• Long-term commitment to technology stack
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
3
The Database is Being Stretched
Big Data Flexible Data § Petabytes vs.
Gigabytes § Democratize BI
§ Multi-structured data § Developer productivity
Fast Data § Low latency expectations § Horizontal scale
Cloud Delivery § Virtualized
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
4
Trends Driving Change in IT
VMware Confidential
Cloud Delivery • Offered “as-a-Service” • Virtualization
Developer Productivity New application frameworks driving increase in application development
New Application Types • Mobile, SaaS, social • Apps released early and often
Data Volume • Web orientation drives exponential
data volumes • Reduced latency and new types of data
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
5
Modern Application
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
6
VMware vFabric: Build, Run, Manage Modern Application
VMware Confidential
Virtual Datacenter Cloud Infrastructure and Management
Rich Web
Programming Model
Social and Mobile
Data Access
Integration Patterns
Batch Framework
WaveMaker Spring Tool Suite
Cloud Foundry
App Monitoring (Spring Insight)
Performance Mgmt (Hyperic)
Automated App Provisioning (AppDirector)
Java Optimizations (EM4J, …)
Java Runtime (tc Server)
Web Runtime (ERS)
Messaging (RabbitMQ)
Global Data (GemFire)
In-mem SQL (SQLFire)
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
7
Build Spring
VMware Confidential PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
8
Productive Programming Model Across Wide Range of Apps
Web
Batch
Integration
Security Social Media
Cloud Platform Services
Mobile
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
9
Spring Integration
§ Spring Integration provides an extension of the Spring programming model to support the well-known enterprise integration patterns
§ Light-weight messaging framework
9 PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
10
Spring Data
10
Data bottlenecks require to change
Proliferation of «Solutions»
How to choose?
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
11
Spring Data
11
Traditional Architecture
Complex Heterogeneous
Database Architecture
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
12
Spring Data
Provide a familiar and consistent Spring based programming model while retaining store-specific features and capabilities
12 PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
13
Spring Web and Spring Mobile
13 PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
14
Run tcServer
VMware Confidential PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
15
Enough is Enough
Too many clients spend far too much time and effort trying to find the products with the most features. Lean shops look for just enough, no more. “ ”
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
16
tc Server: Replace Legacy Java Servers and Apache Tomcat
Efficient, lean, fit-to-purpose runtime platform Lower cost and complexity
Enterprise capabilities on Apache Tomcat-compatible base
vmware.com/go/tc PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
17
vFabric tc Server: Key Highlights
Developer Efficiency
Familiar Spring + Tomcat experience Deep performance insight into Spring apps Agile Spring development experience via STS Enhanced build process via Maven plugin
Operational Control
Performance & SLA management of Spring apps Application provisioning and server administration Rich alert definition, workflows, and control actions Group availability & event dashboards Secure unidirectional agent communications
Deployment Flexibility
Lean server (10 MB) ideal for virtual environments Template-driven server instance creation Integrated experience with VMware environments Open, secure API for all operations
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
18
Imagine a Java app taking customer orders
Guest OS
Java App
vSphere Host
!
!
!
?
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
19
Spring Insight
Guest OS
Java App
vSphere Host
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
20
Which app is slow?
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
21
Which parts of my app is slow?
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
22
Is the problem isolated to one server?
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
23
Are problems affecting all users, or just some?
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
24
Is the database a bottleneck?
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
25
Which parts of code are causing latency?
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
26
Operational Control: Production Deployment Architecture
Web Dashboard
Hyperic Monitoring Web Servers, App Servers, Databases, Caching, Messaging, Directories, Virtualization, etc.
Server 1
Agent tc Server Instance 1
tc Server Instance 1
tc Server Instance 1
Server 1
Agent tc Server Instance 1
Spring
tc Server Instance 1
Spring
tc Server Instance 1
Spring
Server 2
Agent tc Server Instance 1
tc Server Instance 1
tc Server Instance 1
Open API
Management Server Administration, Provisioning, Groups, Metrics, Alerts, Events, Access Control, Agent Upgrades, etc.
Inventory, Metric,
Audit, …
Spring
Spring
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
27
Operational Control: Application Provisioning
" Provision applications to single server or group of servers " Manage deployed applications
" View application status " Start, stop, reload, and undeploy applications
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
28
Operational Control: Server Administration & Configuration
" Centralized administration of server configuration and defaults " Out of box pre-tuned configurations
" JVM Options, production configurations, automated boot scripts, Windows service wrapper " Server Instance Templates
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
29
Operational Control: Alert Workflows and Control Actions
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
30
Use ESX to Share Memory Between VMs Running Java Elastic Memory for Java (EM4J)
Description
• Higher consolidation through memory over-commit for Java workloads
• Protect Java applications against workload spikes without wasting memory
• Lets you give Java more memory than it needs under normal load. Memory your application doesn’t need is returned to ESX but remains available in case it is needed to avoid Java out-of-memory errors
• Use virtualization to overcome the limitation of Java’s static heap
• New ‘memory balloon’ runs inside JVM to capture unused memory and return it to ESX
• Memory returned to JVM when it is under pressure
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
31
Run RabbitMQ
VMware Confidential PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
32
Modern Messaging for the Cloud
Cloud-scale Message Bus RabbitMQ Message Broker
AMQP, HTTP, HTTPS, STOMP, SMTP, XMPP …
Point -to-point and pub-sub
Virtual hosts, dynamic configuration
Cloud Services Devices Applications
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
33
C# C Hundreds of other clients
RabbitMQ runs practically everywhere
Integrate apps built on different technologies, running on different stacks
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
34
Run SQLFire
VMware Confidential PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
35
What’s the Problem?
How do you scale
the data tier?
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
36
vFabric SQLFire in a Nutshell
Databases Other Data Systems File Systems
Conventional Data Storage Systems
vFabric SQLFire Data Fabric
High Throughput Low Latency High Scalability Continuous Availability
Reliable Event Notification Shared Nothing SQL Parallel Execution
WAN Distribution
Enterprise Data Consuming Applications
Data Durability
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
37
CREATE TABLE FLIGHTS ( FLIGHT_ID CHAR(6) NOT NULL, SEGMENT_NUMBER INTEGER
NOT NULL, ORIG_AIRPORT CHAR(3), DEST_AIRPORT CHAR(3) DEPART_TIME TIME, FLIGHT_MILES INTEGER NOT
NULL) PARTITION BY
COLUMN(FLIGHT_ID) REDUNDANCY 1;
Partitioning/Replication/Co-Location/Redundancy of Tables
CREATE TABLE FLIGHTAVAILABILITY ( FLIGHT_AVAILABILITY_ID CHAR(6)
NOT NULL …. Foreign key FLIGHT References Flights
(FLIGHT_ID) PARTITION BY COLUMN (FLIGHT_ID) COLOCATE WITH (FLIGHTS);
CREATE TABLE AIRLINES ( AIRLINE CHAR(2) NOT NULL
PRIMARY KEY, AIRLINE_FULL VARCHAR(24), BASIC_RATE DOUBLE PRECISION, DISTANCE_DISCOUNT DOUBLE
PRECISION,…. ) REPLICATE;
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
38
CREATE TABLE AIRLINES ( AIRLINE CHAR(2) NOT NULL PRIMARY KEY,
AIRLINE_FULL VARCHAR(24), BASIC_RATE DOUBLE PRECISION, DISTANCE_DISCOUNT DOUBLE PRECISION,…. ) REPLICATE;
Replicated Tables
§ Tables that have static data, such as code tables, lookup tables, or metadata, are typically replicated
§ In this example, Airlines is replicated to three servers that are in the distributed system
SQLFire Server 1
SQLFire Server 2
SQLFire Server 3
Airlines Airlines Airlines
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
39
CREATE TABLE FLIGHTS ( FLIGHT_ID CHAR(6) NOT NULL,
SEGMENT_NUMBER INTEGER NOT NULL, ORIG_AIRPORT CHAR(3), DEST_AIRPORT CHAR(3), DEPART_TIME TIME, FLIGHT_MILES INTEGER NOT NULL)
PARTITION BY COLUMN(FLIGHT_ID);
Partitioning Tables
SQLFire Server 1
SQLFire Server 2
SQLFire Server 3
Airlines Airlines Airlines
Several partitioning options, you can partition by any of the following § PARTITION BY PRIMARY KEY § PARTITION BY RANGE (FLIGHT_MILES) (VALUES BETWEEN 0 AND 100, VALUES BETWEEN 100 AND 500, VALUES BETWEEN 500 AND 1000); § PARTITION BY LIST (ORIG_AIRPORT) (VALUES (‘PDX’, ‘LAX’) VALUES (‘AMS’, ‘DUB’)); § PARTITION BY
(HOUR(DEPART_TIME);
Flights Flights Flights
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
40
Redundancy
§ Create redundant partitions on other SQLFire members
§ SQLFire attempts to place copies of the same bucket onto hosts with different IP addresses to protect against machine failure
§ Setting the enforce-unique-host boot property prevents SQLFire from ever placing multiple copies on the same machine
Airlines Airlines Airlines
Flights Flights Flights
Airlines
Flights R
Airlines
Flights R
Airlines
Flights R
CREATE TABLE FLIGHTS ( FLIGHT_ID CHAR(6) NOT NULL,
SEGMENT_NUMBER INTEGER NOT NULL, ORIG_AIRPORT CHAR(3), DEST_AIRPORT CHAR(3), DEPART_TIME TIME, FLIGHT_MILES INTEGER NOT NULL)
PARTITION BY COLUMN(FLIGHT_ID) REDUNDANCY 1;
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
41
CREATE TABLE FLIGHTAVAILABILITY ( FLIGHT_ID CHAR(6) NOT NULL, SEGMENT_NUMBER INTEGER NOT NULL, FLIGHT_DATE DATE NOT NULL, ECONOMY_SEATS_TAKEN INTEGER DEFAULT 0, …) PARTITION BY COLUMN (FLIGHT_ID) COLOCATE WITH (FLIGHTS);
Partitioning Co-Location Tables
§ What if there is a parent child relationship between FlightAvailability and Flights tables?
Airlines Airlines Airlines
Flights Flights Flights
Airlines
Flights R
Airlines
Flights R
Airlines
Flights R
FlightAvailability FlightAvailability FlightAvailability FlightAvailabilityR FlightAvailabilityR FlightAvailabilityR
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
42
Disk Persistence
§ Additional redundant backups § Speed up recovery of data from disk versus loading from RDBMS
Airlines Airlines Airlines
Flights Flights Flights
Airlines
Flights R
Airlines
Flights R
Airlines
Flights R
FlightAvailability FlightAvailability FlightAvailability FlightAvailabilityR FlightAvailabilityR FlightAvailabilityR
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
43
Manage Application Director
VMware Confidential PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
44
What is Application Director?
An application provisioning solution that provides deployment automation and self-service provisioning
OS • Windows • Red Hat • CentOS
Custom or Packaged App binaries, config .war, .jar, .tar, .zip etc
Middleware App servers, messaging, web servers, databases, load balancers, etc
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
45
Manage Hyperic
VMware Confidential PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
46
If You Can Measure It, You Can Manage It
Identify, Organize
KPMs
Establish Baselines
Define Alerts,
Automate Response
Define Escalation Workflows
Learn from Data
Identify, Organize
KPMs
Establish Baselines
Define Alerts,
Automate Response
Define Escalation Workflows
Learn from Data
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
47
How do you keep your monitored system inventory current?
Hyperic Auto-Discovery • Continuously discovers new and modified ESX Hosts, Guests OSs, and Physical Servers.
• Automatically adds to and updates inventory. Automatically adjusts monitoring.
• Can use to trigger alerts.
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
48
Application Frameworks
Application Servers
Web Servers / Caches
Messaging Middleware
Databases
Directory Services
Email Servers
Operating Systems
Virtualization
Hyperic collects a broad set of application performance data…
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
49
Establish Baselines and Automate Anomaly Detection
Hyperic SmartTrend • Automatic baselining for any period. • Set alerts • Analyze trends • Measure SLA compliance
Out of Bounds Metrics Identified and Bubble-up to Dashboard
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor
50
Learn more at www.vmware.com
PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor