Next Enterprise Application Architecture underConvergence of Cloud, Mobile, Social and Big Data

KOSTA 2014 IT Strategy Seminar

Professor June Sung Park, KAIST

http://flavors.me/june_sung_park

Copyright © 2014. Dr. June Sung Park.

2

I. Overview of IT Trends

II. Impact of Mobile Cloud on Enterprise Application Architecture

III. Impact of Social and Big Data on Enterprise Application Architecture

IV. New Principles of Enterprise Application Architecture

1. Mobile Application Architecture for Front-End

2. SOA-based Cloud Service Architecture for Back-End

3. Pace-Layered Architecture for Integrity-based Agility

4. Event-Driven Architecture for Real-Time Intelligent Process

This talk is a sequel to my previous talk on “Mobile Cloud Architecture” given in the 2013 Korea Software Architect Conference.

Agenda

1

3

1. Evolution of Enterprise IT

2. Evolution of Web

3. Current Trends of Enterprise IT

4. Impact of New IT Trends on Enterprise Architecture

I. OVERVIEW OF IT TRENDS

2

Evolution of Enterprise IT

2010s2000s1980s1970s1960s1950s 1990s

EA-B

ase

dP

roce

ss In

tegr

atio

n

Pro

cess

-Orc

he

stra

ted

C

lou

d S

erv

ice

s

OnlineComputing(1970-80)

Client/ServerComputing(1990-1995)

E-Business(1995-2005)

ITSt

and

ard

izat

ion

IT M

od

ern

izat

ion

SOA

-Bas

ed

Pro

cess

Orc

he

stra

tio

n

Mo

bile

+ S

oci

al +

C

lou

d +

Big

Dat

a

IT Dark Age(1980-90)

Pro

cess

Re

en

gin

ee

rin

g

Mobile CloudComputing(2010-2015)

Technical Debt Payoff(2005-2010)

BatchComputing(1950-1970)

3

2003-20101991-1993 2011-20151994-2002

Amazon opens e-tail shop.

(1994)

AOL bought Time Warner.

(2000)

WebVan fails losing $1B.

(2001)

Amazon launches AWS

(2003)

E-business ends by 2008.

(Gartner 2000)

SOA prevails by 2008

(Gartner 2003)

By 2012 80% of Fortune

1000 use cloud services

(Gartner 2009)

Evolution of Web

4

Built and run on

Drive use of

Store and process

Generate

Mobile Application

Big DataCloud

Social

Current Trends of Enterprise IT

New business opportunities and use cases of IT made possible by the nexus of cloud, mobile, social and big data require a fundamental transformation of enterprise architecture in all layers: business, application, data and technical infrastructure.

In this talk, we discuss:

• How should the enterprise application architecture be transformed to enable the business to take the bestadvantage of new IT trends?

• What are the new architecture principles that should be applied in designing business applications today?

Applications created in 2013 using traditional architecture models will be IT-constraining legacy by 2016 [Gartner]

Impact of New IT Trends on Enterprise Architecture

5

8

1. Mobile Cloud Services

2. Impact of Mobile Cloud Services on EA

II. IMPACT OF MOBILE CLOUD ON EAA

6

Mobile Cloud Services

Mobile cloud services change the way business people work and the enterprise architecture has to be changed to support the new way of working.

• Business applications are always on (24X7), and accessed intermittently.

• Applications are personalized and contextualized.

• IT needs to support real-time decisions in response to real-time events (including social and machine-generated events)

• Applications are exposed to hacking.

• Enterprise apps are a part of global-class computing, where IT is expected to be a profit center.

• UI must be easy to learn and easy to use with any device. (IT Consumerization)

• Interactions between clients and cloud back-end should be less chatty, stateless, cached and queued.

• Applications are controlled by outsiders and continually changed.

By 2015, mobile app dev projects targeting smartphones and tables will outnumber PC projects by 4:1. [Gartner]

In 2013, 76% of all mobile traffic is for using clouds. [Cisco]

In 2013, 75% of mobile cloud market is represented by enterprise users. (Juniper Research]

As is true of all profound IT changes, the apparent revolution of cloud computing is, in fact, a synergetic aggregation of evolutions in multiple, well-established technology directions.

Service-oriented architecture enabled the advent of cloud services; viz. cloud service architecture is the state-of-art version of SOA.

Cloud service architecture is the best option today for all application back-end. This is a globally common ubiquitous requirement!

• Whether the application is used only inside the enterprise, or used for B2B, or connected to other cloud services, or offered as a cloud service to the market.

Mobile application architecture should be adopted for the front-end of enterprise apps.

• Even when you develop desktop apps because most of its architectural principles still hold.

Impact of Mobile Cloud Services on EA

7

11

1. Social Business

2. Big Data Analytics

III. IMPACT OF SOCIAL AND BIG DATA ON EAA

8

Social Business

The importance of social business to enterprises is growing, and the line between conventional business and social business is diminishing.

• Social Operation: Social media connects employees to each other, and to information, contents and applications they need to get work done. Social BPM integrates social activities, group workflows and social events with formal business processes.

• Social Marketing: Social media help personalized and context-sensitive interactions, conversation and engagement with customers.

• Social Innovation: Social media creates innovation funnel that allows discovering new solutions and expertise faster and in a serendipitous way.

• Social Analytics: Social media generates a variety of situation-specific data, which often contain social context, sentiment and the pattern of interactions

Social software mostly provide REST APIs so that it can be mashed up in the front-end of mobile and web applications, and/or Web Service APIs so that it can be integrated in the back-end.

OpenSocial is a public specification for social software APIs that allows access to data and functionality of social networks.

9

Big Data Analytics

Big data means structured and unstructured, static and streaming data of large volumes on the order of petabytes (1015) which relational database and data warehouse technologies cannot process fast enough.

A disk drive that can store all the music in the world (a few TBs) is only $600. However, it takes an average of 2.5 hours to read 1TB (1012).

Hadoop Distributed File System (HDFS) and MapReduce opened new possibilities, by supporting scale-out with low-cost commodity hardware, to store and fast process large files (> 100 TB).

Big data could be data accumulated in databases and data warehouses, data generated from web (a.k.a. digital footprints or data exhaust), and data created by sensors and machines.

What makes big data important is the insights we can gain from it through analytics.

Big data storage, processing and analytics services are available from the cloud (e.g., Amazon Elastic MapReduce and DynamoDB, Google BigQuery, Microsoft Azure HDInsight).

From the dawn of civilization until 2003, humankind generated 5 exabytes of data (1018). Now we produce 5 exabytes every two days, and the pace is accelerating.

30 billion pieces of content are shared monthly on Facebook.

Entire data on Internet is about 1 zetabytes (1021).

A GE jet engine produces 20 TB of data in every operating hour.

Sears took 8 weeks to generate personalized promotions using databases and data warehouses containing huge amounts of data collected from its stores. Using a Hadoop cluster the processing time was reduced to 1 week and still dropping. 10

14

1. Mobile Application Architecture for Front-End

2. SOA-based Cloud Service Architecture for Back-End

3. Pace-Layered Architecture for Integrity-based Agility

4. Event-Driven Architecture for Real-Time Intelligent Process

IV. NEW PRINCIPLES OF ENTERPRISE APP ARCHITECTURE

11

Mobile Application Architecture

Simple UI—visual, tactile, aural, vocal, motion

Mix of native code and Web view for UI

Multi-channel delivery based on service-oriented APIs

Dynamic composition of REST services

Cloud access for both users and developers

Context-awareness via sensors.

Personalization—situational, emotional and preferential(with the latter requiring predictive analytics)

Stateful (so that clouds can be stateless)

Bandwidth saving through low chattiness, queues, offloading

Hub for wearables and connectables

In 2013, 41% of developers use PaaS to develop and deploy mobile applications. [TechTarget]

Mobile app developers apply super-agile dev methods, based on open source SaaS, PaaS (e.g. CloudFoundry, OpenShift), dPaaS (e.g. CloudBees), BaaS (e.g. OpneMEAP) and IaaS (e.g. OpenStack).

Imagine how Instagram added 1M new customers in 12 hours, and IMVU deploys new code 50 times a day.

Cloud Services

User Interface

Business Process

BPaaS

SaaS

PaaS

IaaS

Virtualized Infrastructure

AP

I

Common Cloud Mgmt Platform

OSS BSS

Service Portal

API

SOA

Cloud Service Architecture

Service-oriented

Industrially-packaged and public access ready (accessible via both UI and headless APIs)

Multitenant

Automated for self-service

Elastic and scalable

Monitored and metered

Continuously versionable

Composed and composable

Stateless and hoard-less

Customization via polymorphism

12

SOA supports the design principles of encapsulation, loose coupling and separation of concerns.

Services in SOA are modular, distributable, discoverable, replaceable, shareable and composable.

BPM is used to orchestrate services and to build composite applications. Use BPMN and BPEL for BPM.

Metadata management is a prerequisite for SOA.

EA

TA

DA

AA

BA

Service-Oriented Architecture: Prerequisite for Cloud

13

SOA: Separation of Concerns

User Interface

Process Orchestration and Collaboration Layer

Core Service Layer with Principal Data Stores

Utility Service Layer

Underlying Service Layer

Transaction Service

Analytics Service

Data Service

Application(Firm-Specific)

Framework(Domain-Specific)

Platform(Horizontal)

14

Functional Cohesion and Loose Coupling

Encapsulation

Commonality

Reusability

Commoditization

Use Case Diagram

Use Case Scenario

Sequence Diagram

User Story

UX

Class Diagram

Process Model

Data Schema

Component Diagram

Business Service Spec

Service Interface Diagram

Service Architecture Diagram

Affinity Clustering

SOA: Core Services Identification

15

Be open to consume and provide global services.

Yet control your service ecosystem using omni-channel delivery, process orchestration and collaboration, API management, SOA governance, service AppStore, and metadata management.

SOA: Global-Class Computing

Many DataSource

ManyServices

ManyProcesses

ManyClients

One ServiceEcosystem

18

SOA: Middle-Out Architecture

Mobile Devices, Wireless Sensors, 4G/5G, Social Networks, SaaS, PaaS, IaaS, Hadoop,In-Memory Database, Data Science, Software-Defined Network, M2M, IoT,…

StableBusiness Services

and Metadata

EmergingTechnologies

BusinessInnovations

17

Pace-Layered Architecture

Apps move across pace-layers as they mature.

IT cannot succeed with exploitative apps, and much less explorative apps, on an unstable foundation.

Both exploitative and explorative apps must be developed based on core processes and data in the stable foundation.

IT cannot skip maturity levels.

Stable Digital Foundation- Largely constant

Exploitative Apps- Controlled Change

Explorative Apps- Free to Change

Open Global-Class ComputingService EcosystemBig Data AnalyticsEvent-Driven ArchitectureEnterprise MobilityCloud ServicesSocial BusinessAdaptive Case ManagementInfrastructure VirtualizationBusiness Process ManagementService-Oriented ArchitectureMetadata ManagementEnterprise ArchitectureInfrastructure Consolidation

DisruptiveInnovation

IT-EnabledBusiness Transformation

Technical-Debt Payoff

16

Analytics Architecture

Link BI and big data analytics projects with strategic initiatives.

Align their output with corporate KPIs.

Have business people, not IT people, determine their use cases.

Implement them within business processes.

Consolidate silo data into a single version of standardized and cleansed data.

Choose platforms allowing polyglot persistence.

Pick tools easy for rapid deployment and easy for users.

Measure the value created from the projects.

Through 2015, 85% of Fortune 500 will be unable to exploit big data for competitive advantage. [Gartner]

Extract, Cleansing, Transformation, Loadinge.g. MapReduce, Pig, Sqoop, Kettle

File Systeme.g. HDFS

DWe.g. Hive

Stream Processinge.g. Flume, Scribe

Data Sourcee.g. RDB, Doc, CSV

Analytics Common Services

RT-OLAPe.g. BigQuery

Analytic Algorithme.g. Rhadoop, Mahout

Visualizatione.g. Pentaho

Data Modelse.g. Relational, Multidimensional, Key Value, Column Family, Document, Graph

Analytics Apps

Business Use Cases

NoSQLe.g. Hbase, Redis, Neo4j

MongoDB, Cassandra

Data Sourcee.g. Web, Social

Data Sourcee.g. Sensor, Machine

19

Event-Driven Architecture for Real-Time Intelligent Process

Maturity of enterprise architecture usually advances in steps:

• IT infrastructure consolidation and standardization Process and data standardization and integration Process orchestration based on BPM and SOA Real-time event processing Intelligent process with analytics embedded

Real-Time Event Processing:

• Event signal Publish and subscribe Process trigger Event-driven service

20

Event-Driven Architecture for Real-Time Intelligent Process

Social network, sensor network and analytics are integrated into BPM for real-time situation recognition and immediate action recommendation.

Integration of real-time analytics into operational processes—which contrasts with past approaches that separated analytical work from transactional work—empowers the workforce to make better and faster contextualized decisions in order to guide work toward optimal outcome.

Aquire andSense

Organize

Decide

Act Analyze

21

Event-Driven Architecture for Real-Time Intelligent Process

Process is the unifying construct for intelligent operations.

Integration of BPM and automated analytics into SOA-based iBPM is an important business evolution underway.

Using Talend open source solutions, you can load data from multiple sources into a master data hub as a SoR, apply the data quality tool to resolve data conflicts, and provide clean data services (including Hadoop services) for automated decisions in business processes or for business workers whose workflow is orchestrated by BPM.

22

Thank You!

KOSTA 2014 IT Strategy Seminar

Professor June Sung Park, KAIST

http://flavors.me/june_sung_park

Copyright © 2014. Dr. June Sung Park.