a case of fusion middleware (iloug tech days, july 2013)
DESCRIPTION
NOPERU is the organization responsible for handing out permits that Corporations, Governments and Citizens need for activities that involve emissions -such as CO2 - or resource usage - such as energy or water. NOPERU works with 100s users from five locations. A transition is taking place from decentralized data, Oracle Forms based application silos, paper based documents, data oriented, outdated character based user interfaces to a future steeped in Fusion Middleware, with modern pages (ADF), a serviced oriented architecture with decoupling and reuse (SOA Suite and OSB), a unified database (Oracle 11gR2) and digital documents (WebCenter Content). Then a business process approach (BPM Suite & BAM) that aligns with how business really works and provides operational insight to managers. This presentation tells the story of the transition, functional, architectural and technological aspects.TRANSCRIPT
Lucas Jellema (AMIS – The Netherlands)
IlOUG – Tech Days, Jerusalem, 10 & 11 July 2013
A Case of Fusion Middleware
Overview
• Introducing NOPERU
• Business Objectives
• IT goals
• Architecture
• Technology selection
• IT organization 2.0
• Project roadmap
• Learning experience and conclusions
3
Emissions and Resource Usage
Introducing NOPERU
• National Organization for Permits for Emission and Resource
Usage
• Commercial enterprise, government agency or a private person
requires permit for non-trivial activities (‘fair use’)
• Permit application processing takes from three weeks up to two
years, including appeals
• The process can end with:
– A permit, free of charge
– A permit provided some for of fee or tax is paid
– Rejection
• During the process, NOPERU may require an applicant to
provide additional information
Introducing NOPERU
• NOPERU has five regional branches with three departments each: commercial, governmental, citizens
• Interactions with NOPERU are on paper or through fax and telephone; responses are only given during office hours
commercial
citizen
government
commercial
citizen
government
commercial
citizen
government
commercial
citizen
government
commercial
citizen
government
Introducing NOPERU
• NOPERU has decentralized computer facilities: local application &
database instances with on-site staff
• NOPERU’s applications were developed in the early 90s and have
undergone several upgrades
– Some are Oracle Forms, all run on Oracle Database (11g)
• Each sector has a very similar system for the permit application process
– they all stem from the same base application
– then after cloning were independently developed
• The applications are unattractive and have a steep learning curve
– They provide a window on data – not a wizard style, task driven UI
7
Business ObjectivesGo Forward 2010-2018
• Faster processing of permit requests• 24/7 availability (the shop is always open for customers)• Multi-channel (web portal, mobile, …)• Ban on paper – everything digital • Self-service (type in yourself, verify yourself, lookup yourself, …)• Flexible change process with short time to market• Cost reduction• Improve Data Quality• Shorter learning curve for new employees• Automate as much of the permit application process as possible
8
The Road Ahead
Paperless
24/7
Self-service
Multi Channel
Agile
Process Driven
Data Quality
Cost Reduction
9
IT Objectives
• Work under ‘enterprise architecture’: structured, planned, controlled• Consolidation
– Data– Applications– Infrastructure
• Leverage industry standards and open technology components– Use standard applications (COTS) whenever possible (buy before build)– Never build generic, non-business specific components
• Work with a small number of strategic vendors (that take responsibility)• Setup Master Data and Information Lifecycle Management
– Data integrity– Archival procedures
10
Architecture
• Enterprise Architecture and High-level Information Architecture design• Mid-term architecture roadmap• Reference Architecture
– Patterns, decisions, guidelines,
• Domains: – Documents, – Finance, – Permits & Applications, – CRM, – Expertise (Legal, Scientific,…)
CRM Finance
Expertise Documents
Permits & Applications
11
Layered Architecture
Data
Business
Interface
12
Focused Layered Architecture
Data
Business
Interface• Modern• Volatile• Customized• Variety• Visible
• Reuse• Consolidation• Slowly
evolving• Long lasting
13
Decentralized Silos to central, consolidated Layers
14
Decentralized Silos to central, consolidated Layers (2)
15
Data
Business
Service Oriented Architecture
Interface
CRM Expertise Docs Finance P&A
16
Data
Business
Service Oriented Architecture
Interface
CRM Expertise Docs Finance P&A
elementary service
elementary service
elementary service
elementary service
elementary service
17
Data
Business
Service Oriented Architecture
Interface
CRM Expertise Docs Finance P&A
elementary service
elementary service
elementary service
elementary service
elementary service
18
Data
Business
Service Oriented Architecture
Interface
CRM Expertise Docs Finance P&A
business process
presentation service
presentation service
composite service
composite service
elementary service
elementary service
elementary service
elementary service
elementary service
Utility service
19
Data
Business
Service Oriented Architecture
Interface
CRM Expertise Docs Finance P&A
business process
presentation service
presentation service
composite service
composite service
elementary service
elementary service
elementary service
elementary service
elementary service
Utility service
Decision Svc
20
Business
Data
Central Event Facility
Events for decoupled interaction
Interface
CRM Expertise Docs Finance P&A
business process
presentation service
presentation service
composite service
composite service
elementary service
elementary service
elementary service
elementary service
elementary service
Utility service
Decision Svc
Content Repository
EmailEmail
Service Bus
DWHDWH LDAPLDAP
Bulk Data Integration
BPM & Workflow
Internal UI ApplicationsInternal UI
Applications
PortalPortalWeb
ApplicationsWeb
Applications
BAMBAM
External Web
Services
Data
Integration
Process orchestration
Iden
tity
Man
agem
ent &
Sec
urity
Citizens, Companies, Government, Agencies
Internal StaffB2B
MobileMobile
User InterfaceApplication Interface
Web Sites & Content
Web Sites & Content
Business RuleService
Orchestration
Technology Adapters
Content & Document
management
Adm
inis
trat
ion,
moi
nto
ring
, co
nfig
ura
tion
CRMCRMKnowledgeKnowledge P&AP&A Finance
Required components
Event Handler
22
Vendor & Technology SelectionVendor Requirements
• Vendor and Technology/Product are a package deal: you select one and the other
• Small number of strategic, committed vendors– Ideally taking responsibility for results
• Proven technology with verifiable references• Products that are strategic to the vendor• Vendors with enough substance• Products that are open and support industry standards
– And run on common platform
• Products with substantial community support– Partners, specialists, books and educational resources, active forums, social media
activity, user groups and conferences
23
Vendor & Technology SelectionProcess
• Phase 1– RFI– Market Analysis– Peer investigation
• Screening => Short List• Phase 2
– Proposal and plan – what, how, (creative) conditions– Presentation of Customer References and Product strategy & roadmap– (Reference Visit)
• Selection• Negotations => Deal or No Deal
24
Mapping to vendor portfolio
Content Repository
EmailEmail
Service Bus:Oracle Service
Bus
DWHDWH LDAPLDAP
BPM & Workflow:SOA Suite & BPM Suite
Internal UI Applications:ADF & Forms
Internal UI Applications:ADF & Forms
PortalPortalWeb
ApplicationsWeb
Applications
BAM:Oracle BAM
BAM:Oracle BAM
External Web
Services
Data
Integration
Process orchestration
Iden
tity
Man
age
men
t &
Sec
urity
MobileMobile
User InterfaceApplication Interface
Web Sites & Content
Web Sites & Content
Business Rule:SOA Suite
Service Orchestration:SOA Suite (BPEL)
Technology Adapters:SOA Suite
Content & Document management:
WebCenter Content
Adm
inis
trat
ion,
moi
nto
ring
, co
nfig
ura
tion
CRMCRMKnowledgeKnowledge P&AP&A Finance
Bulk Data Integration:
ODI Event Handler:SOA Suite EDN
25
Open ends
• Portal and Mobile technology selection are kept separate– The decoupled architecture allows for this distinction
• Identity Management & Security is also deferred– The OPSS (Oracle Platform Security Services in WebLogic Server) make this a
viable option too
26
IT-Organization 2.0
• Agile Software Development– Frequent delivery– Continuous feedback to and prioritizing by business owner– One team with designers, developers, testers
27
Scrum board
28
Team organization: Focused on technology – specialize on skill
Data
Business
Interface
CRM Expertise Docs Finance P&A
business process
presentation service
presentation service
composite service
composite service
elementary service
elementary service
elementary service
elementary service
elementary service
Utility service
business process
business process
29Team organization: per domain/feature slide and one team cross domain
Data
Business
Interface
CRM Expertise Docs Finance P&A
business process
presentation service presentation
service
composite service
composite service
elementary service
elementary service
elementary service
elementary service
elementary service
Utility service
presentation service
business process
presentation service
business process
30
IT-Organization 2.0
• Training & Coaching program– Bring in experienced consultants to do and show how to do (enabling is key) – And to apply best practices and to avoid pitfalls– Set up Competence Centers (virtual expertise guilds)
• The Platform is crucial– Database and Middleware (Application Server, Enterprise Service Bus, Process
Engine, …)
• Quick time to market requires break down of walls between Development, Administration and Maintenance
• => DevOps
The “Platform”
Hardware
VM
O/S
Application Server
Database
Application
Application
Application
WebLogic & platform-components
WebLogic Server
SOA Suite & BPM FormServerWC Portlets
OBI EEOSB
SOA Composite
WS*, Java/JEE & ADF apps
Forms App
UCM
OSB Service
AdaptersJEE services++
Cache, Messaging, Timers, Work Managers, …
BPM Process
Activities and areas
Prepare Execute (run-time)
design build test monitor modify• urgent• planned
Application(functional)
Platform(non functional)
Infrastructure
• operational• analysis
Hardware
VM
O/S
Application Server
Database
Application
Activities and areasPrepare Execute (run-time)
design build test monitor modify• urgent• planned
Application(functional)
Platform(non functional)
Infrastructure
BPM & SOA Suite/Service Bus, BAM WebCenter Content (UCM) WebLogic Java EE Applicatie Server Database
Operating System (Virtual Machines) Network, Storage, Hardware
req
SLA
UI: ADF, HTML/JS, Java, BAM Process: BPM(N) Services: OSB & SOA Suite; BI Publisher Database: SQL & PL/SQL
QA, Gover-nance, ALM
• operational• analysis
Oracle Enterprise Manager- RUEI
- FMW Console- - WLS Console
- Grid Control
FMW Composers(dt @ rt)
- Page Composer- Process Composer
- SOA Composer- OSB Console
- WC Content Editor- Applicatie
voorzieningen
DevOps – to bring together
Prepare Execute (run-time)
design build test monitor modify• urgent• planned
Application(functional)
Platform(non functional)
Infrastructure
• BPM & SOA Suite/Service Bus, BAM• WebCenter Content (UCM)• WebLogic Java EE Applicatie Server• Database
• Operating System •(Virtual Machines)• Network, Storage, Hardware
req
SLA
• UI: ADF, HTML/JS, Java, BAM• Process: BPM(N)• Services: OSB & SOA Suite; BI Publisher • Database: SQL & PL/SQL
QA, Gover-nance, ALM
• operational• analysis
Oracle Enterprise Manager- RUEI
- FMW Console- - WLS Console
- Grid Control
FMW Composers(dt @ rt)
- Page Composer- Process Composer
- SOA Composer- OSB Console
- WC Content Editor- Applicatie
voorzieningen
DevOps
36
Automate software engineering
• Code quality control• Testing:
– Web Services: SoapUI (functional) , LoadUI (load & stress) – Java and ADF Business Components: jUnit (function & load) – Web applications: JMeter (functional and primarily load); Selenium (functional) ;
Oracle Application Testing Suite (under evaluation for both functional and load testing)
– Database: SoapUI, QCTO, DB Unit, SQL Developer
• Build: Ant, Maven• Deploy: Hudson• Incident Management and Scrum backlog management: Jira• Source Control: Subversion (Git?)• Collaboration & Knowledge Sharing: MediaWiki, Sharepoint, MS Lync (IM)
37
Project Roadmap
• B2B Corporate Sector – automated exchange; central db and replication with regional instances
• Digital Documents (central content server, internal application used from regional offices)
• UI for internal staff – sector Citizens – ADF, CRUD style (central DB)• Forms 11g upgrade (from Forms 4.5 Client Server); regional instances• Portal for Corporate clients• Mobile app for internal managers (developed by outsourcing party)
Future:• Standard Application for CRM• BPM for process approach (operational insight through dashboard,
process improvement, integrated document flow)
Learning experience & conclusions
• Consolidation has made a huge difference– Far less administrative effort, much faster upgrades/rollouts, easier
‘cross the board’ reporting
– Central infrastructure becomes a crucial element whose failure impacts thousands of users => special attention is required
• The transition is scary and overwhelming
– Guidance, reassurance, explanations, and almost spiritual support are absolutely necessary to motivate and enable staff in almost every role
• Communication is essential for the success of the kind of rapid evolution NOPERU is going through
• Database is [still] hugely important (for performance, integrity, development) – that is sometimes overlooked
Learning experience & conclusions
• Scrum has made a huge difference– Flexibility, team commitment, real time insight, business link
• SOA requires up-front investments– Reuse can only happen when there are reusable assets
– Bookkeeping & Project Accounting enters a new realm as well
• Governance of reusable assets should quickly be established across the teams
• Layered Architecture and Encapsulation ensures that teams can work in parallel, legacy can be wrapped and reused and local optimizations can continually be applied– Chunks of Work can be outsourced fairly easily
40
Data
Business
The glue between teams and components: common data model
Interface
CRM Expertise Docs Finance P&A
business process
presentation service
presentation service
Common Languagefor Interfaces, Messages and Events
composite service
composite service
elementary service
elementary service
elementary service
elementary service
elementary service
Utility service
Summary
• Many organization share similar objectives, predicaments and solutions
• Common requirements: – Multi-channel, flexible, 24/7, cost efficiency, lower risk, paperless– Consolidation, standardization, under architecture, decoupling
• Layered, decoupled architecture – upside down piramid• Common Technology: (for example from Oracle)
– Enterprise Service Bus, BPMN process engine, Digital Content Management, Enterprise Database
– Tools to quickly create modern UIs for various channels• Organizational changes:
– Agile software development approach – close cooperation between business and IT => “embrace change”
• Roadmap: frequent, small steps with business value