making workflows work prof. yike guo dept. of computing imperial college london inforsense...
TRANSCRIPT
Making Workflows Work
Prof. Yike Guo
Dept. of Computing Imperial College London
InforSense Limited
Proprietary and Confidential
DiscoveryNet Project Funding
One of the Eight UK National e-Science Projects (£2.2 M) Sept 2001 – March 2005
Partners
Achievements Constructing the World’s First Infrastructure for Building Analytical Services by
Scientists For the First time Discovery Net Realises the Dynamic Construction of Compositional
Services on GRID for Real Time Knowledge Discovery and Decision Making
Outputs Software Research: DNet platform commercialized by InforSense Ltd (>100
customers) Total user numbers > 2000 Applications Research: Application out puts in sensor technology commercialized by deltaDot Ltd Number papers published: 10 Journal Papers, 30 Conference Papers 8 PhD completed and 50 Master students Ranked OUTSTANDING at the project final review
Proprietary and Confidential
InforSense Introduction100+ customers (70% Fortune 200 companies)
2006 3rd fastest growing company in UK (Sunday Times Tech Track)
2007 8th fastest growing venture-based company in UK (Financial Times)
Global footprint with offices in London (HQ and R/D), Boston (USA HQ) and Shanghai (Asia HQ and Development base)
Global sales with 70% outside of Europe
7 years of delivering products and services to pharmaceutical and Financial industries
Spin out from Imperial College London
Invented “Distributed Data Mining ”
‘98 First Enterprise Deployment
Embedding Analytics Technology3rd fast growing company in UK
‘01
‘05
2004
,03
IEEE Super-computing Award –Grid based analytics
InforSense FormedIntroducedKDE Analytics Platform
Discovery Net Project
Embedding Analytics in Major Enterprise Systems
‘06
‘00
Innovation in Embedding Analytics
Proprietary and Confidential
CAMBRIABIOSCIENCES
Those who are using our workflow
Proprietary and Confidential Excel
EMR Database
s
OraclePre-processing
OraclePre-processing
3rd partyAnalytics
3rd partyAnalytics Web servicesWeb services Biomedical
Informatics tools
BiomedicalInformatics tools
Multiple data sources
Multiple data sources
Interactive Knowledge Discovery
Interactive Solution Building
Rapid Application Deployment
Portal / Dashboard
Application
InforSense Workflow Methodology
Files
Automation & Scheduling
Data
Applications
Components
InforSenseAnalytics
InforSenseAnalytics
Integrative Analytics Workflow Environment
Delivery to End User
Dynamic Data & App Integration
Business Process
Administrator Clinician Disease Biologist
Proprietary and Confidential
What is InforSense WF System Designed for ? InforSense workflow system is not an application but a
framework to build and deliver applications directly to scientist/business user:
Chem-Studio
ADMET Browser
Proprietary and Confidential
Pipelining
Web ServiceOrchestration
ETL
EnterpriseService Bus
Data/Text Mining
Business ProcessManagenment
Simulation& Modelling
InforSense Generic Workflow Engine
Proprietary and Confidential
Experience of 7 years in WF business
Building workflow is easy ! However,
Building a USABLE workflow is not easy Building a REUSABLE workflow is hard Building a REUSABLE workflow applications is
very hard Building a REUSABLE workflow application for
EVERYONE is very very hard Building a function is easy, building an application
is hard, it is even harder if we enable a non-IT person to build a good reliable application for other people to use everyday!
Proprietary and Confidential
InforSense Workflow System Development
Workflow ExecutionReliable Enterprise Wide
Execution
Workflow ManagementCollaborative Knowledge Management
Workflow Deployment:Building Reusable WF Applications
WorkflowWarehousing
Resource Mapping
Service Abstraction
Workflow AuthoringComposing services
Condor-GCondor-G
Native MPINative MPI OGSA-serviceOGSA-service
Web ServiceWeb Service
UnicoreUnicoreOralce 10g
Web WrapperWeb WrapperSun Grid Engine
Workflow EmbeddingPervasive WF applications
Proprietary and Confidential
Three Tiers of Workflow Framework
Building Layer
Application Layer
Embedding Layer
Analytical Workflow Development
Rapid Application Development
Service Orchestration
Business Rules
Embed in Other Applications
Analytic Service Encapsulation
Publish Services for Display
BPEL
InforSense Workflow Building:
Not about another graph notation but about how to build a meaningful graph
Proprietary and Confidential
Current model of workflow authoring/execution
No help provided to user (authoring/execution) Model is based on expert user who know about services Model requires user to be trained in a workflow language/system Interoperability between workflow systems is only at run-time
Proprietary and Confidential
The key the success : End User Oriented Workflow Construction
Build semi-automatic tools that advise/assist user in wf authoring
Make use of previous knowledge about developing workflows
Explicit/Expert knowledge Implicit knowledge in previous
workflows
The aim is to help user, not replace him
Proprietary and Confidential
Guided Workflow Construction
User is presented by high-level descriptions of predefined task steps
User is guided iteratively in instantiating the task descriptions using workflow templates
User can retrieve workflows and workflow templates from repository
Approach supports using workflows from multiple systems using existing run-time interoperability mechanisms
Proprietary and Confidential
Extended infrastructure:Workflow warehousing and mining
Workflow Advisor Initial implementations of prototype for bio
applications
Workflow Assistant Abstract component initial prototypes
Workflow Mining Repository of workflows from Southampton
Workflow Annotations independent from
workflow language
Warehouse Search and execute web services/Grid
services and workflows Syntactic and semantic search
Proprietary and Confidential
Extended infrastructure: Workflow warehouse/registry
InforSense Embedding and Deployment
Workflow output is not a data, but an application/service
Proprietary and Confidential
InforSense KDE Deployment Strategies
Deploy workflows to InforSense portal
Deployment features: multi-page, service chain, layout editor
Multi-stage applications: group workflows into stages
Component based deployment
Portlet based deployment
Portlet component: JSR 168 compatible portlet components
Business process workflow
Based on control flow orchestrated workflows and role based deployment
Proprietary and Confidential
Web-based Deployment
Portal Container allows users to
build dashboards
Each Workflow generate data for a
dashboard component
Workflow results viewed in simple charts - can be linked to other
pages
Proprietary and Confidential
Deployment Features (2)
Define multiple pages
Move to next page
Proprietary and Confidential
Chip QC Normalise Analyse Interpret
Design ExperimentDesign Study groups for transcriptomics portal
Gene Expression ProfilingPre-process and Analyse the results of a gene expression analysis to compare control vs. test populations
Splice Variance AnalysisPre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations
Results
Slide 1
Slide 2
Slide 3
Slide 4
Slide 5
Slide 6
QC Step
Recommended rerun
Chip must be rerun Normalisation services
•RMA (recommended)•LiWong•ETC
Next Steps
Submit to Report>
Example ApplicationAnalytical stage
Workflow configured to group according to stage Portal look and feel can
be customized by style sheet
Proprietary and Confidential
Chip QC Normalise Analyse Interpret
Design ExperimentDesign Study groups for transcriptomics portal
Gene Expression ProfilingPre-process and Analyse the results of a gene expression analysis to compare control vs. test populations
Splice Variance AnalysisPre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations
Results
Analysis services
•Volcano Plot (recommended)•PCA•Dendrogram
Next Steps
Submit to Report>
Example Application
Proprietary and Confidential
Chip QC Normalise Analyse Interpret
Design ExperimentDesign Study groups for transcriptomics portal
Gene Expression ProfilingPre-process and Analyse the results of a gene expression analysis to compare control vs. test populations
Splice Variance AnalysisPre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations
Results
Next Steps
Submit to Report>
Save Result to Report
Analysis services
•Select Transcripts
•Filter Data
Example Application
Proprietary and Confidential
Chip QC Normalise Analyse Interpret
Design ExperimentDesign Study groups for transcriptomics portal
Gene Expression ProfilingPre-process and Analyse the results of a gene expression analysis to compare control vs. test populations
Splice Variance AnalysisPre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations
Results
Next Steps
Submit to Report>
Save Selected Items to Report
Interpretation services
•Send Data to Ingenuity•Send Data to Gene Go •Send Data to•Text Analysis
Example Application
Proprietary and Confidential
Chip QC Normalise Analyse Interpret
Design ExperimentDesign Study groups for transcriptomics portal
Gene Expression ProfilingPre-process and Analyse the results of a gene expression analysis to compare control vs. test populations
Splice Variance AnalysisPre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations
Results
Next Steps
Submit to Report>
Interpretation services
•Send Data to Gene Go•Text Analysis
Related Pathway
Save to Report
Example Application
Proprietary and Confidential
Chip QC Normalise Analyse Interpret
Design ExperimentDesign Study groups for transcriptomics portal
Gene Expression ProfilingPre-process and Analyse the results of a gene expression analysis to compare control vs. test populations
Splice Variance AnalysisPre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations
Results
Next Steps
Submit to Report>
Interpretation services
•Send Data to Gene Go•Text Analysis
Related Pathway
Select Subset for Text Analysis
Example Application
Proprietary and Confidential
Business Process Management Development
A Business Process Management (BPM) describes the orchestration of different tasks to complete a specific business objective
Business Processes need to orchestrate
Automated Tasks
User Tasks
Exception Handling
Running Tasks in parallel
Synchronisation of parallel tasks
Business Process Workflow (1)
Proprietary and Confidential
InforSense Control Flow
InforSense Control Flow for Orchestrating Workflows for Business Process
Run
Task
Handle Exceptions
Initiate Parallel Tasks
Synchronize Parallel Tasks
Apply Rules
Business Process Workflow (2)
Proprietary and Confidential
Orchestra business analytics by control flow
Workflow A Workflow B Workflow C
Sub-process 1 Sub-process 2
Control Flow Represents a Business Process
Deploy to Portal
ApplicationBuilding Blocks
services
Process Building Blocks
definition of linkage/control
and user interactions
Business Process Workflow (3)
Proprietary and Confidential
Workflow interoperabilityWorkflows and business processes (BPEL)
Proprietary and Confidential
Embedding Workflow Analytics into Applications
Process View
Lifetime Value Service
Risk Service
Churn Service
Embeddable Analytic Applications
Analytical Workflows
Model Repository
Business Rules and Model
Deploy New Actions
customer data
Predictive scores
Risk data
Risk Evaluation
Acceptable Risk?
Yes
No
Get Value Score
Normal Service
Get Churn Score
Risk Assessment
Upgrade offer
KVM
Proprietary and Confidential
Integrating Analytics with Business Rules: Adaptive Business Process
En
terp
ris
e S
erv
ice
s B
us
Business Process
Business Portal
Business
operational
data
Analytics to drive adaptive processes
Rule engine Rule Engine
Proprietary and Confidential
Embedding with Applications
InforSense Tools as one item in Windows based application system
Proprietary and Confidential
Proprietary and Confidential
“One of the biggest barriers to achieving productivity and responsiveness is IT – it has become a bottleneck. Another barrier to achieving the goal is the lack of intelligence that drives most IT applications. They are just operating as a rapid functional replacement, and failing to exploit the data which is being generated within other elements of the IT infrastructure.
A product that could meet that challenge and enable business to generate and deploy intelligence with speed, accuracy and without the need for specialized skills would be remarkable.
I believe that InforSense is that remarkable tool.”
-- David Norris, Senior Analyst, Bloor Research
Making Workflow Work
Proprietary and Confidential
Thank You !