smart international symposium for next generation infrastructure: bio-inspired cost effective access...
DESCRIPTION
A presentation conducted by Dr Jun Shen, School of Information Systems and Technology University of Wollongong. Presented on Tuesday the 1st of October 2013 With the rapid proliferation of services and cloud computing, Big Data has become a significant phenomenon across many scientific disciplines and sectors of society, wherever huge amounts of data are generated and processed daily. End users will always seek higher-quality data access at lower prices. This demand poses challenges to service composers, service providers and data providers, who should maintain their service and data provision as cost-effectively as possible. This paper will apply bio inspired approaches to achieving equilibrium among the otherwise competitive stakeholders. In addition to novel models of cost for Big Data provision, bio-inspired algorithms will be developed and validated for dynamic optimisation. Furthermore, the optimised algorithms will also be applied in the data-mining research on the Alpha Magnetic Spectrometer (AMS) experiment, which is aiming to find dark matter in the universe. This experiment typically receives 200G and generates 700G data daily.TRANSCRIPT
Monday, 30th September 2013: Business & policy Dialogue
Tuesday 1 October to Thursday, 3rd October: Academic and Policy Dialogue
www.isngi.org
ENDORSING PARTNERS
The following are confirmed contributors to the business and policy dialogue in Sydney:
• Rick Sawers (National Australia Bank)
• Nick Greiner (Chairman (Infrastructure NSW)
www.isngi.org
Bio-inspired cost-effective access to big
data Presented by: Dr Jun Shen, School of Information Systems and Technology
University of Wollongong,
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Bio-inspired cost-effective access to big data
Lijuan Wang Jun Shen
School of Information Systems and TechnologyUniversity of Wollongong, Australia
ISNGI 2013
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Outline
Introduction
Problem statementBio-inspired cost-effective to access big dataConclusion and future work
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
A few streams of big data
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Outline
IntroductionProblem statement
Bio-inspired cost-effective to access big dataConclusion and future work
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Basic concepts
Services
Abstract servicesConcrete servicesQuality of service (QoS)Web service composition
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Basic concepts
ServicesAbstract services
Concrete servicesQuality of service (QoS)Web service composition
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Basic concepts
ServicesAbstract servicesConcrete services
Quality of service (QoS)Web service composition
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Basic concepts
ServicesAbstract servicesConcrete servicesQuality of service (QoS)
Web service composition
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Basic concepts
ServicesAbstract servicesConcrete servicesQuality of service (QoS)Web service composition
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Service and data usage and charging relationship
Data Provider
Service Provider
Service Composer
request provide
request provide
pay
paycharge
chargeelementary service
data set
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Optimizations in data-intensive service composition
AS1
AS2
ASn
abstractservices
csn,1
csn,2
csn,m-1
csn,m
concreteservices
dataset 1
dataset 2
dataset k
dataset k-1
datasets
replica 1
replica 2
replica l
replica l-1
data replicas
optimisation point 1 optimisation point 2
Application
optimisation point 3
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Outline
IntroductionProblem statementBio-inspired cost-effective to access big data
Conclusion and future work
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Why bio-inspired algorithms
Global optimization approach
Less computation timeFeatures such as autonomy, scalability, adaptability androbustness
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Why bio-inspired algorithms
Global optimization approachLess computation time
Features such as autonomy, scalability, adaptability androbustness
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Why bio-inspired algorithms
Global optimization approachLess computation timeFeatures such as autonomy, scalability, adaptability androbustness
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Bio-inspired algorithms
Biological systems are autonomous entities andself-organized
Simplicity and rapid convergenceStrengths in optimizing dynamic negotiations
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Bio-inspired algorithms
Biological systems are autonomous entities andself-organizedSimplicity and rapid convergence
Strengths in optimizing dynamic negotiations
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Bio-inspired algorithms
Biological systems are autonomous entities andself-organizedSimplicity and rapid convergenceStrengths in optimizing dynamic negotiations
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Case study: Alpha Magnetic Spectrometer (AMS)
Monte Carlo Simulation
Analog Detectors Simulation Data
Raw Data
Data reconstruction
Data reconstruction
Correction Data
Physical Analysis Result Storage
Query and Display
CEANT3 AMS-02Package
ROOT
Visualization
AMS-02Data Capture
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Outline
IntroductionProblem statementBio-inspired cost-effective to access big dataConclusion and future work
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
GA and MIP
10 20 30 40 500
1000
2000
3000
4000
5000
6000
7000
8000
Number of abstract services
Com
puta
tion
Tim
e (m
sce)
Genetic AlgorithmMixed Integer Programming
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
GA and MIP
100 200 300 400 500 600 700 800 900 10000
200
400
600
800
1000
1200
1400
Number of candidate services per class
Com
puta
tion
Tim
e (m
sec)
Genetic Algorithm
Mixed Integer Programming
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
ACS and GA
10 20 30 40 500
1000
2000
3000
4000
5000
6000
7000
8000
9000
Number of abstract services
Com
puta
tion
Tim
e (m
sec)
QWS
ACSGA
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
ACS and GA
100 200 300 400 500 600 700 800 900 10000
1000
2000
3000
4000
5000
6000
7000
8000
9000
Number of candidate services per class
Com
puta
tion
time
(mse
c)
QWS
ACSGA
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
MOACS and MOGA
3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 5 5.2
x 104
1.5
2
2.5
3
3.5
4
4.5
5x 10
5
Overall Cost
Ove
rall E
xecu
tion
Tim
e
Median Summary Attainment Surface
MOACS:n30m50MOGA:n30m50
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data
Outlines Problem statement Bio-inspired cost-effective to access big data Conclusion and future work Summary
Thank you very much!Questions and suggestions are welcome.
Lijuan Wang, Jun Shen University of Wollongong, Australia
Bio-inspired cost-effective access to big data