introducing pentaho reporting solutions for cern’s ... · pentaho reporting solutions 16...
TRANSCRIPT
![Page 1: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/1.jpg)
Introducing Pentaho reporting solutions for CERN’s administrative dataGabriele Thiede - CERN
06.03.2018 1
![Page 2: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/2.jpg)
? !
![Page 3: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/3.jpg)
What is CERN?
• CERN = "Conseil Européen pour la Recherche Nucléaire"
• Founded in 1954
• Situated in Switzerland, Geneva
• 12 founding Member States: Belgium, Denmark, France, the Federal Republic of Germany, Greece, Italy, the Netherlands, Norway, Sweden, Switzerland, the United Kingdom, and Yugoslavia
• Mandate:Establish a world-class fundamental physics research organization in Europe
![Page 4: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/4.jpg)
CERN Member States – 64 years later
![Page 5: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/5.jpg)
The Activities of CERN
• Probing the fundamental structure of the universe
• Study the basic constituents of matter - the fundamental particles -and the forces acting between them
• Constructing and running particle accelerators and detectors
![Page 6: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/6.jpg)
How CERN is financed
• Budget: ca. 1 Mrd. CHF per year
• Calculation of contributions based on GDP (gross domestic product) of each Member State.
• Additional contributions from Associate Member States, Associate Member States in the pre-stage to Membership etc.
![Page 7: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/7.jpg)
People at CERN
Total: 21.840
• Members of the personnel (17.540)o Employed members of the personnel (3.440)
o Associated members of the personnel (14.100)
• Contractor’s staff + Temporary Labour (4.300) 16%
64%
20%
Employed members of the personnel
Associated members of the personnel
![Page 8: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/8.jpg)
Administrative data and reporting
![Page 9: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/9.jpg)
Administrative Domains at CERN
9
The Classics
Human resources Finance Purchasing Logistics
Extras
Training Access Transportation Accomodations
Social security
Health insurance Pension Fund School Fees Unemployment benefits
Host states
Customs formalities Removals Licence plates
Work and residence
permits
06.03.2018
![Page 10: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/10.jpg)
Infrastructure – current situation
3/7/2018 10
![Page 11: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/11.jpg)
Infrastructure – coming soon
11
• Analyser
• Report Designer
• C-Tools
Pentaho
PDI
06.03.2018
![Page 12: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/12.jpg)
Time Frame
12
• Start: 2016
• End: 2018
= 2,5 years for• Replacement of data warehouse solution
(responsible service: Infrastructure section)
• In parallel: Migration of existing reporting(responsible service: Reporting section)
06.03.2018
![Page 13: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/13.jpg)
Migration of existing Reporting
13
• Ca. 380 existing Business Objects reports
• Covering all administrative domains
• From small & simple to huge & complex reports with over 80 tables and 20 graphs
• Self service reports, scheduled reports
• For internal and external clients
06.03.2018
![Page 14: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/14.jpg)
Resources for migration
14
• 3 permanent staff (avg. 1,87 FTE/year)• Migration in addition to regular work
• Temporary help, short term contracts (5 persons = avg. 2,2 FTE/year)• Dedicated to migration, only
06.03.2018
![Page 15: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/15.jpg)
Project Organisation
15
Inventory Migration Plan
• List of all used reports
• Main characteristics
• Frequency of usage
• Data sources
• Client(s)
• Priorities• Complexity
• Availability of DWH tables
• Preferred Pentaho solution
06.03.2018
![Page 16: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/16.jpg)
Pentaho reporting solutions
16
• Pentaho Analyser• Predefined reports (scheduled + on demand)
• Analyser dashboards
• Ad-hoc analysis on cubes
• Report Designer (scheduled + on demand)
• CTools dashboards
06.03.2018
![Page 17: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/17.jpg)
Requirements for report content
Provision of data in DWH
Implementation of new report
Organisation of work with clients
17
Analysis of needs
Inventory of BO reports
Discussion and identification of solutions
Investi-gation of solutions
TestImplementa
tionUseRelease
06.03.2018
![Page 18: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/18.jpg)
Interim Balance
![Page 19: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/19.jpg)
Where are we now?
19
• 4 temporary resources trained, 1 in training
• 17 dashboards developped
• Ca. 280 reports migrated
• Backlog: reports for training and recruitment
• Many new technical competencies acquired
06.03.2018
![Page 20: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/20.jpg)
Our experience with Pentaho (1/3)
20
Pentaho Analyser for specialised users
Comfortable tool to establish ad-hoc data analysis
Not self explanatory, training necessaryOccasional users get lost and refer to reporting service
Consequences: Desired effect (more autonomous users) did not materialize
06.03.2018
![Page 21: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/21.jpg)
Our experience with Pentaho (2/3)
21
Access right management of Analyser
Separate access rights to report folders and to cubes
Access right to corresponding cube necessary to executeAnalyser reportNo possibility to disable layout panel and list of available fields
Consequences: For users without access to cube ony Report Designer reports possible
06.03.2018
![Page 22: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/22.jpg)
Our experience with Pentaho (3/3)
22
Pentaho‘s strenghts and weaknesses
High acceptance of end users:- Attractive reports and dashboards with Pentaho; Analyser
Developpers see room for improvement:- Report Designer cumbersome- Pentaho components patch work, missing integration- Poor scheduling and search functions and graphs
Consequences: no use of Interactive reports; inhouse development of scheduler and search tool; Excel workaround for graphs
06.03.2018
![Page 23: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/23.jpg)
1st Lesson learned
23
Parallel work on infrastructure and reporting
User need drivenReporting serves as validation
Time consuming, delaysFixes of infrastructure impact reporting (rewrite of queries)
Recommendation: Infrastructure („data lake“ like) before reporting
06.03.2018
![Page 24: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/24.jpg)
2nd Lesson learned
24
Temporary short term resources for reporting
Working on Pentaho is attractive, many interested candidates
Business knowledge (important but complex!) is lackingTraining adds workload to permanent staffKnowledge transfer at departure
Recommendation: avoid short term resources
06.03.2018
![Page 25: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/25.jpg)
3rd Lesson learned
25
Version control and documentation
• Preserve and transfer of knowledge
• Coordinate team work
• Use of tools like GIT and Sourcetree, JIRA
06.03.2018
![Page 26: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/26.jpg)
Conclusion
26
Migration aslearningprocess
Migration as chance
• Technical knowledge aboutPentaho
• Reporting techniques
• Project management
• Massive tidy up
• Synergy effects
• Standardisation and harmonisation
• Improvement of data and reporting quality
06.03.2018
![Page 27: Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16 •Pentaho Analyser •Predefined reports (scheduled + on demand) •Analyser dashboards](https://reader030.vdocuments.site/reader030/viewer/2022012322/5c1c86c809d3f25e398b6543/html5/thumbnails/27.jpg)
Thank you for your attention