![Page 1: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/1.jpg)
N E T W O R K E D M A C H I N E L E A R N I N G
J O A Q U I N VA N S C H O R E N ( T U / E ) , 2 0 1 4
#OpenML
![Page 2: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/2.jpg)
Research different.
![Page 3: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/3.jpg)
1 6 1 0
G A L I L E O G A L I L E I D I S C O V E R S S A T U R N ’ S R I N G S
‘ S M A I S M R M I L M E P O E TA L E U M I B U N E N U G T TA U I R A S ’
![Page 4: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/4.jpg)
Research different. Royal society: Take nobody’s word for itScientific Journal: Reputation-based culture
![Page 5: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/5.jpg)
3 0 0 Y E A R S L AT E RJ O U R N A L S S H O W L I M I T S
• Complex code not included
• Large data sets not included
• Experiment details scant
• Results hard to reproduce
• Papers not updatable
• Slow, incomplete tracking of
paper impact
• Publication bias
• No online public discussion
• Open access?
![Page 6: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/6.jpg)
J O U R N A L S : L O N G - T E R M M E M O R Y I N T E R N E T: S H O R T- T E R M W O R K I N G M E M O R Y
N E T W O R K E D S C I E N C E
O N L I N E D A TA B A S E S
O P E N S O U R C E C O D E
W E B S E R V I C E S , A P I S
C O L L A B O R A T I V E T O O L S
!
O P E N , S C A L A B L E C O L L A B O R A T I O N
R E A L - T I M E D I S C U S S I O N
C O M B I N E , R E U S E S C I E N T I F I C R E S U LT S
C I T I Z E N S C I E N C E
![Page 7: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/7.jpg)
Research different. Polymaths: Solve math problems through massive collaboration (not competition)
Broadcast question, combine many minds to solve it
Solved hard problems in weeks
Many (joint) publications
![Page 8: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/8.jpg)
Research different. SDSS: Robotic telescope, data publicly online (SkyServer)
+1 million distinct users vs. 10.000 astronomers
Broadcast data, allow many minds to ask the right questions
Thousands of papers
![Page 9: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/9.jpg)
Research different. Galaxy Zoo: citizen scientists classify a million galaxiesOffer right tools so that anybody can be a scientistMany novel discoveries by scientists and citizens
![Page 10: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/10.jpg)
Research different. Sharing data sparks discovery
Designed serendipity: - What’s hard for one scientist is
easy for another - Surprising ideas, observations
can spark new discoveries
Share, organise data for easy, large-scale collaboration
Data exploding in all sciences: collaborative data analysis needed
![Page 11: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/11.jpg)
Building reputationAuthorship: easy to contribute + contributions stored, visible online
Collaboration: build trust, work with new people
Citation: more people see, build upon, and cite your work. Tell people how to cite data and code.
Altmetrics: track reuse/interest online (ArXiv)
![Page 12: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/12.jpg)
N E T W O R K E D M A C H I N E L E A R N I N G
![Page 13: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/13.jpg)
Machine learningComplex code, large-scale data, experiments (impossible to print)
Experiments not shared online: impossible to build on prior work: inhibits deeper analysis (e.g. meta-learning)
Low reproducibility, generalisability (studies contradict)
What if we could all connect with each other, and with other scientists, to explore and apply machine learning?
Few collaborative tools to speed up research
![Page 14: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/14.jpg)
OpenMLPlace to share data, code, experiments in full detail
All results organised, linked together for further (meta)analysis, reuse, discussion, study, education
Links to (open-source) code, open data anywhere online.
Anyone can post data to analyse, anyone can share code and results (models, predictions, evaluations)
Integrated in ML platforms (R, Weka, Rapidminer,…) to automatically load data, upload results
Scientists can work in teams, but results only publicly visible if data, code shared
![Page 15: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/15.jpg)
OpenML: benefits for scientistsMore time: automates routinizable work: - find data and/or code - setup and run large-scale experiments - results compared to state-of-the-art - log experiment details for future reference
More control: - state how others should cite your work - track reuse - share results more easily
More knowledge: - more time for actual research - build directly on prior work - easier, large-scale collaboration + interaction
![Page 16: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/16.jpg)
![Page 17: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/17.jpg)
![Page 18: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/18.jpg)
![Page 19: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/19.jpg)
![Page 20: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/20.jpg)
![Page 21: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/21.jpg)
![Page 22: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/22.jpg)
![Page 23: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/23.jpg)
![Page 24: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/24.jpg)
![Page 25: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/25.jpg)
![Page 26: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/26.jpg)
![Page 27: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/27.jpg)
![Page 28: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/28.jpg)
![Page 29: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/29.jpg)
![Page 30: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/30.jpg)
![Page 31: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/31.jpg)
![Page 32: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/32.jpg)
![Page 33: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/33.jpg)
![Page 34: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/34.jpg)
![Page 35: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/35.jpg)
![Page 36: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/36.jpg)
![Page 37: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/37.jpg)
![Page 38: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/38.jpg)
![Page 39: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/39.jpg)
![Page 40: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/40.jpg)
![Page 41: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/41.jpg)
![Page 42: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/42.jpg)
![Page 43: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/43.jpg)
![Page 44: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/44.jpg)
![Page 45: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/45.jpg)
![Page 46: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/46.jpg)
![Page 47: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/47.jpg)
![Page 48: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/48.jpg)
![Page 49: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/49.jpg)
![Page 50: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/50.jpg)
![Page 51: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/51.jpg)
![Page 52: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/52.jpg)
![Page 53: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/53.jpg)
![Page 54: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/54.jpg)
![Page 55: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/55.jpg)
Plugins: WEKA
![Page 56: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/56.jpg)
Plugins: MOA
![Page 57: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/57.jpg)
Plugins: RapidMiner
1 . O P E R AT O R T O D O W N L O A D TA S K ( TA S K T Y P E S P E C I F I C )
2 . S U B W O R K F L O W T H AT S O LV E S T H E TA S K , G E N E R AT E S R E S U LT S
3 . O P E R AT O R F O R U P L O A D I N G R E S U LT S
![Page 58: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/58.jpg)
OpenML: under developmentOpenML studies - collection of datasets, flows, runs, results in a study - online counterpart of paper (with url) - construct by simply tagging resources - easily include (build on) data of others
Reputation building - Profile page: statistics of activity and impact on OpenML - Collaborative leaderboards: best contributors to solving a task
Teams - Add scientists in teams (circles) - Share resources, results within team only - Make public at any time (e.g. after publication)
Meta-learning support - Data/Flow qualities: easy adding, better overviews - Algorithm selection techniques running on website (vs humans?)
![Page 59: OpenML Tutorial: Networked Science in Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022042816/5590f1231a28ab50528b45a2/html5/thumbnails/59.jpg)
J O I N T H E C L U B