the 9 th annual workshop 25-27 september 2013 inria, grenoble, france
TRANSCRIPT
![Page 1: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/1.jpg)
The 9th Annual
Workshop25-27 September 2013INRIA, Grenoble, France
http://boinc.berkeley.edu/trac/wiki/WorkShop13
![Page 2: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/2.jpg)
The BOINC Community
UC Berkeleydevelopers (2.5)
Projects PC volunteers(240,000)
Other volunteers:testing
translationsupport
Computerscientists
![Page 3: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/3.jpg)
Workshop goals
● Learn what everyone else is doing● Form collaborations● Get ideas● Steer BOINC development
![Page 4: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/4.jpg)
Hackfest (Thu/Fri)
● Goal: get something done– design and/or implement software– improve docs– learn and use a new feature
● Bring your ideas
![Page 5: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/5.jpg)
The state of volunteer computing
● Volunteership: stagnant– 240K people (down from 290K)– 350K computers
● Science projects: stagnant● Computer Science research: a little● Let’s keep trying anyway
![Page 6: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/6.jpg)
Requests to projects
● Do public outreach– Notices (with pictures)– Automated reminder emails– News emails– Message boards– Mass media
● Use current server code– Avoid code divergence
![Page 7: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/7.jpg)
To developers/researchers
● Talk with me before starting anything– especially if it’s of general utility
● Let me know if you need data
![Page 8: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/8.jpg)
What’s new in BOINC?
● Funding● Integration projects● Remote job and file management● Android● Scheduler● GPU and multicore apps● Client● Plans
![Page 9: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/9.jpg)
Funding
● Current NSF grant runs another 18 months● Not clear if current model will continue● Collaborations are important for future funding● Projects may need to help fund BOINC directly
![Page 10: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/10.jpg)
Integration projects
● HTCondor (U. of Wisconsin)– Goal: BOINC-based back end for Open Science
Grid or any Condor pool
BOINCserver
Condor node
Grid manager
BOINC GAHP
Remote operations
![Page 11: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/11.jpg)
Integration projects
● HUBzero (Purdue U.)– Goal: BOINC-based back end for science portals
such as nanoHUB
BOINCserver
Hub
![Page 12: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/12.jpg)
Integration projects
● Texas Advanced Computing Center (TACC)– Android/iOS app– They supply
● Interfaces, visualization, support for scientists● Storage● BOINC server
![Page 13: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/13.jpg)
Remote input file management
● Issues
– Naming/immutability– Efficiency– Garbage collection
● User file sandbox (web-based) used by CAS
![Page 14: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/14.jpg)
Content-based file management
● Server file names based on MD5
● DB table for file/batch association; garbage collection
● Web RPCs to query lists of files, upload files
BOINC serverSubmit host
Jf_ec3056e9ed14c837e3e68c80bb14871f
Jf_dac0160fd3d7f910bae550ec26a164a8
Submit host
Input.dat
Input.dat
Foo.dat
![Page 15: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/15.jpg)
Remote job submission
● Web RPCs– Batch: estimate, create, query, abort, retire– Batch expire time– Job: query, abort– App: get templates
● Input file modes– Local, local-staged, semilocal, remote, inline
● C++, PHP bindings
![Page 16: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/16.jpg)
Output retrieval
● Web RPCs to– Get specific output files– Get zip of job’s outputs– Get zip of batch’s outputs
![Page 17: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/17.jpg)
BOINC on Android
● New GUI● Battery-related issues● Device naming● Released July 22
– Google Play Store, Amazon App Store– ~30K active devices
![Page 18: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/18.jpg)
Job size matching
● Problem: 1000X speed difference GPU vs Android
● An app can have jobs of N “size classes”● “size_census.php”: computes quantiles of
effective speed for each app● Scheduler tries to send jobs of size class i to
devices in quantile i● “size regulator” makes sure jobs of all size
classes are available to send
![Page 19: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/19.jpg)
New score-based scheduler
for each resource type (starting w/ GPUs)
scan job array starting at random point
make list of jobs with app version for resource
assign score (include job-size term)
sort list
for each job in list
do quick checks
lock array entry, do slow checks
send job
if request satisfied, break
![Page 20: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/20.jpg)
BOINC client
● New work-fetch, job scheduling– Handle GPU exclusions
● “App config” mechanism– User can set device usage parameters, limit # of
concurrent jobs per app
● Maintain/report current, previous uptime● Maintain list of completed jobs● Sub-second CPU throttling
![Page 21: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/21.jpg)
GPU and multicore apps
● Support Intel GPUs● Support OpenCL CPU apps
– Detect, advertise multiple OpenCL libraries
● Develop OpenCL example app● Detect GPUs in a separate process
– Mac notebooks: allow system to use low-power GPU
![Page 22: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/22.jpg)
BOINC runtime system
● Replace heartbeat with PID check– Not on Win2K: PID reuse
● Support apps that are in a critical section most of the time (e.g. GPU apps)
![Page 23: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/23.jpg)
Volunteer storage
● Finished data archival system– Store large files for long periods– Multi-level erasure coding
● Developed simulator for testing, performance study
![Page 24: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/24.jpg)
Software engineering
● Finished SVN → git migration● Automated translation process
– build_po → Pootle → commit → deploy
● Code hardening– strcpy() → strlcpy()– MAXPATHLEN
![Page 25: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/25.jpg)
Didn’t start
● OpenID/OpenAuth support● Remodel computing preferences● BOINC in app stores (Windows, Apple)
![Page 26: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/26.jpg)
Planned
● Automated build/test using Jenkins– Server code release management
● Accelerated batch completion● Apple iOS client
![Page 27: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/27.jpg)
My wish list: new GPU design
● Current: all GPUs of a given vendor are equivalent– Scheduler requests ask for NVIDIA jobs, not jobs
for a specific NVIDIA GPU– This doesn’t work well for machines with
heterogeneous GPUS– Work-arounds (GPU exclusions) cause problems
● Proposed: treat each GPU as a separate resource
![Page 28: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/28.jpg)
My wish list: fully embrace latency-oriented scheduling
● Types of workload– Throughput-oriented– Small/fast batches– Large/slow batches
● Suppose a project has all three?– Goal: client requests and processes short jobs even
if fast jobs are in progress– Requires complete redesign of scheduling policies
![Page 29: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/29.jpg)
Science@home
● The “project ecosystem” hasn’t materialized– Creating a project is too difficult, too risky– Volunteers tend to be passive– Marketing and PR: too many brands
● Umbrella projects: good, but not enough
![Page 30: The 9 th Annual Workshop 25-27 September 2013 INRIA, Grenoble, France](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649dbd5503460f94ab090a/html5/thumbnails/30.jpg)
Science@home
● Single “brand” for volunteer computing● Register for science areas rather than projects● Facebook/Google login● Use account-manager architecture● How to allocate computing power?
– Involve the HPC, scientific funding communities