state of the art web mapping with open source
DESCRIPTION
OSCON 2012 workshop given by Dane SpringmeyerTRANSCRIPT
![Page 1: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/1.jpg)
State of the ArtWeb Mapping
with Open SourceOSCON 2012 | Dane Springmeyer
![Page 2: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/2.jpg)
@springmeyergithub / twitter
![Page 3: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/3.jpg)
see also:Justin Miller
Building a mobile, offline mapping stack using open tools & data
5pm Wednesday, F150
![Page 4: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/4.jpg)
Background
Engineer @ MapBox
Building TileMill and Mapnik
Web performance / rendering
![Page 6: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/6.jpg)
![Page 7: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/7.jpg)
We provide services & open source tools
![Page 8: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/8.jpg)
open source toolsto cover
![Page 9: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/9.jpg)
![Page 10: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/10.jpg)
![Page 11: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/11.jpg)
![Page 12: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/12.jpg)
![Page 13: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/13.jpg)
CartoDB
![Page 14: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/14.jpg)
![Page 15: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/15.jpg)
![Page 16: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/16.jpg)
TileMill
![Page 17: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/17.jpg)
•
![Page 18: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/18.jpg)
maps are simple(a primer)
![Page 19: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/19.jpg)
geodata as just another
data field / type
![Page 20: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/20.jpg)
cartography as just a sexy form of data
visualization
![Page 21: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/21.jpg)
location: lat/lon, x/y
attributes:name, type, value
![Page 22: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/22.jpg)
styles separate from data akin to css/html
![Page 23: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/23.jpg)
CartoCSS@motorway: #90BFE0;
.highway[TYPE='motorway'] {
.line[zoom>=0] {
line-color: spin(darken(@motorway,36),-10);
line-cap:round;
line-join:round;
}
.fill[zoom>=10] {
line-color:@motorway;
line-cap:round;
line-join:round;}}
![Page 24: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/24.jpg)
point: •line: -----------polygon: _ _ _ _ _ | _ _ _ _ _ |
spatial types
![Page 25: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/25.jpg)
Multi* typesmany to one
geometries feature
![Page 26: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/26.jpg)
latitude longitude name
45.5 -122.6 PDX
tabular geo-csv
![Page 27: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/27.jpg)
WKT Name
MULTIPOINT ((10 40), (40 30), (20 20), (30 10)) Group of Cities
tabular geo-csv(multipoint)
![Page 28: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/28.jpg)
{ "type": "FeatureCollection", "features": [ { "type": "Feature", "properties": { "name": "PDX" }, "geometry" : { "type": "Point", "coordinates": [ -122.6, 45.5 ] } }]}
geojson
Works everywhere: e.g QGIS, TileMill, Web clients
![Page 29: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/29.jpg)
postgispostgis=# select 'POINT(-122.6 45.5)'::geometry as geom,'PDX'::text as name;
geom | name --------------------------------------------+------ 01010000006666666666A65EC00000000000C04640 | PDX(1 row)
WKB(Well Known Binary)
![Page 30: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/30.jpg)
postgispostgis=# select ST_Distance('POINT(-122.6 45.5)'::geography,'POINT(-122.3 47.6)'::geography)/1609.344 as dist_in_miles_from_pdx_to_sea;
dist_in_miles_from_pdx_to_sea ------------------------------- 145.755555956692(1 row)
![Page 31: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/31.jpg)
minimal code for simple maps, both server and client
![Page 32: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/32.jpg)
Mapnik
var mapnik = require('mapnik');var map = new mapnik.Map(256, 256);map.loadSync('map.xml');map.zoomAll();map.renderFileSync('map.png');
![Page 33: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/33.jpg)
Leaflet<html><head> <link rel="stylesheet" href="http://cdn.leafletjs.com/leaflet-0.3.1/leaflet.css" /> <script src="http://cdn.leafletjs.com/leaflet-0.3.1/leaflet.js"></script></head><body>! <div id="map" style="width: 100%; height: 100%"></div> <script> var map = new L.Map('map'); var osm = new L.TileLayer('http://tile.osm.org/{z}/{x}/{y}.png'); map.setView(new L.LatLng(45.5, -122.65), 12).addLayer(osm); var pdx = { "type": "FeatureCollection", "features": [ { "type": "Feature", "properties": { "name": "PDX" }, "geometry" : { "type": "Point","coordinates": [ -122.65, 45.5 ]}}] }; map.addLayer(new L.GeoJSON(pdx)); </script></body></html>
![Page 34: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/34.jpg)
•
![Page 35: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/35.jpg)
but maps are hard
![Page 36: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/36.jpg)
geodata can be messy and multi-
resolution
![Page 37: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/37.jpg)
geodata can behuge
![Page 38: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/38.jpg)
geodata can be dynamic
![Page 39: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/39.jpg)
data storytakes too long
![Page 40: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/40.jpg)
maps as the single lock-in point (google)
or point of failure(Slow WMS, IE support, clashing
design)
![Page 41: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/41.jpg)
•
![Page 42: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/42.jpg)
how modern web maps work
![Page 43: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/43.jpg)
or, how to tell stories with maps quickly,
ensure they are fast under load and
work in IE
![Page 44: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/44.jpg)
open dataosm.org
naturalearthdata.comus census (geo/www/tiger)local governments portals
![Page 45: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/45.jpg)
serversidepre-processing
![Page 46: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/46.jpg)
gradually work clientside
![Page 47: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/47.jpg)
tile renderers mapnik / mapserver
![Page 48: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/48.jpg)
fast app serversNode.js/Python/C++
![Page 49: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/49.jpg)
pre-processedpre-rendered
cacheable
![Page 50: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/50.jpg)
beautiful graphicsAntigrain Geometry
Cairo Graphics
![Page 51: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/51.jpg)
standard formatsgeojson, wkt, csv, shapefile, postgis,
geotiff
![Page 52: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/52.jpg)
tiled data
![Page 53: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/53.jpg)
bake big data bitmaps
![Page 54: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/54.jpg)
pre-render where possible, but beware the
world is big
![Page 55: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/55.jpg)
![Page 56: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/56.jpg)
![Page 57: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/57.jpg)
![Page 58: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/58.jpg)
![Page 59: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/59.jpg)
![Page 60: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/60.jpg)
coming...optimized tiled formats like msgpack, protobuf
(not just bitmaps)
![Page 61: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/61.jpg)
coming ...optimized tiled storage
and indexing
![Page 62: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/62.jpg)
coming ...more robust and
configurable clientside renderers
![Page 63: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/63.jpg)
Storage: Database / Flat file (Spatially Indexed)
Data Tiles:Optimized vectors
Image Tiles:Software Vector
Renderer
Display:<img> tiles
Render:Hardware Vector
Renderer
UI: Browser JS client / Mobile Native client
Data Processing: GIS Apps, Scripting
![Page 64: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/64.jpg)
Storage: Postgres-PostGIS / CSV / GeoJSON
Data Tiles:TileStache / Kothic.js /TileMill ?
Image Tiles:Mapnik via TileMill
or TileStache+ UTFGrid interactivity
Display:all browers
support
Render:Kothic.js / Vecnik
(Canvas/WebGL insome browsers)
UI: Leaflet / ModestMaps / RouteMe
Data Processing: Qgis, R stats, Python, SQL
![Page 65: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/65.jpg)
•
![Page 67: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/67.jpg)
Installing TileMill
![Page 68: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/68.jpg)
TileMill: Ubuntu
sudo apt-add-repository ppa:developmentseed/mapbox
sudo apt-get -y update
sudo apt-get -y install tilemill
sudo start tilemill
![Page 69: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/69.jpg)
TileMill: Mac
![Page 70: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/70.jpg)
TileMill: Basics
Cross platform - Linux, Win, OS X
Same code both desktop & web
Outputs PNG, MBTiles, Mapnik XML
Written in Javascript (Node.js) and C++ (Mapnik)
![Page 71: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/71.jpg)
Art of the possible
![Page 72: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/72.jpg)
http://project.wnyc.org/stop-frisk-guns/
![Page 74: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/74.jpg)
foursquare.com
![Page 77: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/77.jpg)
Millions of points without sacrificing speed
![Page 78: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/78.jpg)
TileMill: Live
http://bit.ly/MFjLnGhttp://bit.ly/SFeBfJEc2 machines only available on July 17,2012
set one up yourself like: http://mapbox.com/tilemill/docs/guides/ubuntu-service/
![Page 79: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/79.jpg)
Demos...
• TileMill: layer ordering, fonts, labeling, plugins, mbtiles export, mapnik xml export, svg/marker-transforms
• TileMill: reinforce basics through demos: arc.js geojson, cartodb csv, etherpad csv
• OSM-bright setup
![Page 80: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/80.jpg)
Thanks!
@springmeyergithub / twitter
![Page 81: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/81.jpg)
•
![Page 82: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/82.jpg)
Do not miss Stamen and Vizzuality
(Cartodb)
![Page 83: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/83.jpg)
![Page 84: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/84.jpg)
![Page 85: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/85.jpg)
TileMill Extra Details
![Page 86: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/86.jpg)
![Page 87: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/87.jpg)
![Page 88: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/88.jpg)
![Page 89: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/89.jpg)
![Page 90: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/90.jpg)
![Page 91: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/91.jpg)
![Page 92: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/92.jpg)
![Page 93: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/93.jpg)
PERSPECTIVEdoi:10.1038/nature10836
The case for open computer programsDarrel C. Ince1, Leslie Hatton2 & John Graham-Cumming3
Scientific communication relies on evidence that cannot be entirely included in publications, but the rise ofcomputational science has added a new layer of inaccessibility. Although it is now accepted that data should be madeavailable on request, the current regulations regarding the availability of software are inconsistent. We argue that, withsome exceptions, anything less than the release of source programs is intolerable for results that depend on computation.The vagaries of hardware, software and natural language will always ensure that exact reproducibility remainsuncertain, but withholding code increases the chances that efforts to reproduce results will fail.
T he rise of computational science has led to unprecedentedopportunities for scientific advance. Ever more powerful computersenable theories to be investigated that were thought almost
intractable a decade ago, robust hardware technologies allow data collec-tion in the most inhospitable environments, more data are collected, andan increasingly rich set of software tools are now available with which toanalyse computer-generated data.
However, there is the difficulty of reproducibility, by which we meanthe reproduction of a scientific paper’s central finding, rather than exactreplication of each specific numerical result down to several decimalplaces. We examine the problem of reproducibility (for an early attemptat solving it, see ref. 1) in the context of openly available computerprograms, or code. Our view is that we have reached the point that, withsome exceptions, anything less than release of actual source code is anindefensible approach for any scientific results that depend on computa-tion, because not releasing such code raises needless, and needlesslyconfusing, roadblocks to reproducibility.
At present, debate rages on the need to release computer programsassociated with scientific experiments2–4, with policies still ranging frommandatory total release to the release only of natural language descrip-tions, that is, written descriptions of computer program algorithms.Some journals have already changed their policies on computer programopenness; Science, for example, now includes code in the list of itemsthat should be supplied by an author5. Other journals promoting codeavailability include Geoscientific Model Development, which is devoted,at least in part, to model description and code publication, andBiostatistics, which has appointed an editor to assess the reproducibilityof the software and data associated with an article6.
In contrast, less stringent policies are exemplified by statements suchas7 ‘‘Nature does not require authors to make code available, but we doexpect a description detailed enough to allow others to write their owncode to do similar analysis.’’ Although Nature’s broader policy states that‘‘...authors are required to make materials, data and associated protocolspromptly available to readers...’’, and editors and referees are fullyempowered to demand and evaluate any specific code, we believe thatits stated policy on code availability actively hinders reproducibility.
Much of the debate about code transparency involves the philosophy ofscience, error validation and research ethics8,9, but our contention is morepractical: that the cause of reproducibility is best furthered by focusing onthe dissection and understanding of code, a sentiment already appreciatedby the growing open-source movement10. Dissection and understandingof open code would improve the chances of both direct and indirectreproducibility. Direct reproducibility refers to the recompilation and
rerunning of the code on, say, a different combination of hardware andsystems software, to detect the sort of numerical computation11,12 andinterpretation13 problems found in programming languages, which wediscuss later. Without code, direct reproducibility is impossible. Indirectreproducibility refers to independent efforts to validate something otherthan the entire code package, for example a subset of equations or a par-ticular code module. Here, before time-consuming reprogramming of anentire model, researchers may simply want to check that incorrect coding ofpreviously published equations has not invalidated a paper’s result, toextract and check detailed assumptions, or to run their own code againstthe original to check for statistical validity and explain any discrepancies.
Any debate over the difficulties of reproducibility (which, as we willshow, are non-trivial) must of course be tempered by recognizing theundeniable benefits afforded by the explosion of internet facilities and therapid increase in raw computational speed and data-handling capabilitythat has occurred as a result of major advances in computer technology14.Such advances have presented science with a great opportunity to addressproblems that would have been intractable in even the recent past. It isour view, however, that the debate over code release should be resolved assoon as possible to benefit fully from our novel technical capabilities. Ontheir own, finer computational grids, longer and more complex compu-tations and larger data sets—although highly attractive to scientificresearchers—do not resolve underlying computational uncertainties ofproven intransigence and may even exacerbate them.
Although our arguments are focused on the implications of Nature’scode statement, it is symptomatic of a wider problem: the scientificcommunity places more faith in computation than is justified. As weoutline below and in two case studies (Boxes 1 and 2), ambiguity in itsmany forms and numerical errors render natural language descriptionsinsufficient and, in many cases, unintentionally misleading.
The failure of code descriptionsThe curse of ambiguityAmbiguity in program descriptions leads to the possibility, if not thecertainty, that a given natural language description can be convertedinto computer code in various ways, each of which may lead to differentnumerical outcomes. Innumerable potential issues exist, but mightinclude mistaken order of operations, reference to different model ver-sions, or unclear calculations of uncertainties. The problem of ambiguityhas haunted software development from its earliest days.
Ambiguity can occur at the lexical, syntactic or semantic level15 and isnot necessarily the result of incompetence or bad practice. It is a naturalconsequence of using natural language16 and is unavoidable. The
1Department of Computing Open University, Walton Hall, Milton Keynes MK7 6AA, UK. 2School of Computing and Information Systems, Kingston University, Kingston KT1 2EE, UK. 383 Victoria Street,London SW1H 0HW, UK.
2 3 F E B R U A R Y 2 0 1 2 | V O L 4 8 2 | N A T U R E | 4 8 5
Macmillan Publishers Limited. All rights reserved©2012
![Page 94: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/94.jpg)
![Page 95: State of the Art Web Mapping with Open Source](https://reader033.vdocuments.site/reader033/viewer/2022061300/54c7a4984a7959c0408b4637/html5/thumbnails/95.jpg)