aqp: a new framework for quantitative analysis of …...2800+ packages on cran: 100+ packages on...
TRANSCRIPT
AQP: A New Framework for Quantitative Analysisof Pedon Data
D.E. Beaudette, J.M. Skovlin, P. Roudier, A.T. O’Geen
USDA-NRCSLandcare Research, NZ
University of California Davis, CA
What is R? (if you use NASIS, you already have R!)
R is a drop-in replacement for 99% of tasks done via spreadsheet, and more
univariate / multivariate summaries
graphical representations of complex data
extensible via “packages”
2800+ packages on CRAN: 100+ packages on GIS, ecology, and soils!
→ repeatable, self-documenting work
R is a convenient environment testing ideas
testing by eye: simple graphical “grammar” used to plot data
formalized testing: correlation, regression, classification, ordination, . . .
→ algorithm development by experts, application by trained users
R is good for rapid development of new software
I/O capabilities: file, URL, SOAP, SQL, ODBC, PDF, PNG, SHP, KML, . . .
optimizers, matrix operations, custom data structures, . . .
→ well-tested, huge support base, mailing lists, free
The Algorithms for Quantitative Pedology (aqp) Package
Soil Profile Visualization
sketches depicting horizon sequence, depth, color, . . .
soil colors translated from Munsell system to RGB triplets
ordering of sketches along environmental gradients
Soil Profile Aggregation
summary of soil property or morphology along regular depth slices
transcends typical problems with soil profile collections:→ high variability in horizon description style and horizon types / depths
Soil Profile Classification
similar to above: distance metric computed along regular depth slices
can assist with similar/dissimilar evaluation
Soil-Specific Classes/Methods
data structures & vocabulary for working with collections of soil profiles
The Soil Database Interface (soilDB) Package
PedonPC
get all pedons from a ’pedon.mdb’ file: fetchPedonPC(dsn)
Local NASIS, after querying national database
get all associated pedons: fetchNASIS()
get all assocaited DMU data: fetchNASIS component data()
SSURGO Data - SDA
get SSURGO (tabular data) via SQL: SDA query(query)
get SSURGO linework via BBOX: mapunit geom by ll bbox(BBOX)
get SSURGO mukey(s) via BBOX: MUKEYS by ll bbox(BBOX)
Results
SoilProfileCollection object → spatial-site-diagnostic-pedon-hz
colors converted from Munsell to RGB
hz-errors flagged
Soil Profile Sketches
(Digital) Soil Profile Sketches: Pedon Kind
library(soilDB) # load packagef <− fetchNASIS() # load pedons from NASISnew.order <− order(f$pedon kind) # new ordering vectorplot(f, name='hzname', plot.order=new.order) # plot pedons as sketchslegend(...) # make a legend (details ommited)
Soil Profile Sketches & Landscape Gradients
(Digital) Soil Profile Sketches & Climatic Gradients
x <− read.csv('dahlgren−granitics.csv') # load data from CSV filex$soil color <− with(x, munsell2rgb(hue, value, chroma)) # convert Munsell to RGBx$HzD <− hzDistinctnessCodeToOffset(substr(x$hz boundary, 0, 1)) # convert hz−distinct code
depths(x) <− id ˜ top + bottom # init SoilProfileCollection objectsite(x) <− ˜ elev + MAAT + MAP + geo # split site vs. hz−level data
g.new.order <− order(x$elev) # generate plotting order via elevationplot(x, name='name', plot.order=g.new.order, hz.distinctness.offset='HzD')axis(1, at=1:length(x), labels=x$elev[g.new.order], line=−2)
Aggregate Representation of Soil Profile Collections
Base Saturation at pH 8.2(25th−50th−75th percentiles)
Dep
th (
cm)
0
20
40
60
80
100
120
140
20 40 60 80
0
20
40
60
80
100
120
140
MLRA18 22
Slice-wise aggregation of soilproperties
library(aqp)library(lattice)# load lab data from CSV filex <− read.csv('MLRA 18 22−lab data.csv')
# init SoilProfileCollection objectdepths(x) <− pedon id ˜ hztop + hzbotsite(x) <− ˜ site id + lon + lat + mlra
# slice−wise aggregation by MLRA of \%BS at pH 8.2a <− slab(x, mlra ˜ bs82)
# plotxyplot(top ˜ p.q50, groups=mlra, data=a, lower=a$p.q25, upper=a$p.q75,ylim=c(160,−5), alpha=0.5, scales=list(cex=1.25,y=list(tick.num=7, alternating=3), x=list(alternating=1)),panel=panel.depth function, prepanel=prepanel.depth function,ylab=list('Depth (cm)', cex=1.25),xlab=list('Base Saturation at pH 8.2', cex=1.25), par.settings=list(superpose.line=list(col=c('black','blue'), lty=c(1,2), lwd=2)),auto.key=list(columns=2, title='MLRA', points=FALSE, lines=TRUE))
Aggregate Representation of Soil Profile Collections
% Clay
Dep
th (
cm)
0
20
40
60
80
20 30 40 50
fine−loamy
20 30 40 50
0
20
40
60
80
loamy−skeletal
LoafercreekDomingopeak
GopheridgeWhiteflag
% FragsD
epth
(cm
)
0
20
40
60
80
0 20 40 60 80
fine−loamy
0 20 40 60 80
0
20
40
60
80
loamy−skeletal
LoafercreekDomingopeak
GopheridgeWhiteflag
# load packagelibrary(soilDB)# load MVO soils from local NASIS (query national first)f <− fetchNASIS()# slice−wise aggregation of select propertiesa <− slab(f, sampled as ˜ clay + total frags pct + phfield)# plotting details ommitted...
Aggregate Representation of Soil Profile Collections
% Clay
Dep
th (
cm)
0
20
40
60
80
100
120
140
160
10 20 30 40 50 60
91 %
100 %
88 %
82 %
64 %
55 %
36 %
27 %
15 %
12 %
Granite (33)
10 20 30 40 50 60
100 %
75 %
50 %
50 %
25 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
Andesite (4)
10 20 30 40 50 60
100 %
64 %
55 %
9 %
9 %
0 %
0 %
0 %
0 %
0 %
Latite (11)
10 20 30 40 50 60
96 %
94 %
77 %
44 %
21 %
11 %
5 %
2 %
0 %
0 %
Metavolcanics (214)
10 20 30 40 50 60
88 %
74 %
39 %
18 %
11 %
5 %
2 %
0 %
0 %
0 %
Slate (57)
10 20 30 40 50 60
94 %
100 %
91 %
65 %
44 %
29 %
13 %
10 %
5 %
0 %
0 %
Phyllite (78)
10 20 30 40 50 60
90 %
95 %
87 %
72 %
51 %
38 %
28 %
13 %
8 %
4 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
Schist (78)
10 20 30 40 50 60
91 %
100 %
91 %
73 %
64 %
36 %
27 %
9 %
0 %
0 %
Marble (11)
10 20 30 40 50 60
0
20
40
60
80
100
120
140
160
100 %
96 %
60 %
25 %
6 %
2 %
0 %
0 %
0 %
0 %
Serpentinite (48)
% Frags
Dep
th (
cm)
0
20
40
60
80
100
120
140
160
0 20 40 60
100 %
100 %
94 %
88 %
76 %
67 %
58 %
55 %
36 %
33 %
Granite (33)
0 20 40 60
100 %
100 %
75 %
75 %
75 %
75 %
50 %
50 %
50 %
50 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
25 %
Andesite (4)
0 20 40 60
100 %
82 %
64 %
55 %
55 %
36 %
36 %
36 %
36 %
9 %
Latite (11)
0 20 40 60
100 %
100 %
96 %
86 %
75 %
65 %
63 %
62 %
57 %
47 %
Metavolcanics (214)
0 20 40 60
100 %
98 %
93 %
79 %
74 %
70 %
70 %
67 %
49 %
42 %
Slate (57)
0 20 40 60
100 %
100 %
97 %
86 %
77 %
67 %
60 %
60 %
47 %
45 %
1 %
Phyllite (78)
0 20 40 60
100 %
100 %
97 %
92 %
83 %
69 %
59 %
54 %
47 %
40 %
5 %
5 %
3 %
1 %
1 %
1 %
1 %
1 %
1 %
1 %
1 %
1 %
1 %
1 %
1 %
1 %
Schist (78)
0 20 40 60
100 %
100 %
100 %
100 %
100 %
73 %
73 %
55 %
36 %
36 %
Marble (11)
0 20 40 60
0
20
40
60
80
100
120
140
160
100 %
100 %
94 %
90 %
81 %
81 %
81 %
81 %
75 %
67 %
Serpentinite (48)
pH
Dep
th (
cm)
0
20
40
60
80
100
120
140
160
5.5 6.0 6.5 7.0
70 %
79 %
70 %
64 %
52 %
45 %
30 %
24 %
15 %
9 %
Granite (33)
5.5 6.0 6.5 7.0
75 %
50 %
25 %
25 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
Andesite (4)
5.5 6.0 6.5 7.0
100 %
64 %
55 %
9 %
9 %
0 %
0 %
0 %
0 %
0 %
Latite (11)
5.5 6.0 6.5 7.0
53 %
53 %
40 %
24 %
13 %
7 %
4 %
2 %
1 %
1 %
Metavolcanics (214)
5.5 6.0 6.5 7.0
74 %
56 %
35 %
16 %
9 %
5 %
2 %
0 %
0 %
0 %
Slate (57)
5.5 6.0 6.5 7.0
68 %
74 %
68 %
46 %
31 %
23 %
10 %
8 %
4 %
0 %
0 %
Phyllite (78)
5.5 6.0 6.5 7.0
69 %
71 %
63 %
51 %
38 %
29 %
21 %
9 %
8 %
4 %
0 %
0 %
Schist (78)
5.5 6.0 6.5 7.0
73 %
64 %
64 %
45 %
45 %
27 %
18 %
0 %
0 %
0 %
Marble (11)
5.5 6.0 6.5 7.0
0
20
40
60
80
100
120
140
160
62 %
62 %
35 %
12 %
2 %
0 %
0 %
0 %
0 %
0 %
Serpentinite (48)
Aggregate Representation of Soil Profile Collections
Probability
Dep
th (
cm)
0
20
40
60
80
100
120
0.0 0.2 0.4 0.6 0.8 1.0
99 %
97 %
91 %
73 %
44 %
18 %
3 %
Non−SoilA
BABt
claypanCr
Quantitative (pair-wise) Comparison of Soils
(a) (b) (c)
“is a more like b, as compared to c?”
ideally transcending horizonation and description style
Pair-wise dissimilarity along depth-slices (Moore et al, 1972)
Oi
A
AB
01
Oe
A
AB1
02
A
AB
03Oi/A
A
Bt1
04
Oi
Oe/A
Ad
Bw
05
A
AB
Bt1
Bt2
06
A
AB1
AB2
07
A1
A2
Bt1
08
50 cm
40 cm
30 cm
20 cm
10 cm
0 cm
0
01 02 03 04 05 06 07 08
50 cm
40 cm
30 cm
20 cm
10 cm
0 cm
soil properties at slice 15
clay vcs ln_tc cec A1 6.4 21.5 -1.5 3.8 8.12 10.6 17.6 -1.5 5.9 4.93 8.8 10.5 -0.2 7.5 6.04 9.1 12.6 -0.8 5.6 5.95 6.8 16.2 -0.5 5.0 8.26 17.9 11.0 -0.1 9.4 5.27 7.0 11.7 -0.4 4.7 6.38 6.3 17.0 -0.1 4.7 7.9
pair-wise dissimilarity at slice 15
1 2 3 4 5 6 72 423 68 464 51 29 195 30 46 40 306 96 59 30 45 687 48 45 20 16 22 488 32 50 39 37 11 64 24
1
2
3
4
5
6
7
8
Results can support simmilar/dissimilar evaluation
A
BA
Bt1
Bt2
Bt3
BC
A
BA
Bt1
Bt2
Bt3
BCt
A
BA
Bt1
Bt2
Bt3
Bt4
A
BAt
Bt1
Bt2
C
A
BAt
Bt1
Bt2
Bt3
C
A
BA
Bt1
Bt2
Bt3
BC
A
BA
Bt1
Bt2
Bt3
Bt4
C
ABA
Bw
Bt1
Bt2
Bt3
BCt
100 cm
80 cm
60 cm
40 cm
20 cm
0 cm
Domingopeak
Component Relationship Graphs: CA630
Component Relationship Graphs: MT637
Component Relationship Graphs: San Joaquin Series
Possible Application and Relevance to SDJR
Visualization: rapid evaluation of complex data
outliers / mistakes tend to stick-out (profile sketches / aggregate data)
dendrograms depicting similarity (similar / dissimilar eval.)
graph-networks depicting connectivity (associated / related soils)
Aggregation: data-driven synthesis of a “representative soil profiles”
depth-slices can transcend variations in description style (with expert review!)
universally meaningful Low–RV–High values: 5th–50th–95th pctiles
OSDs / DMUs via direct aggregation: morphology+properties (data permitting!)
Slice-Wise Similarity: property-driven eval. of “similarity”
rapid ID of mistakes / outliers / missing data (!)
similar / dissimilar discussion for map unit design / re-correlation / joins
(semi-)automated re-allocation of profiles to new revised legend or series
Thank You
Online Resources:
R Manuals: http://cran.r-project.org/manuals.html
Some Favorites: http://casoilresource.lawr.ucdavis.edu/drupal/node/100
R-Inferno: http://www.burns-stat.com/pages/Tutor/R_inferno.pdf
Patrick Burns’ Tutorials: http://www.burns-stat.com/pages/tutorials.html
R-Forge Project Page: http://aqp.r-forge.r-project.org
In-Depth Examples: http://casoilresource.lawr.ucdavis.edu/drupal/taxonomy/term/56
Packages on CRAN (http://cran.r-project.org)
http://cran.r-project.org/web/packages/aqp
http://cran.r-project.org/web/packages/soilDB