six sigma with r

Six Sigma With R(Springer, 2012)

November, 2012

Emilio L. CanoJavier M. Moguerza

Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control

VII Further and Beyond

Backmatter

Six Sigma with RStatistical Engineering for Process

Improvement

Emilio L. Cano, Javier M. Moguerzaand Andres Redchuk

November 20, 2012

Facultad de Estudios EstadısticosUniversidad Complutense de Madrid

Book Presentation UCM 1/59

http://www.springer.com/statistics/book/978-1-4614-3651-5


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Contenido

1 Frontmatter

2 MainmatterI BasicsII DefineIII MeasureIV AnalyzeV ImproveVI ControlVII Further and Beyond

3 Backmatter




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Contents

1 Frontmatter


3 Backmatter




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Publisher







November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Book website

http://www.sixsigmawithr.com/






November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

R Package

http://cran.r-project.org/web/packages/SixSigma/index.html






November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Frontmatter

ForewordPreface

Why Six Sigma with RWho is this book forConventionsProductionResourcesAbout the Authors

Acknowledgements

Contents

List of Tables and Figures

Acronyms

http://link.springer.com/book/10.1007/978-1-4614-3652-2//page/1



http://link.springer.com/book/10.1007/978-1-4614-3652-2//page/1


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Contents

1 Frontmatter


3 Backmatter




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

1. Six Sigma in a Nutshell

Herbert Spencer“Science is organised knowledge”




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

The DMAIC Cycle




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Six Sigma Roles

In Six Sigma, everyone in the organization hasa role in the project. Six Sigma methodologyuses an intuitive categorization of these roles.




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

2. R from the Beginning

Linus Torvalds“Software is like sex; it’s better when it’s free”




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

The R Project

http://www.r-project.org






November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

The R Environment




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

3. Process Mapping with R

Charles Franklin Kettering“A problem well stated is a problem half

solved”




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

A Process Map

Six Sigma Process Map

Paper Helicopter Project

INPUTSX

operators tools raw material facilities

INSPECTION

INP

UT

S

sheets...

Param.(x): width NCoperator CMeasure pattern Pdiscard P

Featur.(y): ok

ASSEMBLY

INP

UT

S

sheets

Param.(x): operator Ccut Pfix Protor.width Crotor.length Cpaperclip Ctape C

Featur.(y): weight

TEST

INP

UT

S

helicopter

Param.(x): operator Cthrow Pdiscard Penvironment N

Featur.(y): time

LABELING

INP

UT

S

helicopter

Param.(x): operator Clabel P

Featur.(y): label

OUTPUTSY

helicopter LEGEND(C)ontrollable(Cr)itical(N)oise(P)rocedure




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

4. Loss Funtion Analysis with R

W. Edwards DemingDefects are not free. Somebody makes them,

and gets paid for making them




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

A Loss Function Example

> ss.lfa(ss.data.bolts , "diameter", 0.5, 10, 0.001,

lfa.sub = "10 mm. Bolts Project",

lfa.size = 100000 , lfa.output = "both")

$lfa.k

[1] 0.002

$lfa.lf

expression(bold(L == 0.002 %. % (Y - 10)^2))

$lfa.MSD

[1] 0.03372065

$lfa.avLoss

[1] 6.74413e-05

$lfa.Loss

[1] 6.74413




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

A Loss Function Example (cont.)

Loss Function Analysis

10 mm. Bolts Project

0e+00

1e−04

2e−04

3e−04

4e−04

5e−04

LSL USL

T

9.6 9.8 10.0 10.2 10.4Observed Value

Cos

t of P

oor

Qua

lity

L = 0.002 ⋅ (Y − 10)2

Data

CTQ: diameterY0 = 10∆ = 0.5

L0 = 0.001Size = 1e+05

Mean = 10.0308k = 0.002

MSD = 0.0337Av.Loss = 1e−04

Loss = 6.7441




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

5. Measurement System Analysis

Lord Kelvin“If you cannot measure it,

you cannot improve it.”




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Repeatability & Reproducibility

●●

●

● ●●

Repetible and Reproducible

●●

●

●

●

●

Repetible but non Reproducible

●●●

●●

●

Reproducible but non Repetible

● ●

●

●

●

●

Non Repetible & Non Reproducible




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

MSA with R

Six Sigma Gage R&R Study

Helicopter Project

Components of Variation

Per

cent

0

20

40

60

80

G.R&R Repeat Reprod Part2Part

%Contribution %Study Var

Var by Part

var

1.0

1.2

1.4

1.6

1.8

prot #1 prot #2 prot #3

●

●

●●●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

Var by appraiser

var

1.0

1.2

1.4

1.6

1.8

op #1 op #2 op #3

●

●

●●●

●

●

●

●

●●●●

●

●

●●

●

●

●

●

●

●●

●

●

●

Part*appraiser Interaction

var

1.1

1.2

1.3

1.4

1.5

1.6

1.7


●●

●

●

●

●

●

●

●

op #1op #2

op #3

x Chart by appraiser

part

var

1.1

1.2

1.3

1.4

1.5

1.6

1.7


●●

●

op #1


●

●

●

op #2


●

●

●

op #3

R Chart by appraiser

part

var

0.1

0.2

0.3

0.4

0.5


●

●

●

op #1


●

●

●

op #2


●

●

●

op #3




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

6. Pareto Analysis with R

OvidioCausa latet: vis est notissima. [The cause ishidden, but the result is known.]




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Pareto Principle (80/20 rule)

Examples20 % of customers make 80 % of incomes

20 % of students get 80 % of good marks

80 % of cost of quality is due to 20 % ofthe possible causes




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Cause-and-effect diagram

Six Sigma Cause−and−effect Diagram

Paper Helicopter Project

Flight Time

Operatoroperator #1

operator #2operator #3

Environmentheight

cleaning

Toolsscissors

tape

Design

rotor.lengthrotor.width2

paperclip

Raw.Material

thicknessmarks

Measure.Tool

calibratemodel




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Pareto Chart

Del

ays

Mat

eria

ls

Cus

tom

er

Trai

ning

Rew

ork

Err

ors

Rai

n

Win

d

Per

mis

sion

s

Inad

equa

te

Tem

pera

ture

Pareto Chart for b.vector

Fre

quen

cy

020

4060

●

●

●

●

●

●

●

●

●●

●

80%

Cum

ulat

ive

Per

cent

age




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

7. Process Capability Analysis

Johann Wolfgang von GoetheOne cannot develop taste from what is ofaverage quality but only from the very best.




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Capability Analysis Output

Six Sigma Capability Analysis Study

Winery Project

Histogram & Density

LSL

Target

USL

740 745 750 755 760

Check Normality

●●

● ●●●

●●●●●●●●●●

● ●

●

● Shapiro−Wilk Testp−value: 0.07506

Lilliefors (K−S) Testp−value: 0.2291

Normality accepted when p−value > 0.05

Density Lines LegendDensity STTheoretical Dens. STDensity LTTheoretical Density LT

SpecificationsLSL: 740

Target: 750USL: 760

ProcessShort Term

Mean: 749.7625SD: 2.1042

n: 20Zs: 3.14

Long Term

Mean: 753.7239SD: 2.6958

n: 40Zs: 2.33

DPMO: 9952.5

IndicesShort Term

Cp: 1.5841CI: [1.4,1.7]

Cpk: 1.5465CI: [1.4,1.7]

Long Term

Pp: 1.2365CI: [1.1,1.3]

Ppk: 0.7760CI: [0.7,0.8]




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

8. Charts with R

John Tukey“The greatest value of a picture is when it

forces us to notice what we never expected tosee.”




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Multi-vari chartMulti−vari chart for Volume by color and operator

Filler

Vol

ume

1415161718

1 2 3

●

●

●

●

●

●

●●

●

●

●

●

B1

●

●

●●

●●

●

●

●●

●

●

C1

●

●●

●

●●

●

●

●

●

● ●

B2

1415161718

●

●

●

●

●

●●

●

●

●

●●

C2

1415161718

● ● ●●

● ●● ●●

●

●●

B3

1 2 3

● ● ●

●

●●

● ●

●

● ● ●

C3

batch1 2 3 4● ● ● ●




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

9. Statistics and Probability with R

Aaron Levenstein“Statistics are like bikinis. What they reveal is

suggestive, but what they conceal is vital.”




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Distributions

0 1 2 3 4

0.0

0.3

Hypergeometric

Elements in class A

Pro

babi

lity

0 10 30

0.00

0.10

Geometric

Items extracted until first success

Pro

babi

lity

10 20 30 40

0.00

0.06

Negative Binomial

Number of trials until 3 events

Pro

babi

lity

0 5 10 20

0.00

0.15

Poison

Number of successful experiments per unit

Pro

babi

lity

0 1 2 3 4 5

0.0

0.6

Exponential

Random Variable X

Pro

babi

lity

Den

sity

0 2 4 6

0.0

0.4

0.8

Lognormal

Random Variable X>0

Pro

babi

lity

Den

sity

−0.5 0.5 1.5

0.0

0.6

1.2

Uniform

Random Variable X

Pro

babi

lity

Den

sity

0 2 4 6 8

0.0

0.2

0.4

Gamma

Random Variable X

Pro

babi

lity

Den

sity

0.0 0.4 0.8

0.0

1.0

2.0

Beta

Random Variable X

Pro

babi

lity

Den

sity

0 2 4 6

0.0

0.3

0.6

Weibul

Random Variable X

Pro

babi

lity

Den

sity

−4 0 2 4

0.0

0.3

t−Student

Random Variable X

Pro

babi

lity

Den

sity

1.73

95%5%

10 30 50

0.00

0.06

Chi−squared

Random Variable X

Pro

babi

lity

Den

sity

30.14

95% 5%

0 1 2 3 4

0.0

0.6

F

Random Variable X

Pro

babi

lity

Den

sity

2.34

95%5%




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

10. Statistical Inference with R

George E.P. Box“All models are wrong; some models are

useful.”




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Confidence Interval Example

Confidence Interval for the Mean

Mean:StdDev:n:Missing:

950.0160.267

1200

95% CI:P−Var:t:

[949.967, 950.064]unknown

1.98

Shapiro−WilksNormality Test

0.985 p−value: 0.19

●●●

●●●

●●●●●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●

●●●●

Normal q−q Plot

0.0

0.5

1.0

1.5

949.0 949.5 950.0 950.5Value of len

dens

ity

Histogram & Density Plot




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

11. Design of Experiments with R

R.A. Fisher“Sometimes the only thing you cando with a poorly designedexperiment is to try to find out whatit died of”




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

The Importance of Experimenting

“An engineer who does not knowexperimental design is not anengineer”

Comment made by to oneof the authors [of“Statisticsfor experimenters”] by anexecutive of the ToyotaMotor Company.




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

12.Process Control with R

Walter A. Shewhart“Special causes of variation may be found and

eliminated.”




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Control Chart Plotting

p Chartfor stockouts

Group

Gro

up s

umm

ary

stat

istic

s

1 3 5 7 9 11 13 15 17 19 21

0.00

0.05

0.10

0.15

0.20

0.25

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

LCL

UCL

CL

●

Number of groups = 22Center = 0.1212294StdDev = 0.3263936

LCL is variableUCL is variable

Number beyond limits = 1Number violating runs = 0




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

13. Other Tools andMethodologies

Johann Wolfgang von GoetheInstruction does much, but encouragementeverything.




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Other topics

Failure Mode, Effects, and CriticalityAnalysis

Design for Six Sigma

Lean

Gantt Chart

Some Advanced R Topics




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Contents

1 Frontmatter


3 Backmatter




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Case Study




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Helicopter Template

> ss.heli()

null device

1

> #vignette (" HelicopterInstructions ") to get

instructions



Six Sigma with R | Paper Helicopter template

cut

fold ↑ fold ↓tape?

cut

fold

↓↓

cut

fold

↑↑

cutta

pe?

tape

?

clip?

min

(6.5cm)

std

(8cm)

max

(9.5cm)←

bod

y le

ngth

→← body width →min

(4cm)

min

(4cm)

max

(6cm)

max

(6cm)

min

(6.5cm)

std

(8cm)

max

(9.5cm)

← w

ings

leng

th →


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Enjoy the Case Study!




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

So what?

The Scientific Method




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

The Scientific Method and SixSigma

DefineAsk a question

Measure

Analyze

Improve

Control

Do some backgroundresearch

Construct a hypothesis

Test the hypothesiswith an experiment

Analyze the data and draw conclusions

Communicate results

DMAIC Cycle Scientific Method




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

The Key to Success

“Six Sigma speaks the language of business”

ISO 13053-1:2011

Six Sigma methodology is a quality paradigmthat translates the involved scientificmethodology into a simple way to apply thescientific method within every organization.




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Opportunities

For Today’s Graduate, Just One Word: Statistics(The New York Times, August 2009)

“I keep saying that the sexy job in thenext 10 years will be statisticians”

Data Scientist: The Sexiest Job of the 21stCentury (Harvard Business Review, October 2012)

. . . the “data scientist.” It’s ahigh-ranking professional with thetraining and curiosity to makediscoveries in the world of big data . . .



http://www.nytimes.com/2009/08/06/technology/06stats.html?_r=2&

http://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century/ar/1


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Opportunities (cont.)Gartner Sees 4.4M Big Data Jobs by 2015(Information Management, October 2012)

Lack of data scientists could derail big dataprojects: IBM (CIO, October 2012)

Son las matematicas, estupido (El Paıs,Noviembre 2012)

La economıa del conocimiento exigeuna educacion sustentada en tresfundamentos: un nivel avanzado enmatematica y estadıstica, unacapacidad elevada para escribir unargumento y un nivel avanzado deingles



http://www.information-management.com/news/Gartner-Sees-4-Million-Big-Data-Jobs-by-2015-10023420-1.html

http://www.cio.com.au/article/439798/lack_data_scientists_could_derail_big_data_projects_ibm_/

http://elpais.com/elpais/2012/11/12/opinion/1352747017_116039.html

http://elpais.com/elpais/2012/11/12/opinion/1352747017_116039.html


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

R Final Remarks

kdnuggets.com(2012) link



http://www.kdnuggets.com/polls/2012/analytics-data-mining-big-data-software.html


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

R Final Remarks (cont.)

Community4131 packages at CRAN (18/11/2012)

Task views

Manuals

Publications

http:

//cran.r-project.org/web/packages/



http://cran.r-project.org/web/packages/

http://cran.r-project.org/web/packages/


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter


CustomizationA company can develop a package that fits itsinner procedures and methods.

InnovationA company can develop and deploy aninnovative method from its R&D department,or from the result of other publishedresearches.




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter


Businesshttp:

//www.revolutionanalytics.com/

http://www.openanalytics.eu/

http://www.fellstat.com/

http://www.rstudio.com/ide/

http://www.datanalytics.com/

. . .



http://www.revolutionanalytics.com/

http://www.revolutionanalytics.com/

http://www.openanalytics.eu/

http://www.fellstat.com/

http://www.rstudio.com/ide/

http://www.datanalytics.com/


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter


GUI, IDERStudio

Eclipse + StatET

EMACS + EES

Deducer

. . .




November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter





November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

http://r-es.org/



http://r-es.org/


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

http://www.r-project.org/useR-2013



http://www.r-project.org/useR-2013


November, 2012


Andres Redchuk

Frontmatter

Mainmatter

I Basics

II Define

III Measure

IV Analyze

V Improve

VI Control


Backmatter

Discussion

Thanks !

[email protected]@emilopezcano

http://www.sixsigmawithr.com



http://www.sixsigmawithr.com

six sigma with r

Education

sigma rolesnovember

2page1 book presentation

book presentation ucm1359

book presentation ucm459

productionvi controlvii

book websitenovember

beyondbackmatter http

ri basicsii denewho