creating statistical reports in the past, present and ... · software fortran r,python,d3 julia,js...

Creating statistical reports in thepast, present and future

On reproducible research and literate programming

Gergely Daróczidaroczig@rapporter.net

Easystats Ltd, UK

27 March 2014

”The further back you can look, thefurther forward you are likely to see.”

– Churchill

Source: Significance magazine

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 2 / 17

HardwareFrom the slide-rule through punch-cards to the cloud

”This apparatus [Hollerith 1890 Census Tabulator] works unerringly as the millsof the gods, but beats them hollow as to speed.”– The Electrical Engineer, 11 Nov 1891.

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 3 / 17

Programming EnvironmentAssembly, Fortran, C(++), S(+), Java, R, Python, Julia, JavaScript

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 4 / 17

User InterfaceProgramming in console, Graphical User Interface, IDE, Software-as-a-Service

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 5 / 17

Reporting toolsA somewhat skewed cross table

Past Present Future (?)

Hardware Mainframes PC, servers Cloud, mobileSoftware FORTRAN R, Python, D3 Julia, JS

Users Statisticians Data scientists EveryoneStats/IT skills High Medium to high Low to high

User interface console IDE, GUI browserMain content tables interactive plots text

Reporting lang. LaTeX HTML markdownMain tool program function template

Workflow writing reproducible literatereports research programming

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 6 / 17

What does “pander” do inside of “rapport”?A tool for literate programming that automatically transforms R objects into markdown

# A quick analyis on mtcars

<% for (v in names(mtcars)) { %>

The mean of <%= v %> is <%= mean(mtcars[, v]) %> and the standarddeviation is <%= sd(mtcars[, v]) %>. Let us also check thefrequency table:

<%= table(mtcars[, v]) %>

## Tables are boring!

<%=set.caption(paste("Histogram of", v))hist(mtcars[, v], xlab = v, col = sample(colors(), 1), main = "")%>

<% } %>

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 7 / 17

What does “pander” do inside of “rapport”?Pandoc.brew: markdown results

# A quick analyis on mtcars

The mean of am is _0.4062_ and the standarddeviation is _0.499_. Let us also check thefrequency table:

-------0 1

--- ---19 13-------

## Tables are boring!


...

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 8 / 17

What does “pander” do inside of “rapport”?Pandoc.brew: calling Pandoc to convert the results to HTML

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 9 / 17

What does “pander” do inside of “rapport”?Pandoc.brew: calling Pandoc to convert the results to MS docx

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 10 / 17

“rapport” overviewThe overall structure of a reusable report template

<!--headmeta:title: ...author: ...description: ...packages:...

inputs:- name: ...class: ...

head-->

<% for (...) { %>

## Subtitle with <%= inline code chunk %>

<%= table(...) %>

<% } %>

http://rapport-package.info/Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 11 / 17

“rapport” demoHeader: meta information on the reporting template and input(s) specification

<!--headmeta:title: Rapport demoauthor: daroczigdescription: This is POC demo on the usage of rapport templatespackages:- ggplot2- pander

inputs:- name: v

label: Variable to analyserequired: yesclass: numericlength:

min: 1.0max: 1.0

- name: colorlabel: Color of the histogramstandalone: yesvalue: redclass: character

head-->

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 12 / 17

“rapport” demoBody: normal text with brew-style code chunks

# A quick analyis on <%= v.name %>

The mean of <%= v.name %> is <%= mean(v) %> and thestandard deviation is <%= sd(v) %>. Let us alsocheck the frequency table:

<%= table(v) %>

## Tables are boring!

<%=set.caption(paste(’Histogram of’, v.name))hist(v, xlab = v, col = color, main = ’’)%>

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 13 / 17

“rapport” demo

> rapport(’rapport-demo’, data = mtcars, v = ’am’)

# A quick analyis on _am_

The mean of am is _0.4062_ and thestandard deviation is _0.499_. Let us alsocheck the frequency table:

-------0 1

--- ---19 13-------

## Tables are boring!


Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 14 / 17

Online demoAnalysing the results of a dialect survey with Google Maps and ordinary English language

http://blog.rapporter.net/2013/07/uk-dialect-maps.html

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 15 / 17

Online demoAnalysing the results of a dialect survey with Google Maps and ordinary English language

http://blog.rapporter.net/2013/07/uk-dialect-maps.html

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 16 / 17

Rapporter packagesReferences: integrating the reporting framework in web applications

pander: A Pandoc’s markdown writer in Rrapport: A report templating system with dynamic inputs

rapportools: Helpers functionssandboxR: Filtering ”malicious” R calls

Further documentation:

http://rapport-package.info/

http://rapporter.github.io/pander/

http://hackme.rapporter.net

Q & A: daroczig@rapporter.net

Gergely Daróczi (rapporter.net) Creating statistical reports then and now 27/3/2014 17 / 17