a/b testing with react · o introduction o our project • about the project • the stack •...

A/B Testing with React

Eivind Arvesen and Lars Dølvik

o Introductiono Our project

• About the project• The stack• Hypothesis- and goal-driven development/Workflow

o Story-timeo What is A/B testing?

• react-ab-test (library)▪ Isomorphism

o Evaluation• Measurements (GA: scrolling and events…)• Interpreting the numbers

o Practical demonstration

Outline

o A recipe focused application

o Runs on Google Cloud Platformo Isomorphico Built with React and Relay in fronto GraphQL as data query languageo Uses ElasticSearch to hold our contento Enonic CMS to publish content

o Hypothesis- and goal-driven development/Workflow

Our project

o Tactical goalo Action What?o Hypothesis Why?o Measurement How?o Estimated effect

Workflow

Story-time

Story-time

A/B-TESTING!

A form of statistical hypothesis testing.

What is A/B testing?

Illustration by Maxime Lorant

Usual steps:

● Have an idea or supposition about our solution● Verify or falsify the hypothesis via experiment(s)

○ Collect various measurements

What is A/B testing?

Why?

It’s normal to be wrong.

Measuring how our development impacts our users makes us able to make decisions based on facts!

What is A/B-testing?

React-AB-Test (on npm)

https://github.com/pushtell/react-ab-test

What is A/B-testing?

Legg gjerne inn et portrett av personen du siterer på dette lysbildet!

”The only true wisdom is in knowing you know nothing.Socrates - allegedly...

● Be skeptical● Don’t measure too much● Have a good basis for comparison

o Google Analytics APIo Scroll and time trackingo Event tracking

o Tracking the correct evento Follow a standardo Write documentationo Less is more(ish)o Evaluating the numbers

Evaluation

o A good grounds for comparisono Avoid noise in your datao Supplement qualitatively...

"Lies, damned lies, and statistics"

Evaluation

Evaluation

Let’s say one of the media elements is an autoplaying video. Clicks lead to another recipe.

What is the user’s real intent?

Evaluation

Evaluation

Let’s see how we can perform A/B-testing in practice: look at code and run it in the browser.

https://github.com/eivind88/ab-testing-react-meetup/tree/master

Demo

o It is not as complicated as it might seem• The thought process is more the half the job

o Keep it simple and structured• Follow a standard and write documentation

o Keep your experiments limitedo You don’t know it if you haven’t proven it

eivind.arvesen@bouvet.no

lars.dolvik@bouvet.no

In summary