The Insider’s Guide To Running Amazon Split Tests

1

Tuesday, August 9th

A Fun Game:Which Test Won? **These are not the exact products used, but representative products to preserve the sellers’ privacy.

2

Enter Your Guesses In The Survey...

3

4

Main Feature Image Test: Larger Product vs. Smaller Product + Include Packaging

Variation A

Variation B

5

Sale Price vs. No Sale Price

Variation A

Variation B

6

Detailed Graphics vs. Lifestyle Product Image

Variation A

Variation B

And The Winners Are...

7

Improved Daily Profit By 45%Improved Conversion Rate 17%

Larger Product, No Packaging

Improved Conversion Rate 21%

No Sale Price

Improved Sales by 22% Improved Conversion Rate

19%

Infographic

The Takeaway:

You never know what will perform best…..until you run tests with actual data!

8

9

Amazon is getting more competitive and crowded. Optimizing your listing is ever more important.

9,000 people selling “bbq gloves”.

How can you compete?

Answer: Run Split Tests

What is split testing?

Controlled | Randomized | Experiment

Between two or more variations...

10

Will lead to Improved Metrics (ie click through rate,

conversions, profits, etc)

How Split Testing WorksControl (original)

Variation

Distribute traffic between original and variation (visitor doesn’t know they are part of a test)

Accumulate enough data to identify a *statistically significant* change in visitor behavior

“Statistically Significant” is the likelihood that a change (ie conversion rate) is not due to random chance.

This gives you confidence that the changes you make actually have an impact!

11

Challenges to split testing on Amazon:

↪ You must work within Amazon’s guidelines (character limits, formatting, etc)

↪ All traffic must go to only one page

↪ Amazon only gives traffic data on a daily basis (therefore must make changes daily at 12 midnight)

12

High Impact Test #1: Price

In a low-priced niche (ie price sensitive consumer), would a change in price change the profits?

13

↪ Large changes +/- $2.00

↪ Small changes $x.97 vs $x.88 vs. $x.99

Recommended Price Tests

14

↪ Highly Competitive Niche - Try small incremental changes

↪ Low Competition - Try Increasing your price by $1 or $2

↪ Differentiated/Improved Product - Increase price by more than you think

Pricing Example: A cheaper price will generate more clicks and sales

$18.60

Sponsored AdsHypothesis:

Adjusting pricing (to $9.99) will generate more clicks, and sales.

I may make less profit per unit sold, but overall will get more sales.

Ultimately, I should increase my profits.

15

$19.99

$14.99

$18.99

$9.99

High Impact Test #2: Main Feature Image

Top search results for “bocce set”

Could the top organic result get a higher click through rate with a different image?

With an undifferentiated product (ie colored balls), would a change in main image increase the click through rate?

16

Sponsored Ads

Image Test: Make it appear a lot is included with your item

You can combine images of your product to feature different aspects.

Variation A shows a large version of the balls, as well as the carrying case.

17

Control Variation A

Image Test: Example 2

VS

18

Image Test: Bring the Product To Life

19

Instead of (photoshopped) two-dimensional images, try showing it in action.

Minimal Propping is OK in main image as long as it does not confuse the customer.

Image Test: “Explain” Your Product Visually

20

Use diagrams and notations to show the differentiating features of your product. This helps quickly convey information that they may not read in your listing’s text.

vs.

vs.

Image Test: Show Interesting Product Photos

21

Roasting sticks would be just a line if you included the whole image.

Competitors Included the whole packaging.

Our solution: Stand out with a unique angle, product composition, close-up image.

Split Testing Best Practices

Run your test until 90% or 95% statistical significance - minimum 2 weeks (ie know that your changes will have a positive impact!)

❗Only test one thing at a time.

❗Start with easier tests (price and image) vs. more nuanced tests (product description)

❗Switch variations every day (at midnight) to account for day-to-day or seasonal changes

22

Automate Your Split Testing with Splitly

Results from a recent pricing test.

Hypothesis: decreasing the price by a dollar would increase sessions, conversion rate, and sales.

29% more page views 66% more sales

33% higher conversion rate 32% HIGHER PROFITS!

23

Splitly Data: Rotation Log

24

Review the key metrics for every variant on a given day...

Splitly Data: Test Summary

Or view the test summary in a

simple dashboard.

The Variation showed

improvements across the

board.

This is a good thing!

25

Impact on Profits from Test ResultsFor example, Jungle Stix sells $16,000/month in sales, $4,800/profit

33% increase in profits =

$1584 per month, $19,000 per year, straight to the bottom line!

Continue to optimize for different metrics (CTR, Conv. Rate, Impressions)

Run automated tests on each product you sell

26

Try Splitly Today:Get 50% Off Your First Month (exclusive offer through 8/12 at 11:59p PST)

www.splitly.com/js27