Site  Search  Analytics   1  

Site  Search  Analytics   1  

Site  Search  Analytics  White  paper  by  SearchBroker  -­‐  Colbenson  

John  Tomlinson  

This paper is about the challenge of using analytics to drive your findability strategy and

measure how well it’s achieving on its objectives.

Contents

Introduction .................................................................................................................................................................... 3

Putting the customer first .......................................................................................................................................... 3

The long neck or the long tail? .................................................................................................................................. 5

Search analytics to drive SEO........................................................................................................................................ 5

Site search is different ................................................................................................................................................... 5

Findability: the foundation of conversion.............................................................................................................. 6

Findability.......................................................................................................................................................................... 6

Search analytics: the long neck ................................................................................................................................. 9

Content or search terms?............................................................................................................................................... 9

Calculating opportunity ...............................................................................................................................................10

The findability ceiling ...................................................................................................................................................12

Search analytics: the long tail..................................................................................................................................14

Step one: create clusters .............................................................................................................................................14

Step two: low findability analysis...............................................................................................................................14

Step three: take action .................................................................................................................................................14

Low findability due to no content..............................................................................................................................14

Low findability due to non-indexed content..........................................................................................................15

Low findability due to customer behaviour...........................................................................................................15

Low findability for other reasons ............................................................................................................................15

Step four: don't stop.....................................................................................................................................................16

Conclusion......................................................................................................................................................................17

3

Introduction  

People have one thing in common: they are all

unique.

This means that people will search in lots of

different ways, and your onsite search engine

must be able to deal with it.

This paper is about that challenge. The challenge

of using analytics to drive your findability strategy

and measure how well it’s achieving on its

objectives.

Putting  the  customer  f irst   The proper place to start is with customer-focussed search analytics.

This means going through all the words (terms) and phrases (queries) and working out what people are

talking about.

Putting each terms or query into a graph, with quantity of searches on the vertical y-axis and each search

term (in quantity order) on the horizontal x-axis, you get something that looks a bit like this:

Roughly speaking, about 80% of your searches will fall into the top 20% of terms (we call this the long

neck), and then the other 80% of terms will be spread across about 20% of your traffic (the long tail).

This "long tail" is a broad range of search queries that tend to be a mix of words and phrases that will

usually have poor findability rates (ie. they don't connect the customer to any site content).

The tactics for dealing wth the long neck are different

from those of the long tail – but which is more

important? Where should you start?

5

 

The   long  neck  or  the   long  tai l?     Search analytics advice is usually focussed on the importance of the "long tail" in driving conversion.

Whilst the "long tail" is important, please be aware, a lot of this advice is focussed on Search Engine

Optimisation (SEO) and not on internal site search.

Search  analytics  to  drive  SEO   The argument runs that using high volume queries like "TV" is less effective in driving conversion than a

much more specific - but much less common - query like "LCD smart HD TV Samsung" for example.

This makes sense, the more specific the search is, the more likely it is that the customer will connect to

the data they are actually looking for, and the more likely it is that the customer is clear about what they

want and serious about finding it.

Many of us search this way. We try a fairly general query then, when faced with too much generic

content, we refine the queries getting more and more specific until we find links that connect us directly

to the content we're looking for.

The challenge of SEO is to get seen in external search engines like Google. In this case, the long-tail is a

great source of data to help you focus your advertising, make use of key words that convert, drive a

content marketing strategy, and other useful techniques for attracting valuable traffic.

Site  search  is  different   It is not the same challenge in site search. In site search the customer is already on the site, so the same

argument simply doesn't apply and a different approach is needed.

In site search, the challenge is to ensure that the customer makes the right connections to the right

data.

We may see a "long tail" query like "Samsung HD Smart LCD TV" with findability at zero, but this is much

less likely. If this is a product (or product type) you stock, then if it is not connecting it is probably due to

data quality or design issues rather than search. If you stock other types of HD TVs, a good search engine

- even a poor search engine - should make that connection!

The long-tail in site search is often more idiosyncratic: variations on terms, common mistakes or off-

catalogue items that you haven't indexed.

This is why search analytics always starts with the customer.

It is vital to use the customer's own language to drive how the search and category navigation functions

deliver. If you don't, you'll be building connections from a business perspective, not from a customer

perspective.

This is why the head of the graph, the long neck that are about 80% of the traffic, becomes hugely

important: that's where the customers are and in site search analytics, that’s where you need to start!

Findabil ity:  the  foundation  of  conversion   Before discussing the specific approaches you should take when dealing with the long neck and long tail

of site search, we need to define the term “findability”.

Findability is mapped on the vertical y-axis of the graph where each term or query has a findability value

associated with it. This is driving the analytics process, and so we must understand the term to be sure

we are building our methodology on solid foundations.

Findabil ity   The term "findability" is a cumbersome and inelegant word, but one that captures an important concept.

The first formal definition is often credited to Peter Morville:

"the ability of users to identify an appropriate Web site and navigate the pages of the site to

discover and retrieve relevant information resources"

(Peter Morville, 20051)

He wasn't the first to use the term, but his definition is most widely accepted.

This definition includes two ideas:

• The ability of the website and its content to be found by customers searching externally.

• The ability of the content of the website to be found by customers internally (i.e. already on the

site).

1 Source: http://findability.org/

7

The first part of the definition is now the field known as SEO: (Search Engine Optimization). This is about

how to ensure your site ranks in Google and other major search engines, and has become a vital tool for

online organisations and a complex science with its own devoted experts.

The second part of the definition is the focus of this paper. Can people who are already on your site

actually find what they're looking for?

This is increasingly important as big data becomes ever more commonplace, and especially so in e-

commerce. The ability of your e-commerce customers to find your products is the different between

making a sale and losing a customer.

This is why we at Colbenson define "findability" as:

"the measure for how easily your customers can make the right connections to your data"

This is findability, and it is the fundamental component of online conversion. Can your customers

connect to your data correctly - if not, they're not going to buy anything.

This also goes beyond site search.

Findability is not just about how onsite search works, it applies generally to how your customers connect

to your data. This could be via promotional advertising, category navigation, social media or any other

channel.

How customers make these connections will vary from sector to sector and from business to business.

This is why “findability” is such a useful concept, it doesn't focus on the tool (e.g. the onsite search

engine), it focuses on the outcome: customers making the right connections to your data, or, to put it

another way, it focuses on conversion.

Reducing Site Search abandons increases Conversion

9

Search  analytics:  the   long  neck   In site search, 80% of your customers are at the long neck; that's where you need to be!

The great thing about long neck search terms is that they are predictable. It doesn't take too much

effort to scan through the top bundle of search terms used, see what's happening, and create a plan for

the kind of connections you want to see made between customers using those terms and your site

content.

The best way to convert the long neck is by creating promotional links to either successful searches or

high-performing content. Or both.

But what links should we create?

The best way to decide is to let the customer decide by using A/B Testing on various different solutions.

A/B Testing

A/B Testing involves trying two or more options and randomly applying these to your customers at

the same times of day. You can then compare which is the most successful at building connections.

Content  or  search  terms?   A search engine like Google will link you to popular search terms by suggesting popular searches as you

type. This is called Google Suggest. In this case, the search is still performed and a page of results shown.

The customer needs to browse these results and select the content they want.

This is great when there is no obvious standout link between the term and the content.

If there is a strong relationship between a search term and site content, you can bypass the search and

results process and directly suggest that content to the customer is the autocomplete space. This means

the customer clicks and is taken directly to that content.

A great example of this is Apple:

Be careful with this, you can overdo it, and end up thinking all of your customers want to make the same

connections to the same content. You must always allow the customer to feel in control of the search,

and allow them to make the connections they want to make.

Calculating  opportunity   A useful way to measure this is to consider the concept of “opportunity”. This process allows you to

prioritise your actions when managing site search.

Opportunity

Opportunity is the measure of what you’re not converting – how many opportunities to sell are you

missing because your search is not making the right connections.

Sticking with the TV example from earlier in this paper, if we have a "long-tail" item like "telvision" that

has a findability of zero, and a frequency of 5, then we are potentially missing out on 5 connections and

therefore have 5 opportunites.

11

At the same time, if a thousand people are searching "TV" and this has a findability of 20%, then that's

800 people (1000 x 80%) not making connections to your products.

Looked at this way, you have a much greater opportunity to make an impact on your overall conversion

rates if you work out why people are not finding what they want when they search "TV" than connecting

"telvision" that had only 5 incidences.

The calculation for working out the number of opportunities (o) each search query presents us with is

simple: the number of incidences (n) multiplied by the percentage not found (1-f%):

o = n x (1-f%)

There is a problem with this method.

We are assuming that it is possible to get a findability percentage (f%) of 100%.

Sadly, we must reluctantly accept that there is a ceiling somewhat lower than perfection for most

queries. Some individual queries with very low frequency (usually pointing at specific content) may have

unusually high findability, but this anayisis uses he more numerous terms found in the "head" of the

graph.

This will vary from industry to industry, and from sector to

sector, but it is rare to see 100% findability - especially if your

results page (or autocomplete) has so much information that

customers need not click on anything to get the information!

The  f indabil ity  cei l ing   To estimate your findability ceiling, first isolate your top search queries. These should be the top ten (or

so) that have the best findability percentage, but that have a similar success rate. It needn't be exactly

ten queries, but you should be able to see a number of queries that cluster at the top with similar

percentages.

Now you need to drill down on the part that is not converting.

Staying customer-focussed, there are only two things that

can be happening if the customer is not connecting to

content. Either they are (a) giving up and abandoning the

search, or (b) they try again and search with new terms or

use filters to refine the results.

This is shown in this diagram. The green channel shows

conversion, the red is abandonment and the yellow is

customers trying again.

Let us assume that our top performing search queries

have an average findability of 40%, and that of the 60%

that's not connecting, it is split equally into 30%

abandoning the search and 30% refining or redoing.

We can now make an assumption that we should be able

to see a 30% improvement in (b) and a 10% improvement

in (a).

The difference is because redos usually signal that the person is not finding what they want to find (i.e. a

search problem), whereas abandonments may signal the customer being put off by a non-search-related

reason such as price, terms and conditions, payment methods, poor reviews, lack of information, or

simply poor page design.

So, adding this up we get the original 40%, plus 10% of abandonments (3%) plus 30% of redos (9%) and

a ceiling target to aim for of 52%. This is much more realistic than 100%. For those who like equations:

o = n x (fc%-f%) (The new term is fc% which is the percentage of the findability ceiling).

13

The queries should then be listed in order of opportunity, giving us a quick and simple priority list for

actions we need to take to drive up conversion.

In this way we are able to focus our efforts in the places where we will have the most impact.

 

Search  analytics:  the   long  tai l  

Although it makes sense to start with the long neck, it is still useful to look at the "long tail”.

This contrasts with the long tail where the search terms are more numerous and less predictable. For the

long tail we need the search engine to be really clever at ranking results organically - but in the long

neck, because we're working with fewer terms and we know what they are, we can take control and

create the links we want.

There is a four-step process for dealing with long tail analytics.

Step  one:  create  clusters   The first step is to examine the data, and where possible, create clusters of terms. If, for example, the

are a lot of terms like "postage", "delivery", "shipping", "packaging" or even errors such as "pakaging"

and "dliver" which are recognisable, then you can see there is demand for content about how products

are delivered.

Step  two:   low  findabil ity  analysis   There are only four reasons why findability is zero or very low:

1. You don't have content on your site that relates to what they are looking for.

2. You do have content, but didn't index it so it cannot be found with the search engine.

3. The customer misspelled, mistyped or described the content in a way that wasn't recognised

4. None of the above. For some other reason they did not choose to click on any result that was

offered.

For each cluster created in Step One (or each term where no clusters could be created), work out what

the reason for the low score is.

Step  three:  take  action   Each of the above four reasons requires a different approach.

Low  findabil ity  due  to  no  content   You have three choices:

1. Create content that includes these terms. This can mean adding a blog post, an FAQ, a technical

description ... it depends on the nature of the query which type of content makes sense.

15

2. Stock the item. If the customer is searching for an item you don't stock, this a great view into

potential demand. If it makes no sense to stock it, you may still be able to create content that

may be of interest and may keep the customer on the site, or you can link the term to a similar

item - perhaps via the "zero results page" ("We don't stock X, but we do have Y").

3. Don't worry about it. Sometimes people will search for things that just don't make any sense ...

but beware, sometimes what might seem to make no sense to you, does make sense to the

customer. Why would people suddenly search for Brad Pitt on the Chanel site? Or search for

Kate Moss on Mango’s site? Before dismissing the search term, ensure you're aware of all

promotional activity your organisation is involved in - there may be a connection there and the

right move is to create some content or direct the search to the relevant product.

Low  findabil ity  due  to  non-­‐indexed  content   This one is fairly obvious. All site content should be indexed, not just your product catalogue. Often

customers will search for store location, returns policies, how to make complaints, how the delivery

process works and other off-catalogue information. This should be there and should be indexed.

Low  findabil ity  due  to  customer  behaviour   This data is a gold mine of common mistakes that your customers will make and allows you to be

proactive about anticipating future errors. If they are often misspelling a word, you will find it here. If

they don't understand how your product is categorized, you'll find their version of the truth represented

in the words and phrases in this data.

Where it is possible to link these words and phrases to content, you should do so by including them in

product descriptions, keywords or synonyms.

The most important action you can take here is to ensure that your autocomplete function offers search

suggestions and content that avoid mistakes in the first place.

An alternative is to link them to similar words or phrases that have good rates of conversion. This is

popular with Google where they will use "Did you mean X?" and perform the search on X which they

know has some good results. If you do this, you must be clear that the results relate to X, and not what

the customer searched for.

A last point on this. Don't assume these are all customer errors, they could just as easily be your errors.

Challenge your own category structures, product names and descriptions: are you the one who is making

the mistakes?

Low  findabil ity  for  other  reasons   This is a very broad category, and could include many factors, not all of which may be negative.

Here are ten of the more common reasons:

• Poor results page design.

• Lack of, or poorly designed, autocomplete function.

• Poor zero results page.

• Poor data quality - the right content is not being offered.

• Lack of, or poorly designed, filters.

• Ineffective dynamic ranking of results.

• Poor or confusing product descriptions so that the customer cannot correctly recognise the

product.

• The lack of appropriate use of images and other visuals.

• So much information shown in the results page or autocomplete that it becomes unnecessary

for the customer to click on anything (not necessarily a bad thing!)

• Non-search related reasons such as wrong pricing, confusing descriptions, generally poor design

destroying customer confidence.

Step  four:  don't  stop   This process is not a straight line, it's a circle. Once these actions have been taken, the process starts

again.

17

Conclusion   A good search engine succeeds because it is part of a customer-centric findability strategy. It is focussed

on helping your customers connect with your content, and measured by how well it achieves this

objective.

The process for acting on these analytics involves taking the long neck and the long tail of search terms

separately. Each is then dealt with differently, and always with the objective of driving up findability and

so increasing the connections your customers make to your content.

This is the foundation of conversion. If your customers can’t connect to the right content, they can’t buy

it.

15 Queen Square, LEEDS, LS2 8AJ United Kingdom

0113 335 0791 / 08454 757 973

http://www.colbenson.com