data is never accurate - and that's ok!

“Errors using inadequate data are much less than those using no data at all”- Charles Babbage

The data is never accurate – and that’s OK!

By: Chanpreet Singh| twitter.com/chanpreets

Part 1: Why is data not accurate?

Part 2: How to ensure data sanity?

Part 3: How do we work with this kind of data?

May 1, 2023 Chanpreet 2

Tracking toolGoogle Analytics• Integrate an Analytics tool – like Google Analytics (GA)


Part 1: Why is data not accurate?


What is the meaning of Accurate

Towards care• (especially of information, measurements,

or predictions) correct in all details; exact.• “Done with care”• Digital measurement - ‘care’ unfortunately

is not enough


Why would data not be AccurateFactors responsible• Digital Analytics programs do not

collect exact number of hits or queries• Designed to give ‘valuable insights’• Digital Analytics is not an Audit• Inherent uncertainties• Human nature


Not designed to collect exact hits or queriesJS limitations

• Different layout engines render JS differently


Layout Engine Browser

TridentInternet Explorer for WindowsMaxthonNetscape 8.1

Tasman Internet Explorer 5 for Macintosh

Gecko

FirefoxNetscape 6 and laterMozillaCaminoK-MeleonSeaMonkeyEpiphany 2.20 and beforeGaleon

KHTML KonquerorTkhtml Html Viewer

Layout Engine Browser

WebKit

SafariChromeiCab 4 and laterEpiphany 2.26 and laterMaxthon 3OmniWebShiiraMidori

Presto OperaiCab iCab 3 and beforeTkhtml Html Viewer

Not designed to collect exact hits or queriesCookies• Cookies

blocked or disabled by users


Not designed to collect exact hits or queriesCookies• Cookie

deletion


Not designed to collect exact hits or queries<noscript>• <noscript> tag


Insights through % of data collectedDigital landscape

• Digital landscape is ever changing• Multiple platforms• Basic functional mobile phones• Networks, CDNs and other systems• Considerations of Outliers and Bots• Sample data to extrapolate performance


Not an AuditAudit involves• Series of checks and balances• Accuracy• Standardized metrics and

methodology• Consistency of process• Transparency of results


Inherent uncertaintiesUncertainties of Digital Analytics

• Algorithms• Models• Data itself• Outcomes


Human NatureHuman factor at play• Humans think differently• Big organization, big teams, multiple functions• Cross functional/ team dependencies• Dependencies on partners• Lack of proper understanding


Human NatureTracking Code• No tracking code


Human NatureTracking Code• Exclusion/ removal of tracking code


Human NatureHuman factor at play• Some users disable JavaScript


Human NatureCommunication• Inter team communication gap


Human NatureTeam conflict• Product and technology teams


Human NatureUTM• Campaign tagging


http://www.visitpa.com/pa-hotels-motels?utm_campaign=EVG_PA_Primary_Lodging&utm_source=google&utm_medium=cpc&utm_term=places-to-stay&utm_content=family-ad

Human NatureEvents• Event tagging


Human NatureAccess• Access levels and profile view referred


AccountPropertyView

Part 2: How to ensure data sanity?


AwarenessBe Aware of differences• Discrepancies vs Differences• Different tools - different data• Sampling• Purpose of tools• Metrics used


Different tools different data – GA vs AdWords Metrics usage and nature differs

• Clicks vs Sessions • AdWords filters invalid clicks, Analytics shows

all data• Landing page might redirect to another• Users browser preferences• Users return during the lifetime of campaign• Users return to your site via bookmarks


Different tools different data – GA vs DFP Metrics usage and nature differs

• DFP counts ad clicks at the source; GA counts pageviews or sessions when a user hits the site

• Metrics are counted at different points in the click-referral cycle


Different tools different data – Real time GA vs Another (Chartbeat)Common to use multiple tools• Chartbeat checks in for visitors every

few secs.• GA checks in once for visitors every

5 mins.• GA > Chartbeat = concurrent users

are high, but not staying long• GA < Chartbeat = most users stay

longer than 5 mins.


Different tools different data – GA vs ComScore

Competition reporting• ComScore• Eliminates/ignores machine generated traffic• Measures people • Eliminates visits from multiple browsers,

multiple screens• Extrapolates numbers based on a panel users • 3 second rule


Limitations within the tool – GA vs GA

GA Sampling• GA sampling and limitations• Standard vs Ad-Hoc reports• GA Standard vs Premium• High-cardinality dimensions• Report Query limit - 1 million rows of data• Data limits - single day/ multi day processed tables• Multi channel funnel report > 1 million conversions• Flow visualization report more than 100k sessions


Part 3: How do we work with this kind of data?


Best attempt to ensure better data Plan> Implement> configure> report> Interpret

• Coding• Admin Panel• Product• Interface usage and reporting• Interpretation


Ensuring correct implementation - CodeCoding• Updated version (UA)• Correct UAID• Correct domain/ multi domain coding• Ensure code is not Commented out• Check for code Breakages• Customizations (session duration, session end)• Placement of code• Duplication of code• Conflict with another code (other tool), events


Ensuring correct implementation - Interface

Admin Panel• Properties and Profile creation• Correct filters• Limitations/ Free_vs_Paid• Custom Dimensions & Metrics• Goals• Cross tool integrations/ agreements

(change)


Ensuring correct implementation - Product

Product Sanity (Website/ Mobile Apps)• URL/ Screen pattern and duplications• UTM tagging• Conflict with tech and tools• Version control• Audience platforms - Desktop vs Mobile• Cache management/ deletion/

exclusions• Bots/ malware• Geo location of audience and network• FirewallsMay 1, 2023 Chanpreet 34

Ensuring correct ReportingInterface usage and reporting• Report/ data pull/ interpretation• Sampling• Time period/ timelines• Data influencing Events

• Data Cleansing - Incomplete/ inaccurate data• Interpretations Differ • Blind to our own biases• Critical points that might influence another outcome

• Working with Limitations


Same data – different interpretation

Interpretation matters• Same data can yield wildly different

results• Redundant data stored in multiple

systems and in different formats• Data transformation is also important


FinallyDirectional Guidance

• Digital Analytics should be about providing directional guidance


Thank youReach me twitter/@chanpreets or email: [email protected]


data is never accurate - and that's ok!

Data & Analytics