predicting pre-click quality for native advertisements
TRANSCRIPT
Offensive ads disengage the users!
D. G. Goldstein, R. P. McAfee, and S. Suri. The cost of annoying ads. WWW 2013.
A. Goldfarb and C. Tucker. Online display adver#sing: Targe#ng and obtrusiveness. MarkeIng Science 2011.
How to measure the pre-‐click quality?
• Is CTR (click-‐through rate) a good pre-‐click
metric?
– A compounding metric:
• Relevance: how ads match user
interests.
• Quality: nature of the ad product and
ad creaIve design decision.
• Pre-‐click metrics solely measure on ad quality?
– Let us elicit from the users (crowdsourcing)
CTR vs. Offensiveness (OFR)
Bad ads a&rac)ng clicks (clickbaits?)
• Correlation between CTR and OFR (very weak)
– Spearman: 0.155 – Pearson: -0.043
• Quantile analysis
– High OFR ⇔ distribute across ads with various CTR
– Higher CTR ⇔ more ads with higher OFR
What makes an ad preferred by users?
● Methodology ○ Pair-‐wise ad preference + reasons ○ Sample ads with various CTR (whole
quality spectrum) ○ Quality based comparison
within category (verIcal)
● Underlying preference reasons ○ Aesthe#c appeal > Product, Brand,
Trustworthiness > Clarity > Layout ○ VerIcal Differences
personal finance (clarity) beauty and educaIon (product)
within verIcal comparison
Can we engineer ad quality features?
brand
readability, senIment
aestheIc, visual
User Reasons Engineerable Ad Crea#ve Features
Brand Brand (domain pagerank, search term popularity)
Product/Service Content (YCT, adult detector, image objects)
Trustworthiness
Psychology (senIment, psychological incenIves) Content Coherence (similarity between Itle and desc) Language Style (formality, punctuaIon, superlaIve) Language Usage (spam, hatespeech, click bait)
Clarity Readability (Flesch reading ease, num of complex words)
Layout Readability (num of sentences, words) Image ComposiIon (Presence of objects, symmetry)
Aesthe;c appeal Colors (H.S.V, Contrast, Pleasure) Textures (GLCM properIes) Photographic Quality (JPEG quality, sharpness)
○ By mining ad copy (Itle and descripIon), image and adverIser informaIon ○ Cold-‐start features
We also use historical features
User Behavior Engineerable Features
Click CTR (click-‐through rate)
Post-‐click Bounce Rate Average Dwell Time
We mine user interacIons with the ads
Feature correla#on with OFR
The offensive ads tend to: start with number maintain lower image JPEG quality be less formal express negaIve senIment in the ad Itle
Data NaIve mobile iOS and Android app 28,664 ads (Sampled from March 01-‐18, 2015) Ad feedback data obtained from Yahoo news stream
Classifier Logis;c Regression as a binary classifier posiIve examples: high quanIle of OFR ads
negaIve examples: all others
EvaluaIon 5-‐fold Cross-‐validaIon Metric: AUC (Area Under the ROC Curve)
Pre-‐click model: Data and evalua#on
brand
readability, sentiment
aesthetic, visual
Overview of model performance
Models based on each feature category:
product > trustworthiness > brand > aestheIc appeal > clarity > layout
Model summary:
• cold start: AUC (0.77)
• User behavior: AUC (0.70)
• cold start + user behavior: AUC (0.79)
A/B Tes#ng online evalua#on
• Baseline System – Score(ad) = bid * pCTR
• Pre-‐click Quality System
– Eliminate the ad from ad ranking if P(Offensive|ad) > 𝛿 – 𝛿 is determined by other constraints (e.g. eCPM)
Mobile: OFR (-‐17.6%) Desktop: OFR (-‐8.7%)
Take-‐away messages
• How to measure pre-‐click ad quality? – Offensive feedback rate as a metric – Capture bad quality be3er than CTR
• What makes an ad preferred by users (reasons)? – AestheIc appeal > Product, Brand, Trustworthiness > Clarity > Layout
• How to model? – Mining ad copy features from ad text, image and adverIser – EffecIve in the predicIon