research frontier of real-time bidding based display ... · research frontier of real-time bidding...
TRANSCRIPT
![Page 1: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/1.jpg)
Research Frontier of Real-Time Bidding based Display Advertising
Weinan Zhang
University College London
http://www0.cs.ucl.ac.uk/staff/w.zhang
August 2015
![Page 2: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/2.jpg)
Basic RTB Process
2
RTB
Ad
Exchange
Demand-Side
Platform
Advertiser
Data
Management
Platform
0. Ad Request
1. Bid Request (user, page, context)
2. Bid Response (ad, bid price)
3. Ad Auction 4. Win Notice (charged price)
5. Ad (with tracking)
6. User Feedback (click, conversion)
User Information
User Demography:
Male, 26, Student
User Segmentations:
Ad science, London traveling
Page
User
![Page 3: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/3.jpg)
Model Bidding Strategy
• A function mapping from bid request feature space to a bid price
• Design this function to optimise the advertising key performance indicators (KPIs)
Bid Request (user, ad, page, context)
Bid Price
Bidding
Strategy
3
![Page 4: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/4.jpg)
Bidding Strategy in Practice
Bid Request (user, ad,
page, context)
Bid Price
Bidding Strategy
Feature Eng. Whitelist /
Blacklist
Retargeting
Budget
Pacing
Bid
Landscape
Bid
Calculation
Frequency
Capping CTR / CVR
Estimation
Campaign
Pricing
Scheme
4
![Page 5: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/5.jpg)
Bidding Strategy in Practice: New Perspective
Bid Request (user, ad,
page, context)
Bid Price
Bidding Strategy
Utility
Estimation
Cost
Estimation
Preprocessing
Bidding Function
CTR, CVR,
revenue
Bid landscape
5
![Page 6: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/6.jpg)
Discussed Topics of This Talk
Fundamentals
• CTR/CVR Estimation
• Bid Landscape Forecasting
• Bidding Strategies
Advances
• Arbitrage
• Unbiased Training and Optimisation
• Conversion Attribution
![Page 7: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/7.jpg)
CTR/CVR Estimation
• A seriously unbalanced-label binary regression problem
– Negative down sampling, calibration
• Logistic Regression
[Lee et al. Estimating Conversion Rate in Display Advertising from Past Performance
Data. KDD 12]
![Page 8: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/8.jpg)
CTR/CVR Estimation
• Follow-The-Regularised-Leader (FTRL) regression [McMahan et al. Ad Click Prediction : a View from the Trenches. KDD 13]
Closed-form solution
![Page 9: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/9.jpg)
CTR/CVR Estimation
• Factorisation Machines
– Explicitly model feature interactions
– Empirically better than logistic regression
– A new way for user profiling
• GBDT+FM
[Oentaryo et al. Predicting response in mobile advertising with hierarchical
importance-aware factorization machine. WSDM 14]
[http://www.csie.ntu.edu.tw/~r01922136/kaggle-2014-criteo.pdf]
![Page 10: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/10.jpg)
Deep Learning Models [our working project]
![Page 11: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/11.jpg)
Bid Landscape Forecasting
Auction
Winning
Probability
Win probability: Expected cost:
Count
Win bid
![Page 12: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/12.jpg)
Bid Landscape Forecasting
• Log-Normal Distribution [Cui et al. Bid Landscape Forecasting in Online Ad Exchange Marketplace. KDD 11]
Auction
Winning
Probability
![Page 13: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/13.jpg)
Bid Landscape Forecasting
• Price Prediction via Linear Regression [Wu et al. Predicting Winning Price in Real Time Bidding with Censored Data. KDD 15]
– Modelling censored data in lost bid requests
![Page 14: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/14.jpg)
Bidding Strategies
• How much to bid for each bid request?
• Bid to optimise the KPI with budget constraint
Bid Request (user, ad, page, context)
Bid Price
Bidding
Strategy
![Page 15: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/15.jpg)
Bidding Strategies
• Truthful bidding in second-price auction
– Bid the true value of the impression
• Non-truthful linear bidding
– With budget and volume consideration
[Chen et al. Real-time bidding algorithms for performance-based display
ad allocation. KDD 11]
[Perlich et al. Bid Optimizing and Inventory Scoring in Targeted Online
Advertising. KDD 12]
![Page 16: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/16.jpg)
Bidding Strategies
• Direct functional optimisation
• Solution: Calculus of variations
CTR winning function
bidding function
budget
Est. volume
16
[Zhang et al. Optimal real-time bidding for display advertising. KDD 14]
![Page 17: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/17.jpg)
Optimal Bidding Strategy Solution
17 [Zhang et al. Optimal real-time bidding for display advertising. KDD 14]
![Page 18: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/18.jpg)
Overall Performance – Optimising Clicks or Conversions
18
iPinYou dataset
[Zhang et al. Optimal real-time bidding for display advertising. KDD 14]
![Page 19: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/19.jpg)
Discussed Topics of This Talk
Fundamentals
• CTR/CVR Estimation
• Bid Landscape Forecasting
• Bidding Strategies
Advances
• Arbitrage
• Unbiased Training and Optimisation
• Conversion Attribution
![Page 20: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/20.jpg)
Discussed Topics of This Talk
Fundamentals
• CTR/CVR Estimation
• Bid Landscape Forecasting
• Bidding Strategies
Advances
• Arbitrage
• Unbiased Training and Optimisation
• Conversion Attribution
![Page 21: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/21.jpg)
Display Advertising Intermediaries
This work: Intermediary arbitrage algorithms in RTB display advertising. 21
[Zhang et al. Statistical Arbitrage Mining for Display Advertising. KDD 15]
![Page 22: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/22.jpg)
Intermediary’s Statistical Arbitrage via RTB
• Statistical arbitrage opportunity occurs, e.g., when
(CPM) cost per conversion < (CPA) payoff per conversion
1000 impressions * 5 cent < 8000 cent for 1 conversion 22
![Page 23: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/23.jpg)
Statistical Arbitrage Mining
• Expected utility (net profit) and cost on multiple campaigns
CVR estimation winning function
bidding function
Cost upper bound
Est. payoff
Prob. of selecting Campaign i
Bid request vol.
23
![Page 24: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/24.jpg)
• Optimising net profit by tuning bidding function and campaign volume allocation
Statistical Arbitrage Mining
Total cost constraint
Risk control
E-Step
M-Step
Total arbitrage net profit
• Solve it in an EM fashion
24
![Page 25: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/25.jpg)
M-Step: Bidding function optimisation
• Fix v and tune b()
25
![Page 26: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/26.jpg)
E-Step: Campaign volume allocation
• Multi-campaign portfolio optimisation
where
Portfolio margin variance
Portfolio margin mean
Net profit margin on each campaign
26
![Page 27: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/27.jpg)
Campaign Portfolio Optimisation Results
27
![Page 28: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/28.jpg)
Dynamic Portfolio Optimisation
28
![Page 29: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/29.jpg)
Online A/B Test on BigTree™ DSP
• 23 hours, 13-14 Feb. 2015, with $60 budget each 29
![Page 30: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/30.jpg)
Discussed Topics of This Talk
Fundamentals
• CTR/CVR Estimation
• Bid Landscape Forecasting
• Bidding Strategies
Advances
• Arbitrage
• Unbiased Training and Optimisation
• Conversion Attribution
![Page 31: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/31.jpg)
Problem of Training Data Bias
• Data observation process
• We want to train the model
• But we train on the biased data
A bid
request Bid
Data
observation
If win
[Zhang et al. Learning and Optimisation with Censored Auction Data in Display Advertising. AAAI 2016 Submission]
![Page 32: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/32.jpg)
Unbiased Training
• Eliminate the data bias via importance sampling
• Training target
• Modelling winning probability via bid landscape
![Page 33: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/33.jpg)
Unbiased Training
• Modelling winning probability via bid landscape
• Only use observed impression data [UOMP]
• Also use lost bid request data (censored data) [KMMP]
nj: # {winning prices > bj} dj: # {winning prices = bj}
![Page 34: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/34.jpg)
Experimental Results
• Winning probability estimation
![Page 35: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/35.jpg)
Experimental Results
• CTR estimation: immediate performance improvement
![Page 36: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/36.jpg)
Discussed Topics of This Talk
Fundamentals
• CTR/CVR Estimation
• Bid Landscape Forecasting
• Bidding Strategies
Advances
• Arbitrage
• Unbiased Training and Optimisation
• Conversion Attribution
![Page 37: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/37.jpg)
Conversion Attribution
• Assign credit% to each channel according to contribution
• Current solution: last-touch attribution [Shao et al. Data-driven multi-touch attribution models. KDD 11]
![Page 38: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/38.jpg)
Multi-Touch Attribution
• How to estimate the contribution of each channel?
[Shao et al. Data-driven multi-touch attribution models. KDD 11]
• A more general formula
[Dalessandro et al. Casually Motivated Attribution for Online Advertising.
ADKDD 11]
![Page 39: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/39.jpg)
[Shao et al. Data-driven multi-touch attribution models. KDD 11]
![Page 40: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/40.jpg)
Bidding in Multi-Touch Attribution Mechanism
• Current bidding strategy
– Driven by last-touch attribution
• A new bidding strategy
– Driven by multi-touch attribution
[Xu et al. Lift-Based Bidding in Ad Selection. ArXiv 1507.04811. 2015]
![Page 41: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/41.jpg)
Value-based bidding v.s. Lift-based bidding
![Page 42: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/42.jpg)
Value-based bidding v.s. Lift-based bidding
• Comparison
– Lift-based bidding help brings more conversions to advertisers
– but its eCPA is higher than value-based bidding because of last-touch attribution
• Lift-based bidding with multi-touch attribution could bring a better eco-system
![Page 43: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/43.jpg)
Taking-home Messages
• Statistical Arbitrage Mining: The internal auction selects the ad with highest arbitrage margin instead of the highest bid price.
• Unbiased Training: Add the weight to each instance to eliminate the auction-selection bias.
• Attribution and Bidding: Bidding proportional to the CVR lift instead of CVR value.
![Page 44: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/44.jpg)
Computational Advertising Research in Academia
Disadvantages
• Lack of data and online test platform
• Lack of specific domain knowledge
Advantages
• Good at mathematic modelling
• Focus on knowledge collection and communication
• More research human resource
![Page 45: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/45.jpg)
OpenBidder Project: www.openbidder.com
• Online open-source benchmarking project
– Bid optimisation, CTR estimation, Bid landscape etc.
• Bridge academia and industry research on computational advertising
45
![Page 46: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/46.jpg)
Collaborations
• Collaborations are more than welcome!
UK: US:
CN:
![Page 47: Research Frontier of Real-Time Bidding based Display ... · Research Frontier of Real-Time Bidding based Display Advertising ... •Arbitrage •Unbiased Training ... •Factorisation](https://reader031.vdocuments.site/reader031/viewer/2022022015/5b589fbe7f8b9ad0048c0f09/html5/thumbnails/47.jpg)
Thank You! Questions?
http://www.computational-advertising.org http://www0.cs.ucl.ac.uk/staff/w.zhang
Ad Science WeChat Group