learning bayesian networks for managing inventory of display advertisements max chickering mad...
DESCRIPTION
AdExpert Microsoft’s System for Delivering Display Advertisements Microsoft Properties Only 7.5 Billion Impressions/Day $1 Billion/Year Revenue Microsoft’s System for Delivering Display Advertisements Microsoft Properties Only 7.5 Billion Impressions/Day $1 Billion/Year RevenueTRANSCRIPT
![Page 1: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/1.jpg)
Learning Bayesian Networks For Managing Inventory Of Display Advertisements
Max ChickeringMad ScientistLive LabsMicrosoft Corporation
![Page 2: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/2.jpg)
Display Advertisements
![Page 3: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/3.jpg)
AdExpert
Microsoft’s System for DeliveringDisplay AdvertisementsMicrosoft Properties Only7.5 Billion Impressions/Day$1 Billion/Year Revenue
![Page 4: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/4.jpg)
AdExpert: Inventory
Health Pages Top
HealthPagesSide
Inventory consists of impressionsof targetable attributes:
1. Page Groups
Set of pages + position~6000 page groups
2. Geographic Targeting3. Demographic Targeting4. Behavioral Tags
Examples:
• 1M imp of males on the HealthTop page group• 1M imp of sports enthusiast on the AutoSide page group
![Page 5: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/5.jpg)
AdExpert: Selling Inventory
Charge per impressionCost depends on page group and targets
High-touch marketInventory is guaranteed
![Page 6: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/6.jpg)
Guarantees Result InInventory Management Problems
Pricing: How much do we chargeper impression?Remaining Inventory: Can we fill this order?Selection: Do we want to?(something better coming)Delivery: Given that we have overbooked, how do we prioritize orders?
![Page 7: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/7.jpg)
Capacity Prediction
Capacity PredictionCapacity Prediction
Pricing Pricing RemainingRemainingInventoryInventory SelectionSelection DeliveryDelivery
How many Old Males are coming next week?How many Old Males are coming next week?
![Page 8: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/8.jpg)
Capacity Prediction
Example: on a particular page group…Example: on a particular page group…
• Existing order: 1.2M impressions of OldExisting order: 1.2M impressions of Old• New customer wants 1.2M impressions of MalesNew customer wants 1.2M impressions of Males
Can we satisfy new request?Can we satisfy new request?
OldOld MaleMale
1.5M1.5M 1.5M1.5M
YesYes
OldOld MaleMaleNoNo
0.5M0.5M1M1M
0.5M0.5M
![Page 9: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/9.jpg)
Capacity Prediction
OldOld
MaleMale
Autos FanAutos Fan
Sports FanSports Fan
LocationLocation
InvestorInvestor
![Page 10: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/10.jpg)
How Many Old Males Next Week? Age Gender Sports
Old Male No
Young Female No
Old Male Yes p(Age,Gender,Sports)
Capacity PredictionCapacity Prediction==
Volume Prediction Volume Prediction XX
Population PredictionPopulation Predictionp(Age=Old,Gender=Male)p(Age=Old,Gender=Male)
Past Volume
Prediction
Volume PredictionVolume Prediction
Population PredictionPopulation Prediction
![Page 11: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/11.jpg)
Capacity Prediction In Earlier System
Age Gender SportsOld Male Yes
Old Female No
Young Male No
Young Female Yes
Old Male No
Young Female Yes
Young Male Yes
Old Female No
Young Male Yes
RandomRandomSampleSample
NMaleOldNMaleOldp ),(),(
92
Old Male No
Young Female No
Old Male Yes
Not Many TargetsNot Many Targets
![Page 12: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/12.jpg)
New Version Of AdExpert:Increase Targeting
Current System Maxed Out
Earlier system could not handle any more targeting
Competitors adding more targeting
New Demographic Targets
Add Behavioral Targets300 Targets300 Targets
![Page 13: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/13.jpg)
Capacity Prediction From Sample
Age Gender B1 B2 … BN SportsOld Male Yes
Old Female No
Young Male No
Young Female Yes
Old Male No
Young Female Yes
Young Male Yes
Old Female No
Young Male Yes
300 Variables300 Variables
Millions Millions of of
SamplesSamples
x 6000!x 6000!
![Page 14: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/14.jpg)
Compressing Tables With Bayesian Networks
Age Gender SportsOld Male Yes
Old Female No
Young Male YesAge Gender Sports
• One node for each columnOne node for each column• Edges represent probabilistic dependenceEdges represent probabilistic dependence• Each node stores p(node|parents)Each node stores p(node|parents)• Joint probability: product of conditionals:Joint probability: product of conditionals:
p(Age, Sports, Gender)=p(Age) x p(Gender) x p(Sports|Gender)p(Age, Sports, Gender)=p(Age) x p(Gender) x p(Sports|Gender)
• More independence leads to more compressionMore independence leads to more compression
Bayesian networkBayesian network: : Graphical model for representing a joint probability distributionGraphical model for representing a joint probability distribution
p(Age)p(Age) p(Gender|Age)p(Gender|Age) p(Sports | Gender)p(Sports | Gender)
![Page 15: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/15.jpg)
10101414 possible combinations possible combinationsOnly 119,350 parameters Only 119,350 parameters
Bayesian Network For Hotmail PG
![Page 16: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/16.jpg)
Training:Constructing The Model From Data
TrainingTraining
Age Gender B1 B2 … BN SportsOld Male Yes
Old Female No
Young Male No
Young Female Yes
Old Male No
Old Female No
Young Male Yes
Age
Gender
B1
B3
B2
Sports
Efficient “Look up” Algorithms: p(Age=Old, Efficient “Look up” Algorithms: p(Age=Old, Gender=Male)Gender=Male)
Pre-release Validation: Pre-release Validation: Accuracy better than existing systemAccuracy better than existing systemTiming requirements metTiming requirements met
OFFLINEOFFLINE
![Page 17: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/17.jpg)
Bayesian Network: Updating Over Time
Easy to UpdateEasy to UpdateLocal ProbabilitiesLocal Probabilities Ag
e
Gender
B1
B3
B2
Sports
Old Male No
Young Female No
Old Male Yes
![Page 18: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/18.jpg)
Current Status
Capacity Prediction is working wellValuable inventory is still selling outFewer under-delivered targeted orders
Targeting is increasing
![Page 19: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/19.jpg)
Lessons Learned (I)
Include cost of probability “look up” in learning algorithmInclude cost of probability “look up” in learning algorithm
![Page 20: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/20.jpg)
Lessons Learned (II)
Allow “preferred edges” – Some dependences areAllow “preferred edges” – Some dependences areapriori importantapriori important
Age
Gender
B1
B3
B2
Sports
![Page 21: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/21.jpg)
Questions?
![Page 22: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/22.jpg)
© 2006 Microsoft Corporation. All rights reserved.Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation.Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft,and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
![Page 23: Learning Bayesian Networks For Managing Inventory Of Display Advertisements Max Chickering Mad Scientist Live Labs Microsoft Corporation Max Chickering](https://reader034.vdocuments.site/reader034/viewer/2022051123/5a4d1b107f8b9ab05998f260/html5/thumbnails/23.jpg)