mlr
DESCRIPTION
MLR. * Used Yahoo Simplified-preferred as baseline, since it has higher relevance than all-language and cn-preferred. Also both used CNB. MLR vs Yahoo. The only place where MLR does really bad is on Latin queries. If we fix this the gain could be much bigger. MLR by Language. - PowerPoint PPT PresentationTRANSCRIPT
MLR
DCG %P+E %Bad %P @ 1
Yahoo CN*
6.89 16.9% 14.6% 4.3%
MLR +2.4% +1% -13% +0
* Used Yahoo Simplified-preferred as baseline, since it has higher relevance than all-language and cn-preferred. Also both used CNB.
MLR vs YahooThe only place where MLR does really bad is on Latin queries. If we fix this the gain could be much bigger.
MLR by Language
Chinese Latin
Baseline 6.6 8.0
JP MLR +4% +1%
EN MLR 0% +3%
Google +3% +6%
Examples 509 91
MLR vs Yahoo by Length
Query Length MLR vs Yahoo
# of Queries
2 -1% 141
3 -2% 118
4 -3% 123
5+ +7% 216
This is because MLR currently always uses bracket.
Best MLR
• If query is latin charset use English (3% Gain)
• If query is >4 characters (or 12 bytes) use MLR/Bracket (7% Gain)
• Else use IRF (0% Gain)
Yahoo vs Competitor
DCG %P+E %Bad %P @ 1
Yahoo CN*
6.89 16.9% 14.6% 4.3%
Google -2% -8% +6% +7%
Baidu -3% -14% -31% +0%
MLR +2.4% +1% -13% +0%
Yahoo vs Google
Here darker color means longer query.
Yahoo vs. Baidu
Yahoo vs. BaiduAll of our gain over Baidu is due to Latin queries.