MLR

DCG %P+E %Bad %P @ 1

Yahoo CN*

6.89 16.9% 14.6% 4.3%

MLR +2.4% +1% -13% +0

* Used Yahoo Simplified-preferred as baseline, since it has higher relevance than all-language and cn-preferred. Also both used CNB.

MLR vs YahooThe only place where MLR does really bad is on Latin queries. If we fix this the gain could be much bigger.

MLR by Language

Chinese Latin

Baseline 6.6 8.0

JP MLR +4% +1%

EN MLR 0% +3%

Google +3% +6%

Examples 509 91

MLR vs Yahoo by Length

Query Length MLR vs Yahoo

# of Queries

2 -1% 141

3 -2% 118

4 -3% 123

5+ +7% 216

This is because MLR currently always uses bracket.

Best MLR

• If query is latin charset use English (3% Gain)

• If query is >4 characters (or 12 bytes) use MLR/Bracket (7% Gain)

• Else use IRF (0% Gain)

Yahoo vs Competitor

DCG %P+E %Bad %P @ 1

Yahoo CN*

6.89 16.9% 14.6% 4.3%

Google -2% -8% +6% +7%

Baidu -3% -14% -31% +0%

MLR +2.4% +1% -13% +0%

Yahoo vs Google

Here darker color means longer query.

Yahoo vs. Baidu

Yahoo vs. BaiduAll of our gain over Baidu is due to Latin queries.