supporting cross-device web search with social navigation-based mobile touch interactions
TRANSCRIPT
Shuguang Han1, Daqing He1, Zhen Yue2 & Peter Brusilovsky1
1. University of Pittsburgh, 2. Yahoo! Labs
1
} Task-centered personalized search ◦ Ahn, J.-w., Brusilovsky, P., He, D., Grady, J., and Li, Q.
(2008) Personalized Web Exploration with Task Models. In: Proceedings of the 17th international conference on World Wide Web, WWW '08, Beijing, China, April 21-25, 2008, ACM, pp. 1-10
} Social navigation for social search ◦ Farzan, R. (2009). A Study of social navigation support
under different situational and personal factors (Ph.D. Thesis). University of Pittsburgh
} What is relevant on a page? ◦ Loboda, T. D., Brusilovsky, P., and Brunstein, J. (2011)
Inferring Word Relevance from Eye-movements of Readers. In: Proceedings of 2011 International Conference on Intelligent User Interfaces,IUI 2011, Palo Alto, California, USA, 2011, pp. 175-184
2
} Cross-device web search ◦ A common search condition in commercial search engines
(Wang et al., 2013) ◦ Refers to the case that a user initializes an information need
on one device but complete it in another (Montañez et al., 2014) � Often observed for exploratory and complex search tasks
} Automatic support of cross-device web search is an important research topic ◦ Support = better ranking of relevant documents ◦ Re-rank relevant documents based on users’ search history
(query history and click-through)
5
} Support queries in the current session using search history from the previous session ◦ Using click-through as relevance feedback (Han et al., 2015) ◦ Re-rank the default Google search results
} Two problems ◦ Using the full-text of click-through documents is too noisy ◦ Click-through history in mobile may be too few
6
q1 q2 q3 q4 q5 q6 q7 q8
To-be-supported queries
Search context modeling and document re-ranking
} Mobile touch interaction (MTI) can be used to locate the most relevant subdocument content ◦ Basic idea: low touch speed may indicate users’ interests on
the corresponding subdocument content (Han et al., 2015) ◦ Applying this method in click-through documents can model
users’ search context more accurately
} MTIs from other users can be used to vote for the most relevant content ◦ Social search based on social navigation (Brusilovsky, et al., 2004)
} Our social navigation MTI approach considers both of the two advantages
7
8
Goal: Rank relevant documents of query qi for U3
Ranking relevant documents based on: 1: pure query keywords 2: query + the full-text of P1 3: query + subdocument chunk (C6) inferred from self MTI 4: query + subdocument chunks (C1, C3 & C6) inferred from social navigation MTIs
Note: We will talk later (in slide 10) to illustrate the way of extracting the subdocument chunks.
} Two conditions ◦ Mobile-to-desktop (M-D) search: 1st session on mobile, 2nd
session on desktop ◦ Desktop-to-desktop (D-D) search: 1st session on desktop, 2nd
session on desktop ◦ We also explore the possibility of applying MTIs from the 1st
session of M-D to the 2nd sessions of both M-D and D-D.
} Three task types ◦ Product (PD) ◦ People (PE) ◦ News (NE) ◦ Total six tasks with two for each type
9
10
} Procedure ◦ Each participant goes through the same six tasks ◦ Three on M-D and three on D-D ◦ Latin square for task and search condition orders ◦ Each participant is asked to rate the relevance for saved
documents
} Data collection ◦ 24 participants. 961 queries. 1,790 visited pages ◦ 3,286 MTIs: drag down/up (71%) are dominated MTIS
} Ground truth and evaluation ◦ We aggregate relevance scores from all users (with Bayesian
smoothing, see Han et al., 2015) for ground-truth ◦ nDCG@20 for evaluation
11
Session1 (8 minutes / Task) Session 2 (7 minutes / Task)
Mobile Desktop Desktop Desktop
PD1, PE1, NE1 PD2, PE2, NE2 PD1, PE1, NE1 PD2, PE2, NE2
M-D (1st session) D-D (1st session) M-D (2nd session) D-D (2nd session)
One example of experiment procedure (task order and device order were rotated in both two sessions)
Experiment system interfaces for desktop search (left) and mobile search (right)
12
q1 q2 q3 q4 q5 q6 q7 q8
To-be-supported queries
Search context modeling and document re-ranking
} Using context-sensitive retrieval model ◦ Infer search context
� from the full-text of click-through � from the self-MTI based subdocument chunks � from the social navigation-MTI based subdocument chunks
◦ Re-rank documents using the learning-to-rank algorithm ◦ Evaluate new ranking list based on the ground-truth
Default search results
13
MTI position P MTI speed S Inactive time I after MTI
Reading zone z = [P – M, P + M]. M = 15 words in this paper.
If S ≤ certain speed and I ≥ certain seconds, z is treated as a relevant document chunk.
14
} Two experiments ◦ Applying social navigation-based MTIs on M-D ◦ Applying social navigation-based MTIs on D-D
� Using MTIs from M-D for desktop click-through documents
} Four models to compare ◦ Pure google results (G) ◦ Re-rank G using full-text of click-through (G + HF) ◦ Re-rank G using self-MTI based document chunks (G + HMTI-S) ◦ Re-rank G using social navigation MTI based document
chunks (G + HMTI-SN)
} Task effects ◦ PD, PE and NE tasks
15
Table 1. nDCG of different runs on M-D. ~HX means a significance comparing to HX
nDCG@20 Sig. (~HF) Sig. (~HMTI-S)
G 0.3974 ↓, p<0.001 ↓, p<0.001
G+HF 0.4298 - ↓, p=0.004
G+HMTI-S 0.4421 ↑, p=0.004 -
G+HMTI-SN 0.4497 ↑, p<0.001 ↑, p=0.003
Experiment results : • MTIs help locate more relevant content: both G+HMTI-S and
G+HMTI-SN are significantly better than G+HF. • Social navigation MTIs (G+HMTI-SN) further improve the
search performance over the self-MTIs.
16
Table 2. Task effects. Numbers in bold (italics) indicates p<0.05 compare with HF (HMTI-S) using Wilcoxon signed-rank test
PD PE NE
G 0.4012↓ 0.4648↓ 0.3286↓
G+HF 0.4300 0.4835 0.3773
G+HMTI-S 0.4276 0.5077↑ 0.3902
G+HMTI-SN 0.4249 0.5197↑ 0.4019↑
Experiment results: • PE and NE show the same trend as overall results, while PD
has no significance. • The necessity of considering task differences when
applying search context-based retrieval model
17
Table 3. nDCG of different runs on D-D. ~HX means a significance comparing to HX
nDCG@20 Sig. (~HF) Sig. (~HMTI-SN)
G 0.4068 ↓, p<0.001 ↓, p<0.001
G+HF 0.4452 - ↓, p=0.610
G+HMTI-SN 0.4491 ↑, P=0.610 -
Experiment results : • Social navigational MTIs do not improve the performance.
o This may indicate that users have different intentions when they visit the same web page from different devices.
• Only using the Social Navigation MTIs under the similar search queries (G+HMTI-SN-Q), we observe the significant performance boost.
G+HMTI-SN-Q 0.4549 ↑, P=0.021 ↑, P=0.006
18
Table 4. Task effects. Numbers in bold (italics) indicates p<0.05 compare with HF (HMTI-S) using Wilcoxon signed-rank test
PD PE NE
G+HF 0.3612 0.5420 0.4308
G+HMTI-SN 0.3575 0.5462 0.4498
G+HMTI-SN-Q 0.3608 0.5529↑(p=0.06) 0.4576↑
Experiment results: • Same as M-D, PE and NE show the same trend as overall
results, while PD has no significance. This indicates the necessity of considering task differences when applying search context-based retrieval model
19
} Conclusions ◦ MTI-based subdocument content gives more fine-grained
relevant information ◦ Social navigation MTIs achieve better performance because of
its ability of aggregating users’ voting for relevance ◦ Social navigation MTIs can be applied not only in M-D but also
in D-D, which could benefit commercial search engines who have both mobile and desktop search logs ◦ Generalizing social navigation MTIs across different search
contexts should be careful
} Future work ◦ Combing social navigation-based MTI with self-based MTIs ◦ Studying the utility of social navigation MTIs in a natural
search setting (rather than control lab study)
20
More details: Han, S., He, D., Yue, Z., and Brusilovsky, P. (2015) Supporting Cross-Device Web Search with Social Navigation-Based Mobile Touch Interactions. In: F. Ricci, K. Bontcheva, O. Conlan and S. Lawless (eds.) Proceedings of 23nd Conference on User Modeling, Adaptation and Personalization (UMAP 2015), Dublin, Ireland, June 29 - July 3, 2015, Springer Verlag, pp. 143-155, also available at http://link.springer.com/chapter/10.1007/978-3-319-20267-9_12.