real-time news recommendation and lifelogging: an ... · real-time news recommendation and...
TRANSCRIPT
![Page 1: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/1.jpg)
Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)
Frank Hopfgartner @OkapiBM25
![Page 2: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/2.jpg)
A few words about me
Past: Various positions in Berlin (TUB), Dublin (DCU), Berkeley (ICSI), and London (QMUL)
Research on Information Access
Systems
Lecturer in Information Studies (HATII, Glasgow)
PhD in Information Retrieval (University of Glasgow)
![Page 3: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/3.jpg)
Overview
Evaluation Campaigns
Overview NEWSREEL
(CLEF) LifeLog
(NTCIR-12)
Introduction
Career Collaborations Research
![Page 4: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/4.jpg)
affiliated with UC Berkeley
![Page 5: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/5.jpg)
*now: INSIGHT Centre
![Page 6: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/6.jpg)
![Page 7: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/7.jpg)
Smart Information Systems
![Page 8: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/8.jpg)
Collaborations
Recommender Systems
Aggregated Search
Multimedia Analysis
Gamification Large-Scale
Evaluation
Semantic Search
Multimedia Access
Lifelogging
Sensor Analysis Multimedia Retrieval
& Recommendation
User Modelling
User Simulation
Co-Author graph of first 100 publications (2006-2015)
![Page 9: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/9.jpg)
Example: Video Search Engine
[ACM Multimedia’08, ECIR’08, SIVP’08, ACM TOIS’11]
Adapt retrieval results based on •User’s search queries •Relevance feedback
![Page 10: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/10.jpg)
Example: Video Search Engine
[ACM/Springer Multimedia Systems (2010), MTAP (2012)]
Recommend news stories based on user’s long-term interests
![Page 11: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/11.jpg)
Example: Video Browsing
[MMM’12-14, Int. Journal of Multimedia Information Retrieval (2014)]
Video Browser Showdown Competition
![Page 12: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/12.jpg)
Example: Other Video Search Engines
![Page 13: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/13.jpg)
Example: Image Browser
[IICAI’09]
Ostensive Browsing
![Page 14: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/14.jpg)
Example: Visual Lifelog Browser
[published as book chapter, Springer Verlag (2013)]
Providing easy access to visual lifelogs
![Page 15: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/15.jpg)
Example: Enterprise Search Engine
[AAMAS’14]
Aggregated Search
![Page 16: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/16.jpg)
Example: Health Information Portal
[PervasiveHealth’13, MIS’14, Book chapter (2015)]
Semantic Search Multilingual Search
![Page 17: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/17.jpg)
Example: Knowledge Management System
[ECIR’14, Book Chapter (2015)]
Targeting intrinsic motivation using Gamification design elements
![Page 18: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/18.jpg)
Multimedia Analysis
Video segmentation
Voice activity detection
Quality assessment
of user-generated
video
Affective content analysis
[ECIR’09, ACM Multimedia (2014), CMBI (‘12, ‘13), MMM (2010, 2013, 2014) , ICMR (2014)]
![Page 19: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/19.jpg)
Data Analysis
Activity and Energy expenditure estimation using accelerometer data
!
![Page 20: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/20.jpg)
Overview
Evaluation Campaigns
Overview NEWSREEL
(CLEF) LifeLog
(NCTIR-12)
Introduction
Career Collaborations Research
![Page 21: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/21.jpg)
How do we evaluate information access systems?
Document
collection
Topic
set
Relevance
assessments
Test co
llection
Document
collection
![Page 22: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/22.jpg)
Evaluation Campaigns
TREC CLEF
FIRE
NTCIR
Common dataset Pre-defined tasks Ground truth Evaluation protocol Evaluation metrics
![Page 23: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/23.jpg)
Focus on different domains
Microblogging
Ad-hoc and Web Search
Multimedia
Federated Web Search
XML Retrieval
Information Access in the Legal Domain
Document Similarity
…
![Page 24: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/24.jpg)
Overview
Evaluation Campaigns
Overview NEWSREEL
(CLEF) LifeLog
(NTCIR-12)
Introduction
Career Collaborations Research
![Page 25: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/25.jpg)
CLEF Tracks
Source: http://www.clef-initiative.eu/
eHealth
ImageCLEF
LifeCLEF
Living Labs for IR (LL4IR)
News Recommendation Evaluation Lab (NEWREEL)
Uncovering Plagiarism, Authorship and Social Software Misuse (PAN)
Question Answering (QA)
Social Book Search (SBS)
CL
EF
’15
![Page 26: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/26.jpg)
In CLEF NEWSREEL, participants can develop news recommendation
algorithms and have them tested by millions of users over the period of a few
months in a living lab.
NEWSREEL
![Page 27: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/27.jpg)
Recommender Systems help users to find items that they were not
searching for.
What are recommender systems?
![Page 28: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/28.jpg)
Items?
![Page 29: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/29.jpg)
Example: YouTube
![Page 30: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/30.jpg)
Example: Netflix
![Page 31: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/31.jpg)
Example: News Articles
Source (Image): T. Brodt of plista.com
![Page 32: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/32.jpg)
What are living labs?
Rely on feedback from real users to develop convincing demonstrators that showcase potentials of an idea or a product.
Real-life test and experimentation environment to fill the pre-commercial gap between fundamental research and innovation.
![Page 33: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/33.jpg)
In an IR context…
“A living laboratory on the Web that brings researchers and searchers together is needed to facilitate ISSS (Information-Seeking Support System) evaluation.”
Kelly et al., 2009
![Page 34: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/34.jpg)
A / B testing
Eval
uat
e
submit to SIGIR
![Page 35: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/35.jpg)
CLEF NEWSREEL
20
14
Frank Hopfgartner
Andreas Lommatzsch
Benjamin Kille
Torben Brodt
Tobias Heintz
Co-Organisers
2015
Frank Hopfgartner
Torben Brodt
Benjamin Kille
Jonas Seiler
Balázs Hidasi
Andreas Lommatzsch
Roberto Turrin
Martha Larson
Scenario
![Page 36: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/36.jpg)
Who are the users?
![Page 37: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/37.jpg)
Devices
![Page 38: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/38.jpg)
• Given a dataset, predict news articles a user will click on
Offline Evaluation
• Recommend articles in real-time over several months
Online Evaluation
CLEF NEWSREEL 2014
TA
SK
1
TA
SK
2
@clefnewsreel http://www.clef-newsreel.org/
![Page 39: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/39.jpg)
Predict interactions based on an OFFLINE dataset
Task 1: Offline Evaluation
DA
TASE
T
EVA
LUA
TIO
N
Traffic and content updates of 9 German-language news content provider websites
Traffic: Reading article, clicking on recommendations
Updates: adding and updating news articles
Recorded in June 2013
65 GB, 84 Million records
[Kille et al., 2013]
Dataset split into different time segments
Participants have to predict interactions of these segments
Quality measured by the ratio of successful predictions by the total number of predictions
![Page 40: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/40.jpg)
Recommend news articles in REAL-TIME
Task 2: Online Evaluation
LIV
ING
LA
B
EVA
LUA
TIO
N
Provide recommendations for visitors of the news portals of plista’s customers
Ten portals (local news, sports, business, technology)
Communication via Open Recommender Platform (ORP)
Provide recommendations within <10ms (VM provided if necessary)
Three pre-defined evaluation periods
5-23 February 2014
1-14 April 2014
5-19 May 2014
Evaluation criteria
Number of clicks
Number of requests
Click-through rate
![Page 41: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/41.jpg)
Living Lab Scenario
…
Publisher A
Publisher n
Researcher 1
Researcher n
…
plista ORP
…
Millions of visitors Publishers Teams
![Page 42: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/42.jpg)
Open Recommender Platform
![Page 43: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/43.jpg)
Number of clicks
[CLEF’14]
![Page 44: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/44.jpg)
Number of requests
[CLEF’14]
![Page 45: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/45.jpg)
Click-Through Rate
[CLEF’14]
![Page 46: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/46.jpg)
Overall results
![Page 47: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/47.jpg)
Advertisement: Join the Living Lab - Tutorial at ECIR’15
![Page 48: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/48.jpg)
Overview
Evaluation Campaigns
Overview NEWSREEL
(CLEF) LifeLog
(NTCIR-12)
Introduction
Career Collaborations Research
![Page 49: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/49.jpg)
NTCIR
Sourc
e: H
ideo J
oho
![Page 50: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/50.jpg)
NTCIR-12 Tasks
NT
CIR
-12
Second round:
Search-Intent Mining
Mobile Click
Temporal Information Access
Spoken Query & Spoken Document Retrieval
QA Lab for Entrance Exam
First round:
Medical NLP for Clinical Documents
Personal Lifelog Access & Retrieval
Short Text Conversation
![Page 51: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/51.jpg)
Encourage research advances in organising and retrieving from lifelog data.
LifeLog @ NTCIR-12
![Page 52: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/52.jpg)
Lifelogging Challenges
The challenges are how to sense the person, their actions, their life and make it accessible using appropriate interfaces, search, recommendation engines and visual/aural feedback. Further, exploiting the lifelog to identify context for adaptive information services.
Source (Graphic): DAI-Labor, Berlin
![Page 53: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/53.jpg)
LifeLog @ NTCIR-12
CO
-OR
GA
NIS
ER
S
TIM
EL
INE
Cathal Gurrin, Dublin City University
Hideo Joho, University of Tsukuba
Frank Hopfgartner, University of Glasgow
Brian Moynagh, Dublin City University
Rami Albatal, Dublin City University
27 Feb 2015: Kickoff event
30 Jun 2015: Task Registration
01 Jul 2015: Dataset release
Sep 2015 – Feb 2016: Formal run
01 Feb 2016: Evaluation Results
01 Mar 2016: Paper for the Proceedings
7-10 Jun 2016: NCTIR-12 conference
![Page 54: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/54.jpg)
Multimodal dataset with information needs
Created by 5-10 individuals over
10+ days
TE
ST
CO
LL
EC
TIO
N
1,500 images, location, GSR, heart-rate, others… per lifelogger per day
Accompanying output of 1,000 concepts
Data processed pre-release (removal of personal content; face blurring, translation of concepts)
Detailed user queries and judgments generated by the lifelogging data gatherers
![Page 55: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/55.jpg)
Evaluate different methods of
retrieval and access.
Tasks
T1:
LIFE
LOG
SEM
AN
TIC
AC
CES
S (L
SAT)
T2:
LIFE
LOG
IN
SIG
HT
Models the retrieval need from lifelogs (Known-Item Search)
Retrieve N segments that match information need
Interactive or Automatic participation
Interactive: Time limit for fair and comparative evaluation in an interactive system with users
Automatic: Fully-automatic retrieval system. Automated query processing
Models the need for reflection over lifelog data
Exploratory task, the aim is to:
encourage broad participation
novel methods to visualise and explore lifelogs
Same data as LSAT task
Presented via demo/poster.
![Page 56: Real-Time News Recommendation and Lifelogging: An ... · Real-Time News Recommendation and Lifelogging: An Introduction to the Evaluation Campaigns NEWSREEL (CLEF) and Lifelog (NTCIR-12)](https://reader030.vdocuments.site/reader030/viewer/2022040514/5e6cddbf32b38346a92a7036/html5/thumbnails/56.jpg)
Thank you
Come and talk to me
Frank Hopfgartner, PhD [email protected]
@OkapiBM25 www.hopfgartner.co.uk