collaborative future event recommendation · event recommendation einat minkov, ben charrow...

COLLABORATIVE FUTURE EVENT RECOMMENDATION

Einat Minkov, Ben Charrow Jonathan Ledlie, Seth Teller, Tommi Jaakkola

1

CIKM 2010

Motivation: Context Aware Devices 2

■  We are interested in building devices that can make good recommendations with minimal feedback

■  In the future these devices will know a person’s location with incredible accuracy and precision

■  We focus on a recommendation system that can use this type of information

Roadmap 3

■  Event Recommendation ■  Individual vs. Collaborative Recommendation ■  User Study + Evaluation ■  Related Work ■  Active Learning + Future Work ■  Conclusion

Event Recommendations

■  Events are standard in that -  they have textual descriptions

(e.g. announcements) -  can be extracted

automatically from e-mail

■  Events are special in that -  they are time and location

dependent -  they have not happened yet

■  We focus on academic talks

4

PHAST: Hardware-Accelerated Shortest Path TreesSpeaker: Daniel DellingSpeaker Affiliation: Microsoft CorporationHost: Prof. Charles E. Leiserson

Date: 10-20-2010Time: 4:15 PM - 5:15 PMLocation: 32-G575 (Theory of Computation Seminar Room)

We present a novel algorithm to solve the nonnegativesingle-source shortest path problem on road networks andother graphs with low highway dimension. After a quickpreprocessing phase, we can compute all distances from agiven source in the graph with essentially a linear sweepover all vertices. Because this sweep is independent of thesource, we are able to reorder vertices in advance toexploit locality. Moreover, our algorithm takes advantageof features of modern CPU architectures, such as SSE andmulti-core. Compared to Dijkstra's algorithm...

5

■  Events require explicit representation in a feature space -  pure collaborative filtering is not possible -  each future event forms a “cold start”

■  We cast recommendations as a ranking problem -  Assume user feedback in the form of pairwise preferences -  Given:

Past events EP , future events EF

Preference pairs: (ej, ek) R(u), ej,ek EP , u U xj preferred over xk by user u

Event Recommendations

Roadmap 6

  Event Recommendation ■  Individual vs. Collaborative Recommendation ■  User Study + Evaluation ■  Related Work ■  Active Learning + Future work ■  Conclusion

Small scale validation cont’d• We can also directly examine changes due to OR1 knock-out

Preliminary validation• The λ-phage (bacteria infecting virus)!"#$%&'#($)*#+,-./

0#,12(#'

3-142(#

5&1-6#

7&8.6,

OR3 OR1OR2cI cro

cI RNAp2

Arkin et al. 1998

• We can find the equilibrium of the game (binding frequencies)as a function of overall protein concentrations.

50

10!2

100

102

0

0.2

0.4

0.6

0.8

1

Binding in OR

3

frepressor

/fRNA

Bin

din

g F

requency (

tim

e!

avera

ge)

RepressorRNA!polymerase

(a) OR3

10!2

100

102

0

0.2

0.4

0.6

0.8

1

Binding in OR

2

frepressor

/fRNA

Bin

din

g F

requency (

tim

e!

avera

ge)


(b) OR2

10!2

100

102

0

0.2

0.4

0.6

0.8

1

Binding in OR

1

frepressor

/fRNA

Bin

din

g F

requency (

tim

e!

avera

ge)


(c) OR1

Figure 3: Predicted protein binding to sites OR3, OR2, and mutated OR1 for increasing amounts of cI2.

became sufficiently high do we find cI2 at the mutatedOR1 as well. Note, however, that cI2 inhibits transcrip-tion at OR3 prior to occupying OR1. Thus the bindingat the mutated OR1 could not be observed without in-terventions.

7 Discussion

We believe the game theoretic approach provides a com-pelling causal abstraction of biological systems with re-source constraints. The model is complete with prov-ably convergent algorithms for finding equilibria on agenome-wide scale.

The results from the small scale application are en-couraging. Our model successfully reproduces knownbehavior of the λ−switch on the basis of molecularlevel competition and resource constraints, without theneed to assume protein-protein interactions between cI2dimers and cI2 and RNA-polymerase. Even in the con-text of this well-known sub-system, however, few quan-titative experimental results are available about bind-ing. Proper validation and use of our model thereforerelies on estimating the game parameters from availableprotein-DNA binding data (in progress). Once the gameparameters are known, the model provides valid pre-dictions for a number of possible perturbations to thesystem, including changing nuclear concentrations andknock-outs.

Acknowledgments

This work was supported in part by NIH grant GM68762and by NSF ITR grant 0428715. Luis Perez-Breva is a“Fundacion Rafael del Pino” Fellow.

References

[1] Adam Arkin, John Ross, and Harley H. McAdams.Stochastic kinetic analysis of developmental path-way bifurcation in phage λ-infected excherichia colicells. Genetics, 149:1633–1648, August 1998.

[2] Kenneth J. Arrow and Gerard Debreu. Existence ofan equilibrium for a competitive economy. Econo-metrica, 22(3):265–290, July 1954.

[3] Z. Bar-Joseph, G. Gerber, T. Lee, N. Rinaldi,J. Yoo, B. Gordon F. Robert, E. Fraenkel,T. Jaakkola, R. Young, and D. Gifford. Compu-tational discovery of gene modules and regulatorynetworks. Nature Biotechnology, 21(11):1337–1342,2003.

[4] Otto G. Berg, Robert B. Winter, and Peter H. vonHippel. Diffusion- driven mechanisms of proteintranslocation on nucleic acids. 1. models and theory.Biochemistry, 20(24):6929–48, November 1981.

[5] Drew Fudenberg and Jean Tirole. Game Theory.The MIT Press, 1991.

10

• Predictions are again qualitatively correct

52

• The goal is to learn user rankings of events (talks) with minimal feedback

• With little feedback about past events, it is difficult to estimate the ranking function separately for each user

Event ranking

event descriptions(e.g., announcements)

relativeranking

Facilitating Complex Scientific Analytics in the CloudSpeaker: Magdalena BalazinskaSpeaker Affiliation: University of WashingtonHost: Sam MaddenHost Affiliation: CSAIL

Date: 9-21-2010Time: 11:00 AM - 12:00 PMRefreshments: 10:50 AMLocation: 32-D463

Abstract: Scientists today have the ability to generate data at anunprecedented scale and rate and, as a result, they must increasingly turn to







acb

1st

2nd

3rd

b)a)

c)user specific

ranking function

Event Ranking 7

■  The goal is to learn user rankings of future events (talks) with minimal feedback

■  With little feedback, it is difficult to estimate a ranking function separately for each user

■  The goal is to learn user rankings of future events (talks) with minimal feedback

■  With little feedback, it is difficult to estimate a ranking function separately for each user

Event Ranking 7

SVMRank for User Specific Ranking Function

Collaborative Event Ranking 8

■  We rank by making: -  a feature mapping shared across users -  user specific ranking functions

Small scale validation cont’d• We can also directly examine changes due to OR1 knock-out

Preliminary validation• The λ-phage (bacteria infecting virus)!"#$%&'#($)*#+,-./

0#,12(#'

3-142(#

5&1-6#

7&8.6,

OR3 OR1OR2cI cro

cI RNAp2

Arkin et al. 1998

• We can find the equilibrium of the game (binding frequencies)as a function of overall protein concentrations.

50

10!2

100

102

0

0.2

0.4

0.6

0.8

1

Binding in OR

3

frepressor

/fRNA

Bin

din

g F

requency (

tim

e!

avera

ge)


(a) OR3

10!2

100

102

0

0.2

0.4

0.6

0.8

1

Binding in OR

2

frepressor

/fRNA

Bin

din

g F

requency (

tim

e!

avera

ge)


(b) OR2

10!2

100

102

0

0.2

0.4

0.6

0.8

1

Binding in OR

1

frepressor

/fRNA

Bin

din

g F

requency (

tim

e!

avera

ge)


(c) OR1

Figure 3: Predicted protein binding to sites OR3, OR2, and mutated OR1 for increasing amounts of cI2.

became sufficiently high do we find cI2 at the mutatedOR1 as well. Note, however, that cI2 inhibits transcrip-tion at OR3 prior to occupying OR1. Thus the bindingat the mutated OR1 could not be observed without in-terventions.

7 Discussion

We believe the game theoretic approach provides a com-pelling causal abstraction of biological systems with re-source constraints. The model is complete with prov-ably convergent algorithms for finding equilibria on agenome-wide scale.

The results from the small scale application are en-couraging. Our model successfully reproduces knownbehavior of the λ−switch on the basis of molecularlevel competition and resource constraints, without theneed to assume protein-protein interactions between cI2dimers and cI2 and RNA-polymerase. Even in the con-text of this well-known sub-system, however, few quan-titative experimental results are available about bind-ing. Proper validation and use of our model thereforerelies on estimating the game parameters from availableprotein-DNA binding data (in progress). Once the gameparameters are known, the model provides valid pre-dictions for a number of possible perturbations to thesystem, including changing nuclear concentrations andknock-outs.

Acknowledgments

This work was supported in part by NIH grant GM68762and by NSF ITR grant 0428715. Luis Perez-Breva is a“Fundacion Rafael del Pino” Fellow.

References

[1] Adam Arkin, John Ross, and Harley H. McAdams.Stochastic kinetic analysis of developmental path-way bifurcation in phage λ-infected excherichia colicells. Genetics, 149:1633–1648, August 1998.

[2] Kenneth J. Arrow and Gerard Debreu. Existence ofan equilibrium for a competitive economy. Econo-metrica, 22(3):265–290, July 1954.

[3] Z. Bar-Joseph, G. Gerber, T. Lee, N. Rinaldi,J. Yoo, B. Gordon F. Robert, E. Fraenkel,T. Jaakkola, R. Young, and D. Gifford. Compu-tational discovery of gene modules and regulatorynetworks. Nature Biotechnology, 21(11):1337–1342,2003.

[4] Otto G. Berg, Robert B. Winter, and Peter H. vonHippel. Diffusion- driven mechanisms of proteintranslocation on nucleic acids. 1. models and theory.Biochemistry, 20(24):6929–48, November 1981.

[5] Drew Fudenberg and Jean Tirole. Game Theory.The MIT Press, 1991.

10

• Predictions are again qualitatively correct

52

• The goal is to learn user rankings of events (talks) with minimal feedback

• We learn the ranking function in two parts- a feature mapping shared across users- user specific ranking functions based on the features

Collaborative event ranking

event descriptions(e.g., announcements)

“perceptual” featurecoordinates

relativeranking

a










b

c

bac

1st

2nd

3rd

b)a)

c)user specific

ranking functionsharedfeaturemapping

Collaborative Event Ranking 8

■  We rank by making: -  a feature mapping shared across users -  user specific ranking functions

Comparing Ranking Methods 9

RankSVM LowRank

Feature Vectors

User Parameters

Estimation

Number of Parameters

A seminar announcement 10

Representing Announcements 11

■  Token Frequency-Inverse Document Frequency (TF-IDF)

■  Latent Dirichlet Allocation (LDA) -  generative model where documents are distributions

over “topics” which are distributions over words -  trained with 100 topics

■  Both representations use large unlabelled corpus

Roadmap 12

  Event Recommendation   Individual vs. Collaborative Recommendation ■  User Study + Evaluation ■  Related Work ■  Active Learning + Future work ■  Conclusion

Feedback Elicitation 13

■  Events compete for user’s resources (primarily, time) ■  Assume a week-long time frame in which events can

be meaningfully compared ■  User study: ask users to select events they would

have liked to attend in a given week.

User Study: Interface 14

User Study: Details 15

■  At MIT: -  30 graduate students, researchers and faculty -  15 consecutive weeks of MIT CSAIL seminar announcements

starting from 9/2007

■  At CMU: -  56 CS grad. students -  15 consecutive weeks of MIT CSAIL seminar announcements

starting from 2/2009

■  7 years of unlabelled MIT CSAIL seminar announcements from 5/2002 – 7/2009

User Study: The Data Set 16

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Nu

mb

er o

f T

alk

s

Week

CMU StudyAdvertised Talks

User Interest

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Nu

mb

er o

f T

alk

s

Week

MIT StudyAdvertised Talks

User Interest

User Study: The Data Set 16

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Nu

mb

er o

f T

alk

s

Week

CMU StudyAdvertised Talks

User Interest

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Nu

mb

er o

f T

alk

s

Week

MIT StudyAdvertised Talks

User Interest

# Announcements

Avg. # Attended talks

Evaluation: Overview 17

■  RankSVM vs. LowRank -  TF-IDF or LDA features -  Perceptual space and parameter sensitivity

■  Experimental setup -  Randomly select k<10 weeks per user to train on -  Test on same 5 weeks across users

■  Measuring results -  Metric: Mean Average Precision (MAP) -  Baseline performance: random recommendation

0.35

0.40

0.45

0.50

0.55

0.60

1 2 3 4 5 6 7 8 9 10

Me

an

Ave

rag

e P

reci

sio

n

Number of Training Weeks

LowRank w/TF-IDFLowRank w/LDA

RankSVM w/TF-IDFRankSVM w/LDA

Evaluation: LowRank vs. RankSVM 18

Random

0.35

0.40

0.45

0.50

0.55

0.60

1 2 3 4 5 6 7 8 9 10

Me

an

Ave

rag

e P

reci

sio

n


LowRank w/TF-IDFLowRank w/LDA

RankSVM w/TF-IDFRankSVM w/LDA

Evaluation: LowRank vs. RankSVM 18

Week 3 performance

Random

0.40

0.42

0.44

0.46

0.48

0.50

0.52

0.54

0.56

0.58

0.60

1 2 3 4 5 6 7 8 9 10

Mean A

vera

ge P

reci

sion


TF-IDF k=12, C=2TF-IDF k=10, C=1

LDA k=12, C=2LDA k=10, C=1

Evaluation: Perceptual Space 19

LowRank: Perceptual Spaces and Slack

0.40

0.42

0.44

0.46

0.48

0.50

0.52

0.54

0.56

0.58

0.60

1 2 3 4 5 6 7 8 9 10

Mean A

vera

ge P

reci

sion


TF-IDF k=12, C=2TF-IDF k=10, C=1

LDA k=12, C=2LDA k=10, C=1

Evaluation: Perceptual Space 19

LowRank: Perceptual Spaces and Slack

LDA

TF-IDF

0.0

0.2

0.4

0.6

0.8

1.0

0.2 0.3 0.4 0.5 0.6 0.7 0.8

CD

F o

f A

ll T

est

ing W

eeks

Average Precision of Testing Week

LDATF-IDFRandom

Evaluation: Individual Users 20

LowRank with 3 weeks of training vs. Random

Roadmap 21

  Event Recommendation   Individual vs. Collaborative Recommendation   User Study + Evaluation ■  Related Work ■  Active Learning + Future work ■  Conclusion

Related Work 22

■  Collaborative filtering via matrix factorization -  J. Rennie and N. Srebro. (ICML 2005) -  T. Hoffman (TOIS 2004) -  S. Tong and D. Koller. (JMLR 2002)

■  Reduced dimensionality recommendations -  B. Bai et. Al (NIPS 2009)

■  Content-Based Recommendations for scientific papers -  Dumais and Nielsen. (SIGIR 1992) -  C. Basu, H. Hirsh, W. W. Cohen, and C. Nevill-Manning. (JAIR 2001) -  D. Yarowsky and R. Florian. (SIGDAT 1999)

■  Collaborative recommendation with feature descriptions -  W. Chu and S.-T. Park. (WWW 2009)

Active Learning 23

■  Problem: users must rate all announcements ■  Idea: model the uncertainty in ranking functions

-  add uncertainty to each user provided label (i.e. add a small flipping probability)

-  use information gain to select which seminars to present to a user

■  We want to run another user study which has an active learning component

Future Work: Features + Deployment

■  Include additional features -  speaker’s past articles -  speaker affiliation -  host name -  venue information

■  Couple the recommendations with an indoor location system

24

PHAST: Hardware-Accelerated Shortest Path TreesSpeaker: Daniel DellingSpeaker Affiliation: Microsoft CorporationHost: Prof. Charles E. Leiserson

Date: 10-20-2010Time: 4:15 PM - 5:15 PMLocation: 32-G575 (Theory of Computation Seminar Room)

We present a novel algorithm to solve the nonnegativesingle-source shortest path problem on road networks andother graphs with low highway dimension. After a quickpreprocessing phase, we can compute all distances from agiven source in the graph with essentially a linear sweepover all vertices. Because this sweep is independent of thesource, we are able to reorder vertices in advance toexploit locality. Moreover, our algorithm takes advantageof features of modern CPU architectures, such as SSE andmulti-core. Compared to Dijkstra's algorithm...

Roadmap 25

  Event Recommendation   Individual vs. Collaborative Recommendation   User Study + Evaluation   Related Work   Active Learning + Future work ■  Conclusion

Contributions 26

■  Presented and motivated the problem of recommending future events

■  Gave a low rank collaborative learning approach ■  Empirically evaluated the approach, showing that it

outperformed a content based recommendation system

■  Created a publicly available dataset http://mis.haifa.ac.il/~einatm

http://www.seas.upenn.edu/~bcharrow

Acknowledgments 27

Einat Minkov Jonathan Ledlie Tommi Jaakkola Seth Teller

Thank you!

34

collaborative future event recommendation · event recommendation einat minkov, ben charrow...

Documents