amy n. langville mathematics department college of charleston [email protected] math meet 2/20/10
TRANSCRIPT
![Page 2: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/2.jpg)
Outline
Short History of Web SearchLink Analysis and Google’s PageRankThe Random SurferGoogle-opolyMarch MadnessConclusion
![Page 3: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/3.jpg)
Thesis
1998
![Page 4: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/4.jpg)
Pre-1998 Web
Trip back in time to 1995– How did you find information then?
![Page 5: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/5.jpg)
Pre-1998 Web
Trip back in time to 1995– How did you find information then?– Better question:
![Page 6: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/6.jpg)
Pre-1998 Web
Trip back in time to 1995– How did you find information then?– Better question: how old were you then?
![Page 7: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/7.jpg)
Pre-1998 Web
Trip back in time to 1995– How did you find information then?– Better question: how old were you then?
![Page 8: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/8.jpg)
Inverted IndexMain tool of pre-1998 search engines
![Page 9: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/9.jpg)
Problems with the Inverted Index
•Too many pages
![Page 10: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/10.jpg)
![Page 11: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/11.jpg)
Problems with the Inverted Index
• Too many pages• Spam
![Page 12: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/12.jpg)
Problems with the Inverted Index
• Too many pages• Spam: human eyes vs. spider eyes
![Page 13: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/13.jpg)
![Page 14: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/14.jpg)
![Page 15: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/15.jpg)
Problems with the Inverted Index
• Too many pages• Spam: human eyes vs. spider eyes
![Page 16: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/16.jpg)
Problems with the Inverted Index
• Too many pages• Spam: human eyes vs. spider eyes
![Page 17: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/17.jpg)
Problems with the Inverted Index
• Too many pages• Spam: human eyes vs. spider eyes
Learn how to make millions
Win a ipod
Text 8 if you’re awake
![Page 18: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/18.jpg)
Link Analysis
• pre-1998 engines only used text analysis.
• Link analysis saved search from SEOs and built companies like Google, Yahoo, Ask.
• Nearly every major search engine uses link analysis.
1998text analysis Link analysis
![Page 19: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/19.jpg)
Link Analysis1998
text analysis Link analysis
![Page 20: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/20.jpg)
![Page 21: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/21.jpg)
Moral #1
Sometimes being perceived as an expert forces you to become one.
![Page 22: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/22.jpg)
What happens when you google?
All the old text analysis + the new link analysis
![Page 23: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/23.jpg)
What happens when you google?
ranked list
1
2
3
4
5 6
7
8
![Page 24: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/24.jpg)
Why are rankings so important?
![Page 25: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/25.jpg)
Web as a graph
Each node is a webpage.
Each arrow is a hyperlink.
![Page 26: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/26.jpg)
In-links vs. Out-links
![Page 27: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/27.jpg)
A Trip to Google-topia
Emmie
Randy, the Random Surfer
video clip
![Page 28: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/28.jpg)
A Random Walk on the Web graph
![Page 29: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/29.jpg)
![Page 30: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/30.jpg)
![Page 31: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/31.jpg)
![Page 32: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/32.jpg)
![Page 33: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/33.jpg)
Matrix Notation
![Page 34: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/34.jpg)
BUT THERE ARE SOME PROBLEMS!
![Page 35: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/35.jpg)
![Page 36: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/36.jpg)
![Page 37: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/37.jpg)
The surfer gets stuck!
This is called a dangling node.
How does Google fix this?
![Page 38: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/38.jpg)
The surfer can “teleport”
We add a link from the dangling node to every other node.
When web surfing, this is equivalent to typing an address in the URL bar.
![Page 39: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/39.jpg)
Probability Matrix
We must also take this into consideration for our probability matrix.
![Page 40: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/40.jpg)
Dangling nodes and teleportation
video clip
![Page 41: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/41.jpg)
Let’s look at another problem.
![Page 42: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/42.jpg)
![Page 43: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/43.jpg)
![Page 44: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/44.jpg)
![Page 45: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/45.jpg)
![Page 46: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/46.jpg)
![Page 47: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/47.jpg)
![Page 48: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/48.jpg)
![Page 49: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/49.jpg)
Our surfer gets stuck in the webpages 4, 5, and 6.
This is called a cycle.
How do we fix this?
![Page 50: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/50.jpg)
Cycling
video clip
![Page 51: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/51.jpg)
Full Teleportation
We must consider the possibility of, at any time, using the URL bar to type an address.
We add an extra link from every vertex to every other vertex.
![Page 52: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/52.jpg)
Surfing vs. teleporting
Do people always use the URL bar as much as they use hyperlinks?
Google doesn’t think so. They think you only use the URL about 15%
of the time.
![Page 53: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/53.jpg)
Computing PageRank by observing Randy
video clip
![Page 54: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/54.jpg)
Summary of Ranking
Search query
Pull out relevant webpages from inverted index
Use PageRank and other information to rank webpages
![Page 55: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/55.jpg)
Creators of Google
Sergey Brin and Larry Page
Computer Science majors
Now entire PhD programs in information retrieval
![Page 56: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/56.jpg)
Creators of Google
Sergey Brin and Larry Page
Computer Science majors
Now entire PhD programs in information retrieval
The world’s largest eigenvector computationThe world’s largest eigenvector computation
![Page 57: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/57.jpg)
Moral #2
Take a leave of absence for brilliant ideas.
![Page 58: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/58.jpg)
More on PageRank
SIAM’s WhydoMath? Project– url =http://dev.whydomath.org/node/google/index.html
DDL on PageRank – url = http://spinner.cofc.edu/~langvillea/DISSECTION-LAB/ClarePageRankModule/
1_WebLetter.html?referrer=webcluster& LOCI: Google-opoly
– url=http://mathdl.maa.org/mathDL/23/?pa=content&sa=viewDocument&nodeId=3355
![Page 59: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/59.jpg)
Moral #3
The more ways you can view a problem, the more likely you are to truly understand it, and hence, solve it.
![Page 60: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/60.jpg)
Google-opoly
applets
![Page 61: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/61.jpg)
March MadnessHow should teams vote?
• Losing teams give one vote to each team that beats them.
• Losing teams vote with margin of victory.
• Both winning and losing teams vote with # points scored.
![Page 62: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/62.jpg)
Point Differential Voting
![Page 63: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/63.jpg)
![Page 64: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/64.jpg)
![Page 65: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/65.jpg)
![Page 66: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/66.jpg)
Moral #4
Now is a great time to do math.
![Page 67: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/67.jpg)
Conclusion
PageRank is a sophisticated algorithm that set Google apart
The Web can be represented with graphs and matrices
PageRank’s idea of Voting has many applications.
![Page 68: Amy N. Langville Mathematics Department College of Charleston langvillea@cofc.edu Math Meet 2/20/10](https://reader031.vdocuments.site/reader031/viewer/2022032802/56649e165503460f94b00b24/html5/thumbnails/68.jpg)
Acknowledgements
Tim ChartierCarl MeyerEmmie DouglasKathryn PedingsClare RodgersErich KreutzerBen KovanichRyan Dumville
Luke IngramAnjela GovanNick DovidioYoshi YamamotoNeil GoodsonColin Stephenson