final presentation undergraduate researchers: graduate student mentor: faculty mentor: jordan...
TRANSCRIPT
![Page 1: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/1.jpg)
Final Presentation
Undergraduate Researchers:
Graduate Student Mentor:Faculty Mentor:
Jordan Cowart, Katie Allmeroth
Krist CulmerDr. Wenjun Zeng
Investigating the effectiveness of EVC and PCC as community-centered centrality measures
![Page 2: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/2.jpg)
2
Outline• Personal Experiences• Project Overview• Implementations
▫Scripts▫Gephi▫Web Crawler▫EVC / PCC
• Results▫Data Analysis
• Conclusion• Accomplishments• Future Work
![Page 3: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/3.jpg)
3
Outline• Personal Experiences• Project Overview• Implementations
▫Scripts▫Gephi▫Web Crawler▫EVC / PCC
• Results▫Data Analysis
• Conclusion• Accomplishments• Future Work
![Page 4: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/4.jpg)
4
Personal Experiences•Learned:
▫Python Built a web crawler
▫Social Networks Centrality measures Social Capital
▫Graph Visualization software; Gephi and SNAP Extend functionality of open source software
(PCC Plugin)▫Research process▫Related research
![Page 5: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/5.jpg)
5
Outline• Personal Experiences• Project Overview• Implementations
▫Scripts▫Gephi▫Web Crawler▫EVC / PCC
• Results▫Data Analysis
• Conclusion• Accomplishments• Future Work
![Page 6: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/6.jpg)
6
Project OverviewWhich node is more important, A or B?
![Page 7: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/7.jpg)
7
Project Overview
• Node A:▫ Current centrality measures
would say this node is more important because it has a large number of connections.
• But what about Node B?▫ It’s connections are more
diverse; indicating it has a higher value of social capital
Which node is more important, A or B?
![Page 8: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/8.jpg)
8
Project Overview
•Goal of project:▫Investigate if/how current centrality
measures take community membership into account and if necessary develop a centrality measure that considers community membership
•Why is this important:▫It will allow us to have a better
understanding of who the most important individuals are in a network
![Page 9: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/9.jpg)
9
Two measures of centralityEigenvector Centrality (EVC)
Principal Component Centrality (PCC)
Ilyas, M.U.; Radha, H. "Identifying Influential Nodes in Online Social Networks Using Principal Component Centrality", Communications (ICC), 2011 IEEE International Conference on, On page
6.
![Page 10: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/10.jpg)
10
Project Overview• 1. Develop a web crawler to extract user data from the
Mizzou Facebook graph, including ground-truth user-defined communities
• 2. Apply Eigenvector Centrality (EVC) and Principal Component Centrality (PCC) to collected and provided data sets to characterize properties of the algorithms
• 3. Develop analysis tools as needed
• 4. Develop a centrality measure that takes community membership and resource availability into consideration
![Page 11: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/11.jpg)
11
Outline• Personal Experiences• Project Overview• Implementations
▫Scripts▫Gephi▫Web Crawler▫EVC / PCC
• Results▫Data Analysis
• Conclusion• Accomplishments• Future Work
![Page 12: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/12.jpg)
12
Scripts
•Wrote a ton of scripts to aid us with our work▫Wrote scripts to:
Clean data Extract information from a text and convert
to a .graph file To visualize SNAP data in Gephi Color graph To compile information to make pie charts
(shown later)
![Page 13: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/13.jpg)
13
Gephi
•Visualization and exploration platform for networks
•Understanding data like below.. is hard.
![Page 14: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/14.jpg)
14
Easier, right?
![Page 15: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/15.jpg)
15
Web Crawler
•Built a web crawler in Python to gather publicly available Facebook Data▫Faster to deploy and more flexible than
Facebook’s API
•Collects only information from Mizzou declared students▫Accomplishes this by using a targeted,
breadth-first search
![Page 16: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/16.jpg)
16
Web Crawler•Collected the following data:
▫Major, gender, home town, Mizzou friends, Mizzou related likes
•Majors are condensed into colleges (e.g. Computer Science -> College of Engineering)
•Likes are condensed into general community resources (e.g. GPC, MSA, GSA -> Student Governance)
![Page 17: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/17.jpg)
17
Web Crawler
![Page 18: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/18.jpg)
18
Web Crawler
•Problems encountered:▫Facebook’s HTML/CSS (Inconsistent
format)▫BeautifulSoup’s Speed and Parsing▫Logging in + staying logged in▫Gathering entire Friends/Like list▫Complete error handling (URLLib / Regex
Errors)▫Facebook probation
![Page 19: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/19.jpg)
19
EVC / PCC•Used the EVC implementation found in the
Stanford Network Analysis Platform (SNAP) and in Gephi
•Developed a PCC analysis tool using functions made available in the SNAP C++ Library
•Problems encountered:▫Poor documentation▫Typedef’s for everything
![Page 20: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/20.jpg)
20
Outline• Personal Experiences• Project Overview• Implementations
▫Scripts▫Gephi▫Web Crawler▫EVC / PCC
• Results▫Data Analysis
• Conclusion• Accomplishments• Future Work
![Page 21: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/21.jpg)
21
Data Analysis
![Page 22: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/22.jpg)
22
36%
6%
6%12%
10%
13%
5%
0% 6%
0% 1% 1% 1% 2%
College Membership
24%
7%
12%
28%
9%
10%
3%
2% 2%3%
EVC Top 20%
Data Analysis - EVC
![Page 23: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/23.jpg)
23
Data Analysis - PCC
36%
6%
6%12%
10%
13%
5%
0% 6%
0% 1% 1%1% 2%
College Membership
38%
8%18%
8%
15%
5%
3% 3% 3%
PCC Top 20 (3 components)
![Page 24: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/24.jpg)
24
Data Analysis
•Looked at friendship demographics for the top 20% ▫Education majors tend to cluster with
themselves▫Graduates brag about credentials more
than undergrads
•Kris plans to look at portion of friends for undergraduate vs. graduates and see how their demographics compare
![Page 25: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/25.jpg)
25
Outline• Personal Experiences• Project Overview• Implementations
▫Scripts▫Gephi▫Web Crawler▫EVC / PCC
• Results▫Data Analysis
• Conclusion• Accomplishments• Future Work
![Page 26: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/26.jpg)
26
Conclusion• Based on analysis from open source data sets, it
appears that a centrality measure that explicitly takes community membership into account is needed to determine the criticality of a node or nodes in a social network
• We developed a framework of principles for the new centrality measure▫Resilience against isolation due to edge severance▫Number of Inter/Extra-community connections▫Value of community membership▫Ability to direct resources to (influence) other nodes
![Page 27: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/27.jpg)
27
Outline• Personal Experiences• Project Overview• Implementations
▫Scripts▫Gephi▫Web Crawler▫EVC / PCC
• Results▫Data Analysis
• Conclusion• Accomplishments• Future Work
![Page 28: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/28.jpg)
28
Accomplishments
•Starting a research project from scratch•Learned Python•Built a web crawler•Learned to work with open source
frameworks•Survived REU experience
![Page 29: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/29.jpg)
29
Outline• Personal Experiences• Project Overview• Implementations
▫Scripts▫Gephi▫Web Crawler▫EVC / PCC
• Results▫Data Analysis
• Conclusion• Accomplishments• Future Work
![Page 30: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/30.jpg)
30
Future Work• Collect more data from Mizzou Facebook
• Improve the web crawler▫ Implement Parallel crawls▫ Make available for other researchers
• Refine our definition of social capital as needed
• Research on other social networks▫ Twitter, LinkedIn, etc.
• Using the previously mentioned principles, develop a new centrality method
![Page 31: Final Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun Zeng Investigating](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649d755503460f94a55d6d/html5/thumbnails/31.jpg)
31
Thank you!