deploying and maintaining ai in a socio-technical system · the rise and decline of an open...
TRANSCRIPT
![Page 1: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/1.jpg)
Deploying and maintaining AI in a socio-technical
system
Aaron HalfakerWikimedia Research
??
![Page 2: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/2.jpg)
![Page 3: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/3.jpg)
![Page 4: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/4.jpg)
![Page 5: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/5.jpg)
Outline
![Page 6: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/6.jpg)
Outline
1. ORES’ vision: Efficiency and innovation
160kEditsPer Day
10%
90%
Need review
Probably OK
![Page 7: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/7.jpg)
Outline
1. ORES’ vision: Efficiency and innovation
2. The social dangers of AI realized!
![Page 8: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/8.jpg)
Outline
1. ORES’ vision: Efficiency and innovation
2. The social dangers of AI realized!
3. Intuitive evaluation -- AKA, how about we ask the humans?
Get feedback Iterate Deploy
![Page 9: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/9.jpg)
Part 1: ORES’ vision
![Page 10: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/10.jpg)
What’s ORES?
![Page 11: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/11.jpg)
What’s ORES?Why is ORES?
![Page 12: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/12.jpg)
2016201520142013201220112010200920082007
![Page 13: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/13.jpg)
2016201520142013201220112010200920082007
![Page 14: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/14.jpg)
2016201520142013201220112010200920082007
![Page 15: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/15.jpg)
2016201520142013201220112010200920082007
![Page 16: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/16.jpg)
2016201520142013201220112010200920082007
![Page 17: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/17.jpg)
2016201520142013201220112010200920082007
![Page 18: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/18.jpg)
2016201520142013201220112010200920082007
![Page 19: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/19.jpg)
2016201520142013201220112010200920082007
ORES
![Page 20: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/20.jpg)
References● Suh, B., Convertino, G., Chi, E. H., & Pirolli, P. (2009, October). The singularity is not near: slowing growth of Wikipedia. In Proceedings of
the 5th International Symposium on Wikis and Open Collaboration (p. 8). ACM.● Halfaker, A., Kittur, A., Kraut, R., & Riedl, J. (2009, October). A jury of your peers: quality, experience and ownership in Wikipedia. In
Proceedings of the 5th International Symposium on Wikis and Open Collaboration (p. 15). ACM.● https://meta.wikimedia.org/wiki/Research:Wikimedia_Summer_of_Research_2011 ● https://meta.wikimedia.org/wiki/Research:Teahouse ● Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2012). The rise and decline of an open collaboration system: How Wikipedia’s reaction
to popularity is causing its decline. American Behavioral Scientist, 0002764212469365.● https://www.youtube.com/watch?v=bwMBLrAHJLM (Snuggle @ Wikimania 2013)● https://www.mediawiki.org/wiki/File:Wikimedia_Research_%26_Data_Showcase_-_February_2014.webm (Wikipedia article creation
analysis)● https://www.youtube.com/watch?v=AUupsnvV1oA#t=35m26s (Articles for Creation workflow analysis)● https://www.youtube.com/watch?v=-We4GZbH3Iw#t=34m10s (Wikipedia as a socio-technical system “The Paramecium Talk”)● https://www.youtube.com/watch?v=Hj7o5d-OEis#t=3m25s (ORES -- The people’s classifier service)
![Page 21: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/21.jpg)
References● Suh, B., Convertino, G., Chi, E. H., & Pirolli, P. (2009, October). The singularity is not near: slowing growth of Wikipedia. In Proceedings of
the 5th International Symposium on Wikis and Open Collaboration (p. 8). ACM.● Halfaker, A., Kittur, A., Kraut, R., & Riedl, J. (2009, October). A jury of your peers: quality, experience and ownership in Wikipedia. In
Proceedings of the 5th International Symposium on Wikis and Open Collaboration (p. 15). ACM.● https://meta.wikimedia.org/wiki/Research:Wikimedia_Summer_of_Research_2011 ● https://meta.wikimedia.org/wiki/Research:Teahouse ● Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2012). The rise and decline of an open collaboration system: How Wikipedia’s reaction
to popularity is causing its decline. American Behavioral Scientist, 0002764212469365.● https://www.youtube.com/watch?v=bwMBLrAHJLM (Snuggle @ Wikimania 2013)● https://www.mediawiki.org/wiki/File:Wikimedia_Research_%26_Data_Showcase_-_February_2014.webm (Wikipedia article creation
analysis)● https://www.youtube.com/watch?v=AUupsnvV1oA#t=35m26s (Articles for Creation workflow analysis)● https://www.youtube.com/watch?v=-We4GZbH3Iw#t=34m10s (Wikipedia as a socio-technical system “The Paramecium Talk”)● https://www.youtube.com/watch?v=Hj7o5d-OEis#t=3m25s (ORES -- The people’s classifier service)
![Page 22: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/22.jpg)
References● Suh, B., Convertino, G., Chi, E. H., & Pirolli, P. (2009, October). The singularity is not near: slowing growth of Wikipedia. In Proceedings of
the 5th International Symposium on Wikis and Open Collaboration (p. 8). ACM.● Halfaker, A., Kittur, A., Kraut, R., & Riedl, J. (2009, October). A jury of your peers: quality, experience and ownership in Wikipedia. In
Proceedings of the 5th International Symposium on Wikis and Open Collaboration (p. 15). ACM.● https://meta.wikimedia.org/wiki/Research:Wikimedia_Summer_of_Research_2011 ● https://meta.wikimedia.org/wiki/Research:Teahouse ● Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2012). The rise and decline of an open collaboration system: How Wikipedia’s reaction
to popularity is causing its decline. American Behavioral Scientist, 0002764212469365.● https://www.youtube.com/watch?v=bwMBLrAHJLM (Snuggle @ Wikimania 2013)● https://www.mediawiki.org/wiki/File:Wikimedia_Research_%26_Data_Showcase_-_February_2014.webm (Wikipedia article creation
analysis)● https://www.youtube.com/watch?v=AUupsnvV1oA#t=35m26s (Articles for Creation workflow analysis)● https://www.youtube.com/watch?v=-We4GZbH3Iw#t=34m10s (Wikipedia as a socio-technical system “The Paramecium Talk”)● https://www.youtube.com/watch?v=Hj7o5d-OEis#t=3m25s (ORES -- The people’s classifier service)
![Page 23: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/23.jpg)
Why is ORES?● Wikipedia has socio-technical problems with newbies.
![Page 24: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/24.jpg)
Why is ORES?● Wikipedia has socio-technical problems with newbies.● Many of Wikipedia’s problems are due to its scale.
https://commons.wikimedia.org/wiki/File:Flickr_-_Official_U.S._Navy_Imagery_-_Sailor%27s_daughter_operates_a_fire_hose_with_crew_member_assistance..jpg
![Page 25: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/25.jpg)
Why is ORES?● Wikipedia has socio-technical problems with newbies.● Many of Wikipedia’s problems are due to its scale.● ORES is an attempt to address both at the same time.
Progress ----->
Ener
gy
Progress catalyst
![Page 26: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/26.jpg)
Why is ORES?● Wikipedia has socio-technical problems with newbies.● Many of Wikipedia’s problems are due to its scale.● ORES is an attempt to address both at the same time.● … in a way that accounts for the complex dynamics of Wikipedia.
~System with specialized
sub-systemsSystem with specialized
sub-systems
![Page 27: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/27.jpg)
Why is ORES?● Wikipedia has socio-technical problems with newbies.● Many of Wikipedia’s problems are due to its scale.● ORES is an attempt to address both at the same time.● … in a way that accounts for the complex dynamics of Wikipedia.
What is ORES?
![Page 28: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/28.jpg)
The machine classifier
https://commons.wikimedia.org/wiki/File:Artificial.intelligence.jpgPublic Domain
![Page 29: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/29.jpg)
The machine classifier
is_anonchrs_addedchrs_removedcust_commentrepeated_chrslongest_tokenbadwords_added
![Page 30: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/30.jpg)
The machine classifier
is_anonchrs_addedchrs_removedcust_commentrepeated_chrslongest_tokenbadwords_added
?
![Page 31: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/31.jpg)
BAD!
The machine classifier
is_anonchrs_addedchrs_removedcust_commentrepeated_chrslongest_tokenbadwords_added
?
Good.
![Page 32: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/32.jpg)
160kEditsPer Day
10%
90%
Need review
Probably OK
Counter vandalism
![Page 33: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/33.jpg)
Without ORES: Reviewing 160k edits per day...
267 Hours (33 people * 8 hours)
With ORES: Reviewing 16k edits per day…
27 Hours(4 people * 8 hours)
![Page 34: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/34.jpg)
Newcomer socialization
1500New EditorsPer Day
20% Vandals
Doing fine
* New editors = “Newly registered users who have saved at least one edit”
40%
40%
Need help
![Page 35: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/35.jpg)
Future work:Make estimates for the amount of time spent socializing a newcomer…
![Page 36: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/36.jpg)
Some day, ORES will make everything easier...
https://commons.wikimedia.org/wiki/File:P%C3%A4ij%C3%A4nne_and_p%C3%A4ij%C3%A4tsalo.jpg (Lake Päijänne at Sysmä, Finland.)
![Page 37: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/37.jpg)
Getting there● Production level web service @ https://ores.wikimedia.org
● Basic counter-vandalism support (ORES review tool) deployed on 6 wikis & active projects with the WMF Collaboration Team and with tool Developers
● 20 wikis and 4 prediction models○ reverted?○ good-faith?○ damaging?○ wp10 assessment?
![Page 38: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/38.jpg)
Part 2: The social dangers of AI realized
http
s://c
omm
ons.
wikim
edia
.org
/wiki
/File
:PEO
-mon
ster
.svg
![Page 39: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/39.jpg)
“Subjective Algorithms”"algorithms, often aided by big data, now make decisions in subjective realms where there is no right decision, and no anchor with which to judge outcomes."Tufekci, Z. (2015). Algorithms in our Midst: Information, Power
and Choice when Software is Everywhere. CSCW (pp. 1918-1918). ACM.
“Subjective algorithms”
https://commons.wikimedia.org/wiki/File:Zeynep_Tufekci_2_(5805394290).jpg
![Page 40: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/40.jpg)
“Subjective Algorithms”
What is good? relevant? important? desirable? valuable?
"algorithms, often aided by big data, now make decisions in subjective realms where there is no right decision, and no anchor with which to judge outcomes."Tufekci, Z. (2015). Algorithms in our Midst: Information, Power
and Choice when Software is Everywhere. CSCW (pp. 1918-1918). ACM.
“Subjective algorithms”
https://commons.wikimedia.org/wiki/File:Zeynep_Tufekci_2_(5805394290).jpg
![Page 41: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/41.jpg)
Who is allowed to participate? Who gets labeled “bad-faith”?
What types of contributions will be labeled “damaging”?
![Page 42: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/42.jpg)
BAD!
is_anonchrs_addedchrs_removedcust_commentrepeated_chrslongest_tokenbadwords_added
?
Good.
![Page 43: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/43.jpg)
Low quality
is_anonchrs_addedchrs_removedcust_commentrepeated_chrslongest_tokenbadwords_added
?
High quality
![Page 44: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/44.jpg)
Civil
is_anonchrs_addedchrs_removedcust_commentrepeated_chrslongest_tokenbadwords_added
?
Harassment
![Page 45: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/45.jpg)
Please exercise extreme caution to avoid encoding racism or other biases into an AI scheme. [...] Wnt (talk) 12:58, 20 February 2015 (UTC)From Wikipedia:Wikipedia_Signpost/2015-02-18/Special_report
![Page 46: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/46.jpg)
Please exercise extreme caution to avoid encoding racism or other biases into an AI scheme. [...] Wnt (talk) 12:58, 20 February 2015 (UTC)From Wikipedia:Wikipedia_Signpost/2015-02-18/Special_report
?
![Page 47: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/47.jpg)
Civil
Harassment
Please exercise extreme caution to avoid encoding racism or other biases into an AI scheme. [...] Wnt (talk) 12:58, 20 February 2015 (UTC)From Wikipedia:Wikipedia_Signpost/2015-02-18/Special_report
?and messages from South Africans
![Page 48: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/48.jpg)
Two stories
Damaging edit
is_anonchrs_addedchrs_removedcust_commentrepeated_chrslongest_tokenbadwords_added
?
Good edit
![Page 49: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/49.jpg)
Two stories
Damaging edit● The Italian word “ha”● Anonymous editors
is_anonchrs_addedchrs_removedcust_commentrepeated_chrslongest_tokenbadwords_added
?
Good edit
![Page 50: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/50.jpg)
The Italian “ha”
Literally: Not a laughing matter
![Page 51: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/51.jpg)
:it:Progetto:Patrolling/ORES
:m:Talk:ORES#Checklist for itwiki setup.
![Page 52: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/52.jpg)
Damaging edit
...
informals_added
badwords_added
...
?
Good edit
![Page 53: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/53.jpg)
Damaging edit
...
informals_added
badwords_added
...
?
Good edit
Badwords: Curse words, racial slurs and other offensive terminology
Informals: Casual speak that would be welcome on a talk page, but not within an article.
![Page 54: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/54.jpg)
Damaging edit
...
informals_added
badwords_added
...
?
Good edit
Badwords: Curse words, racial slurs and other offensive terminology
Informals: Casual speak that would be welcome on a talk page, but not within an article.
For example "hello" or "hahaha".
![Page 55: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/55.jpg)
Badwords: Curse words, racial slurs and other offensive terminology
Informals: Casual speak that would be welcome on a talk page, but not within an article.
Damaging edit
...
informals_added
badwords_added
...
?
Good edit
For example "hello" or "hahaha".
![Page 56: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/56.jpg)
revscoring/languages/english.py#L163 revscoring/languages/tests/test_english.py#L103
![Page 57: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/57.jpg)
revscoring/languages/english.py#L163 revscoring/languages/tests/test_english.py#L103
“Ha” is laughing in English, but “Ha” is not laughing in Italian!
![Page 58: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/58.jpg)
… <snip> ...
![Page 59: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/59.jpg)
https://commons.wikimedia.org/wiki/File:Enduring_Freedom_120707-A-DL472-175.jpg Public Domain (US Gov)
![Page 60: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/60.jpg)
Anonymous editors
![Page 61: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/61.jpg)
Anonymous editors
Otherwise, anons seemed to dominate false-positive reports from every wiki
![Page 62: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/62.jpg)
… maybe anons are really bad.
![Page 63: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/63.jpg)
… maybe anons are really bad.
● Generally, anon edits are twice as likely to be vandalism
![Page 64: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/64.jpg)
… maybe anons are really bad.
● Generally, anon edits are twice as likely to be vandalism
● 90% of anonymous edits are good
![Page 65: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/65.jpg)
https://ores.wmflabs.org/v2/scores/enwiki/damaging/642345235?feature.revision.user.is_anon=false
![Page 66: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/66.jpg)
https://ores.wmflabs.org/v2/scores/enwiki/damaging/642345235?feature.revision.user.is_anon=false
{"prediction": false, "probability": {"false": 0.656, " true": 0.344}}
![Page 67: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/67.jpg)
https://ores.wmflabs.org/v2/scores/enwiki/damaging/642345235?feature.revision.user.is_anon=false
{"prediction": false, "probability": {"false": 0.656, " true": 0.344}}
https://ores.wmflabs.org/v2/scores/enwiki/damaging/642345235?feature.revision.user.is_anon=true
{"prediction": false, "probability": {"false": 0.541, " true": 0.459}}
![Page 68: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/68.jpg)
https://ores.wmflabs.org/v2/scores/enwiki/damaging/642345235?feature.revision.user.is_anon=false
{"prediction": false, "probability": {"false": 0.656, " true": 0.344}}
https://ores.wmflabs.org/v2/scores/enwiki/damaging/642345235?feature.revision.user.is_anon=true
{"prediction": false, "probability": {"false": 0.541, " true": 0.459}}
Just by being “anon”, we score this edit 11.5% more likely to be damaging to the article.
![Page 69: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/69.jpg)
Anon editor{"feature.revision.user.is_anon": true, "feature...seconds_since_registration": 0, "feature.revision.user.has_advanced_rights": false, "feature.revision.user.is_admin": false, "feature.revision.user.is_bot": false, "feature.revision.user.is_curator": false}
New editor (2h since registration){"feature.revision.user.is_anon": false, "feature...seconds_since_registration": 18000, "feature.revision.user.has_advanced_rights": false, "feature.revision.user.is_admin": false, "feature.revision.user.is_bot": false, "feature.revision.user.is_curator": false}
User:EpochFail (8 years since registration) {"feature.revision.user.is_anon": false, "feature...seconds_since_registration": 257995021, "feature.revision.user.has_advanced_rights": false, "feature.revision.user.is_admin": false, "feature.revision.user.is_bot": false, "feature.revision.user.is_curator": false}
User classesModeling strategies
https://en.wikipedia.org/wiki/Gradient_boosting
https://en.wikipedia.org/wiki/Support_vector_machine#Linear_SVM
![Page 70: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/70.jpg)
![Page 71: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/71.jpg)
![Page 72: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/72.jpg)
Gradient Boosting Linear SVMNatural user class
![Page 73: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/73.jpg)
Gradient Boosting Linear SVMNatural user class
![Page 74: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/74.jpg)
Gradient Boosting Linear SVMNatural user class
![Page 75: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/75.jpg)
???
Gradient Boosting Linear SVMNatural user class
![Page 76: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/76.jpg)
OK. What if every edit was anon?
![Page 77: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/77.jpg)
Gradient Boosting Linear SVMAnon user class
![Page 78: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/78.jpg)
Gradient Boosting Linear SVMAnon user class
![Page 79: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/79.jpg)
Gradient Boosting Linear SVMAnon user class
![Page 80: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/80.jpg)
What if every edit were from a newcomer?
![Page 81: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/81.jpg)
Gradient Boosting Linear SVMNewbie user class
![Page 82: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/82.jpg)
Gradient Boosting Linear SVMNewbie user class
![Page 83: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/83.jpg)
Gradient Boosting Linear SVMNewbie user class
![Page 84: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/84.jpg)
What if I (EpochFail) saved all the edits?
![Page 85: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/85.jpg)
Gradient Boosting Linear SVMEpochFail user class
![Page 86: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/86.jpg)
Gradient Boosting Linear SVMEpochFail user class
![Page 87: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/87.jpg)
Gradient Boosting Linear SVMEpochFail user class
![Page 88: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/88.jpg)
Gradient Boosting Linear SVMEpochFail user class
![Page 89: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/89.jpg)
Mwahahahaha!
![Page 90: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/90.jpg)
Dec. 2015: - Gradient boosting deployed
![Page 91: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/91.jpg)
Dec. 2015: - Gradient boosting deployed
Still needs work:- Bias against anons/newcomers lessened,
but not gone- New sources of signal
- HashingVectorization- Probabilistic Context-free Grammars
![Page 92: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/92.jpg)
Part 3: A call to actionHow about we ask the humans?
![Page 93: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/93.jpg)
Evaluation of AI
![Page 94: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/94.jpg)
Evaluation of AI
● Historically, focused on quantitative metrics:○ Accuracy, Precision, Recall○ F-score○ ROC-AUC○ PR-AUC○ Etc.
![Page 95: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/95.jpg)
Evaluation of AI
● Historically, focused on quantitative metrics:○ Accuracy, Precision, Recall○ F-score○ ROC-AUC○ PR-AUC○ Etc.
“How well does my classifier work in general?”
![Page 96: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/96.jpg)
Evaluation of AI
“How well does my classifier work in general?”
Both work pretty good in general...
Gradient Boosting Linear SVM
![Page 97: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/97.jpg)
Evaluation of AI
“Does my classifier behave strangely sometimes?”
Both work pretty good in general...
Gradient Boosting Linear SVM
???
![Page 98: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/98.jpg)
Step 1:
“ORES is an experimental technology. We encourage you to take advantage of it but also to be skeptical of the predictions made. It's a tool to support you – it can't replace you. Please reach out to us with your questions and concerns.
”
![Page 99: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/99.jpg)
Step 1:
“ORES is an experimental technology. We encourage you to take advantage of it but also to be skeptical of the predictions made. It's a tool to support you – it can't replace you. Please reach out to us with your questions and concerns.
”Step 2:Get feedback Iterate & retrain Deploy
![Page 100: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/100.jpg)
Insight:“Despite the black-box nature of AI, humans are really good at noticing patterns and trends.”
![Page 101: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/101.jpg)
https://commons.wikimedia.org/wiki/File:Bill_Gates_mugshot.png
![Page 102: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/102.jpg)
?
https://commons.wikimedia.org/wiki/File:Bill_Gates_mugshot.png
?
![Page 103: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/103.jpg)
Human evaluation of AI’s bias should be part of standard practice for AI designers working in social contexts.
![Page 104: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/104.jpg)
![Page 105: Deploying and maintaining AI in a socio-technical system · The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline](https://reader030.vdocuments.site/reader030/viewer/2022040706/5e04e9bc950e6b703378ba82/html5/thumbnails/105.jpg)
??
Thanks!
SCIENCE
SCIENCE
SCIENCESCIENCE
SCIENCE
SCIENCE SCIENCE
Props to my collaborators● User:Ladsgroup● User:Aetilley● Sabya● User:Rotpunkt● User:Eran
Aaron Halfaker
enwp.org/User:EpochFail
https://twitter.com/halfak