reading china - policy change index•readership is dropping overtime. • government officials are...
TRANSCRIPT
![Page 1: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/1.jpg)
Reading China:
Predicting Policy Change with Machine Learning
Julian TszKin Chan (Bates White Economic Consulting)
Weifeng Zhong (Mercatus Center at George Mason University)
Federal Reserve Board (9/18/2019)
The views expressed here are the authors’ and do not represent the views of Bates White, the Mercatus Center, or their other employees.
policychangeindex.org
![Page 2: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/2.jpg)
Policy Change Index (PCI) is a machine learning program that predicts China’s policy moves by “reading” this:
What’s product of this research?
policychangeindex.org
![Page 3: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/3.jpg)
![Page 4: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/4.jpg)
State-run media tell us a lot — about the state!
• Example: People’s Daily ≈ the Pravda of China
• The primary mission is not to report news, but to propagate official viewpoints and signal policies —often in advance
policychangeindex.org
![Page 5: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/5.jpg)
State-run media tell us a lot — about the state!
policychangeindex.org
![Page 6: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/6.jpg)
State-run media tell us a lot — about the state!
• The Soviet tradition:
• “[T]he whole task of the Communists is to be able to convince
the backward elements.” (Lenin, 1920)
• It is a fundamental necessity “to transform the press… into a
serious organ for the economic education of the mass of the
population.” (Lenin, in early USSR years)
policychangeindex.org
![Page 7: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/7.jpg)
State-run media tell us a lot — about the state!
People's Daily: central to China’s propaganda system
Propaganda often precedes policies
Detect changes in newspaper’s priorities
Predict changes in gov’t policies
≈
↓
+
policychangeindex.org
![Page 8: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/8.jpg)
State-run media tell us a lot — about the state!
People's Daily: central to China’s propaganda system
Propaganda often precedes policies
Detect changes in newspaper’s priorities
Predict changes in gov’t policies
≈
↓
+
Machine learning
![Page 9: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/9.jpg)
Policy Change Index (PCI) project
• Data: People’s Daily text, 1946-present (2 million articles).
• Goal: Develop a machine learning program to detect changes in the newspaper’s priorities.
• Output: the Policy Change Index (PCI) for China.
policychangeindex.org
![Page 10: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/10.jpg)
Where’s the added value?
• A tech’ed-up version of “Kremlinology.”
• Deciphering propaganda content has been
complicated due to
• Opaque system;
• Manipulated data;
• Difficulty in validation;…
• Machine learning — with some twists — can help!
policychangeindex.org
![Page 11: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/11.jpg)
Data structure
policychangeindex.org
![Page 12: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/12.jpg)
Constructing the PCI-China
policychangeindex.org 12
![Page 13: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/13.jpg)
Start with something seemingly trivial
Build an algorithm that classifies front-page articles
(“reverse-engineering” the editor’s mind).
policychangeindex.org
![Page 14: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/14.jpg)
Start with something seemingly trivial
Build an algorithm that classifies front-page articles
(“reverse-engineering” the editor’s mind).
But what’s the point?
policychangeindex.org
![Page 15: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/15.jpg)
The framework
. . .
Articles in previous 5 years of period t
Training data Testing data
Frontpage𝑖 = 𝑓 𝑡 𝑡𝑒𝑥𝑡𝑖 Selecting hyper-
parameters
policychangeindex.org
![Page 16: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/16.jpg)
The framework
. . .
Articles in previous 5 years of period t
Training data Testing data
Frontpage𝑖 = 𝑓 𝑡 𝑡𝑒𝑥𝑡𝑖 Selecting hyper-
parameters
Articles in next quarter (t)
“Forecast”
Frontpage𝑖 = 𝑓 𝑡 𝑡𝑒𝑥𝑡𝑖
policychangeindex.org
![Page 17: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/17.jpg)
The framework
PCI ≫ 0 ⇒ Structural difference in the newspaper’s priorities
Test performance
“Forecast” performance
Policy Change Index of period t
. . .
Articles in previous 5 years of period t
Testing data
“Forecast” Training data
Articles in next quarter (t)
policychangeindex.org
![Page 18: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/18.jpg)
Modeling
Texts
Word embedding
Recurrent neural
networks
Metadata
Multilayer perceptron
Multilayer perceptron
Front page?
Input
Neural networks
Output
𝒙 : each article as an obs.
𝒚 = 𝒇 𝒙
𝒇 : a complicated function
policychangeindex.org
![Page 19: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/19.jpg)
Modeling
Texts
Word embedding
Recurrent neural
networks
Metadata
Multilayer perceptron
Multilayer perceptron
Front page?
Input
Neural networks
Output
𝒙 : each article as an obs.
𝒚 = 𝒇 𝒙
𝒇 : a complicated function
policychangeindex.org
![Page 20: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/20.jpg)
Results
policychangeindex.org 20
![Page 21: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/21.jpg)
PCI-China—detecting policy changes
![Page 22: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/22.jpg)
PCI-China—with ground truth
![Page 23: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/23.jpg)
PCI-China—with ground truth
![Page 24: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/24.jpg)
PCI for China — going forward
’18 Q1
’19 Q1
![Page 25: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/25.jpg)
Understanding changes
Content of misclassified articles has policy substance
• False negative: new policy
• False positive: phase-out policy
Classified on front page?
No Yes
Front page? No √ false positives
Yes false negatives √
policychangeindex.org
![Page 26: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/26.jpg)
Understanding changes — 2018 Q1 uptick
policychangeindex.org
![Page 27: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/27.jpg)
Understanding changes — 2018 Q1 uptick
• 2018 Q1 spike represents: • internally: strengthening party authority;
• externally: nationalism and global leadership;
• populist policies to boost political support.
• 2019 Q1 uptick: similar patterns.
• Our consistent prediction since January: curb your enthusiasm.
* 2019 Q3 update coming soon!
policychangeindex.org
![Page 28: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/28.jpg)
Discussion
policychangeindex.org
![Page 29: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/29.jpg)
Understanding policy priority: an infeasible approach
𝑔 ∶ 𝐴𝑟𝑡𝑖𝑐𝑙𝑒, 𝐹𝑟𝑜𝑛𝑡𝑃𝑎𝑔𝑒 → 𝑃𝑜𝑙𝑖𝑐𝑦, 𝑃𝑟𝑖𝑜𝑟𝑖𝑡𝑦
• With the learned function 𝑔:
• 𝑔 "pvt sector is important", front page = reform, high priority ;
• 𝑔 "central planning is great", front page = reform, low priority ;
…
• But where are the training data?
policychangeindex.org
![Page 30: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/30.jpg)
Understanding policy priority: a feasible approach
• Think of policy priorities as a latent variable:
𝒇 𝑃𝑜𝑙𝑖𝑐𝑦,𝑃𝑟𝑖𝑜𝑟𝑖𝑡𝑦 ∶ 𝑨𝒓𝒕𝒊𝒄𝒍𝒆 → 𝑭𝒓𝒐𝒏𝒕𝑷𝒂𝒈𝒆
• Lots of training data to learn each function 𝑓.
• Difference in function ⇒ difference in priorities.
• “Language-free!”
policychangeindex.org
![Page 31: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/31.jpg)
Discussion
• Adversarial attack
• If the Chinese government knows that we can detect their policy change based on the newspaper, would they change their behavior to avoid detection?
• That’s the purpose of propaganda
• What if the Chinese government knew we are reading the newspaper and want to fool us?
• Human judgement
• Readership is dropping overtime.
• Government officials are required to read the People’s Daily.
policychangeindex.org
![Page 32: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/32.jpg)
Other applications
policychangeindex.org
![Page 33: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/33.jpg)
Potential applications
• PCIs for other countries – North Korea, Cuba, …
• PCI-China sub-indices:
• By issue – trade policy sub-index, fiscal policy sub-index, …
• By country – China-US relations, China-EU relations, …
• Some countries may not control the media
• But politicians/policymakers still talk!
policychangeindex.org 33
![Page 34: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/34.jpg)
PCI-Crackdown
policychangeindex.org 34
![Page 35: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/35.jpg)
Policy Change Index (PCI) for Crackdown
policychangeindex.org 35
Protest-related People’s Daily
articles PCI algorithm
Predictor of potential
crackdown
(400+ articles on Tiananmen 1989 protest & Hong Kong 2019 protests)
(daily index update)
![Page 36: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/36.jpg)
PCI-Crackdown algorithm
policychangeindex.org 36
PCI-Crackdown : mapping current dates to Tiananmen timeline
. . .
Articles leading up to Tiananmen crackdown
Recent articles on HK protests
Calculate the “as-if” date
Train a date classifier
![Page 37: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/37.jpg)
PCI-Crackdown: 2019 Hong Kong protests
(Daily index, Jul 01 – Sep 15, 2019)
policychangeindex.org 37
![Page 38: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/38.jpg)
Interested in DIY?
• Website: policychangeindex.org
• Newsletter: mercatus.org/PCInewsletter
• Repository: github.com/PSLmodels/PCI
• A simulated example to show how the PCI works.
Questions?
policychangeindex.org
![Page 39: Reading China - Policy Change Index•Readership is dropping overtime. • Government officials are required to read the People’s Daily. policychangeindex.org . Other applications](https://reader033.vdocuments.site/reader033/viewer/2022060507/5f222608b256c701bd162bd8/html5/thumbnails/39.jpg)
State of the art model
• BERT (Devlin, et al. 2018)
• Machine learning algorithm is performing as good as human (88% accuracy) on language tests, such as:
• On stage, a woman takes a seat at the piano. She
• a) sits on a bench as her sister plays with the doll.
• b) smiles with someone as the music plays.
• c) is in the crowd, watching the dancers.
• d) nervously sets her fingers on the keys.
• the algorithm is not trained to perform those tests.
policychangeindex.org