National Statistics Center
Projects of Open Data for Official Statistics of Japan
Toshihiko AKATANI Deputy Director,
Corporate Development Office,
National Statistics Center
(NSTAC)
Workshop on the Communication of
Statistics
(Washington, D.C., United States of
America, 27 – 29 April 2015)
Table of Contents
1
I. Introduction
II.Project 1: Development of an
environment for advanced use of
statistics by API
III.Project 2: Improvement of statistics GIS
IV.Characteristics of Open Data policy in
an official statistics sector of Japan
V.Ideathon, Hackathon
VI.Future work
2
I. Introduction
II.Project 1: Development of an
environment for advanced use of
statistics by API
III.Project 2: Improvement of statistics GIS
IV.Characteristics of Open Data policy in
an official statistics sector of Japan
V.Ideathon, Hackathon
VI.Future work
Accessed:
approx. 18 million times (FY2013)
*Accesses by crawlers not included
Analysis of trends of users
Personal users: Approx. 50%
Private companies: Approx. 22%
Gov. offices: Approx. 15%
Unis and educational institutions:
Approx. 10%
Academic research institutes:
Approx. 3%
The “Portal Site of Official Statistics of Japan (e-Stat)” established in FY2008 provides
statistical tables of government agencies in a unified and integrated manner.
The database of “Fundamental Statistics” and other statistics has been established.
Past Open Data Initiatives in Statistics
3
Allows you to download statistics
charts or prepare various graphs
including population pyramid.
Allows you to examine in detail
questionnaires or their items for
statistical surveys.
The use of regional statistics
(statistics GIS) allows you to clearly
understand what the region looks like. ・Statistics tables
Approx. 1 million tables for approx.
500 government statistics
・Statistical information database
Approx. 70,000 tables for 40
Fundamental Statistics and 17
other statistics
Operation of e-Stat
NSTAC undertakes
operation and management
of the e-Stat.
INTERNET
The National Statistics Center is required to implement a one-stop service system to offer
statistical information to the citizens by means of operation and management of e-Stat.
Utilization rate of the system:
99.98%
(Working 24 hours, 365 days a year)
Submission
of statistical
information
Submission
of statistical
information
Local departments and
offices of ministries
Cabinet Office
Ministry of Internal Affairs and Communications
Ministry of Justice
Ministry of Finance
Ministry of Education, Culture, Sports, Science
and Technology
Ministry of Health, Labour
and Welfare
Ministry of Agriculture,
Forestry and Fisheries
Ministry of Economy,
Trade and Industry
Ministry of Land, Infrastructure, Transport
and Tourism
National Personnel
Authority
Ministry of the
Environment
The e-Stat permits users to execute such
tasks as:
Searching for statistics of all the
ministries.
Producting of statistics for a small area
unit.
Demonstrating graphs and aggregated
results of statistical data on a map
(Statistical GIS).
Accessing information about recent
entries, publication schedule and
publication history records of statistical
data.
A General Gateway to
Access Official Statistics
GAUSS Project
G Gateway to
A Advanced and
U User-
friendly
S Statistics
S Service
Carl Friedrich Gauss
(1777–1855)
GAUSS PROJECT
Projects of Open Data for Official Statistics
As the central statistical organizations, Statistics Bureau of Japan
(SBJ) and National Statistics Center (NSTAC) are promoting the
following themes which will upgrade the methods for disseminating
voluminous and diversified statistical data to the next-generation level,
and enable their advanced use.
2. Improvement of Statistics GIS
1. Development of an Environment for Advanced Use of
Statistics by API
This will promote advanced use of statistics by the public and
private sectors, support for the creation of services which generate
new added value and of innovative businesses, and so on. 6
7
I. Introduction
II.Project 1: Development of an
environment for advanced use of
statistics by API
III.Project 2: Improvement of statistics GIS
IV.Characteristics of Open Data policy in
an official statistics sector of Japan
V.Ideathon, Hackathon
VI.Future work
Development of an Environment for Advanced Use of Statistics by API
8
SBJ and NSTAC started API functions for official statistics on 31 October 2014 in order
to develop an environment for advanced use of statistical data.
No. of registered users: 3,101
No. of requests for statistical data: Approx. 21.82 million (As of 12 March 2015)
API (Application Programming Interface) function has been newly added to e-Stat
enabling the conversion of statistics data to machine readable data.
Statistics information database
A
P
I
INT
ER
NE
T
Info. system
of private sector
Info. system
of local gov.
Automatic
update
Automatic
update
Example 1:
Update data of e-Stat
automatically
Example 2:
Mash-up with other user data
or data available from the
internet Other info.
or services
Outline of API function
Presenting data in
a bubble graph
Presenting the latest
monthly data every
time the data base is
updated
Presenting data
in a bar graph
9
Analytical example using API functions
The statistics data of consumer spending on dining-out (results of Family Income and Expenditure Survey) by prefectural capital or ordinance-designated city is acquired by API and superimposed with the program available on the Internet (mash-up).
Applications of API Functions
Display various official statistics such as
Population Census in an easy-to-
understand manner
(Kyoto City) http://www2.city.kyoto.lg.jp/sogo/toukei/opendata/jised
ai/index.html
Estimation of property values, using
Population Census data
(Otani&Co., Inc.) http://geeo.otani.co/
10
11
I. Introduction
II.Project 1: Development of an
environment for advanced use of
statistics by API
III.Project 2: Improvement of statistics GIS
IV.Characteristics of Open Data policy in
an official statistics sector of Japan
V.Ideathon, Hackathon
VI.Future work
Improvement of Statistics GIS
12
The addition of functions enabling
①retrieving data held by user
②compiling statistics data in an arbitrarily designated area
Designated
Area(arbitrary)
Data held by
user
自社売上高
Statistics in the
designated area
To improve Statistics GIS on e-Stat, there is a new system “jSTAT MAP –Small area
analytics on maps–” that enables retrieving data held by users and analysis of statistics
data in an arbitrarily designated area.
No. of registered users: 2,228
No. of times logged in: Approx. 12,100 (As of 12 March 2015)
13 In the analysis, municipal information on evacuation buildings or shelters within a city in case of a disaster is
incorporated into the statistical GIS functions to display an estimate of the population within the area of an evacuation
site.
Imported
contents
Possible to enter
on the screen
Analytical example using statistical GIS functions
Plot the user’s data by geo-coding
which converts the address into the
coordinates of longitude and latitude.
Source: Muroran City Muroran Open Data Library
14
In the analysis, municipal information on evacuation buildings or shelters within a city in case of a
disaster is incorporated into the statistical GIS functions to display an estimate of the population
within the area of an evacuation site.
Analytical example using statistical GIS functions
Population within 300 meters around
an evacuation building is color-coded 0 or more but less than 500
500 or more but less than 750
750 or more but less than 1000
1000 or more but less than 1250
1250 or more
Total population
Examples of GIS Function (”Rich Report”)
Just by specifying a central point,
analytical results such as age structure
in the concentric zone are compiled as a
report in EXCEL format.
15
16
I. Introduction
II.Project 1: Development of an
environment for advanced use of
statistics by API
III.Project 2: Improvement of statistics GIS
IV.Characteristics of Open Data policy in
an official statistics sector of Japan
V.Ideathon, Hackathon
VI.Future work
Relationship between e-Stat and API functions
Stored in e-Stat
database
e-Stat (launched in FY2008)
Statistical data available in API functions corresponds with data in e-
Stat labeled “DB”
Acquisition of data using API
same data
June 2013 Japan Revitalisation Strategy – JAPAN is BACK – (adopted by the Cabinet)
◎ Make the 2-year period between FY2014 and FY2015 the intensive period for taking measures
◎ Launch the data catalog website (trial version)
◎ Offer the world’s top-level data sets for disclosure (more than 10,000) by the end of FY2015
Japanese Government’s Efforts for Open Data
18
Data Catalog Site “data.go.jp” has launched (Oct 1, 2014)
Datasets available in data.go.jp (as of 13 March 2015)
6638
5377
2829
555
498
235
218
146
69
264
PDF HTML XLS CSV ZIP XLSX JPEG XML KMZ OTHERS
20
I. Introduction
II.Project 1: Development of an
environment for advanced use of
statistics by API
III.Project 2: Improvement of statistics GIS
IV.Characteristics of Open Data policy in
an official statistics sector of Japan
V.Ideathon, Hackathon
VI.Future work
Ideathon and Hackathon using Statistical data in Japan
LOD Challenge 2014 SBJ and NSTAC : “Data Provision
Partners”
Winners of Ideathon and Hackathon,
held by SBJ and NSTAC internally
Official Smartphone App by SBJ “App on Statistics”
22
“App on Statistics” was released on 15 April 2014
The app interlocks the statistical API functions with the smartphone GPS to display the
statistics data of your current location, and provides other functions that allow you to feel
familiar with statistics data.
Displays statistics data of
your location using GPS
A location may also be
specified to display the
data
Major statistics data available so far:
• Population/households (Population Census)
• Number of private establishments and employees (Economic Census for
Business Activity)
• Major prices (Retail Price Survey)
• Monthly income and expenditure (Family Income and Expenditure Survey),
etc.
*Use of the API functions allows you to always acquire the latest statistics data.
Displays a list of
statistics data on
each item
City Stat
Statistics
Statistics Clock
Link to SBJ top page
Displays the statistical information
and quiz of the day
23
I. Introduction
II.Project 1: Development of an
environment for advanced use of
statistics by API
III.Project 2: Improvement of statistics GIS
IV.Characteristics of Open Data policy in
an official statistics sector of Japan
V.Ideathon, Hackathon
VI.Future work
Five Levels of Open Data
24
★ make your stuff available on the Web (whatever format) under an open license
★★ make it available as structured data (e.g., Excel instead of image scan of a
table)
★★★ use non-proprietary formats (e.g., CSV instead of Excel)
★★★★ use URIs to denote things, so that people can point at your stuff
★★★★★ link your data to other data to provide context
Tim Berners-Lee, the inventor of the Web and Linked Data initiator, suggested a 5 star
deployment scheme for Open Data
(Source: http://5stardata.info/)
Future Work and Some Implications
25
API functions
(Three stars)
Linkage to Other Data Sources
IMF OECD
Local gov. Data catalogue
the Highest Level
(Five stars)
At Present
In the
Future
Thank you for your attention!
26
API functions
http://www.e-stat.go.jp/api/
jSTAT MAP –Small area analytics on maps–
https://jstatmap.e-stat.go.jp/