eu egovernment benchmark

Dr. Alexander SchellongBerlin, 30/09/2011

EU E-Government Benchmark Benchmarking Open Government

Why benchmarking?

© 2011 Capgemini. All rights reserved. 2

EU E-GOVERNMENT BENCHMARK_BARCAMP.P

PTX

• Evaluate• Control• Budget• Motivate• Promote• Celebrate• Learn• Improve

Curent / Past EU eGovernment benchmark



PTX

Full-On line Availability

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Austria

Malta

Portug

al

Slovenia

Unite

d King

domNorw

ay

Swed

en

Germ

any

Estonia

France Ita

lySp

ain

Finlan

d

Denm

ark

Netherla

nds

Belgium

Czech

Rep

ubl ic

Turke

y

Hungary

Icelan

d

Irelan

d

Cyprus

Greec

e

Luxembo

urg

Lithu

ania

Romania

Slova

kiaLa

tvia

Polan

d

Switzerla

nd

Bul gar i

a

Full online availability EU27+

Country Reports

AnnexService Results & Definitions

•More informed / structured Landscaping

Management Summary

Pilot / Elective Measures:

- Service Directive

- Life Events

- PEGS3

National Portal E-procurement

4

Sophistication User-Centricity

12

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Austria

Slove

niaMalta

Portu

gal

Unite

d King

d omFra

nce

Sweden

Eston

ia

No rway

German

ySpa

in

Nethe

rlands

Finland

Belgi

um

Denm

arkIta

ly

Ireland

Czec

h Rep

u blic

Hungar

y

Turke

y

Greece

Iceland

Luxe

mbourg

Cypru

s

Lithua

nia

Switz

erland

Roman

ia

Slovakia

Latvi

a

Polan

d

Bulga

ria

Online sophistication EU27+

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Austria

Slovenia

Malta

Portuga

l

Un ited K

ingdom

Franc

e

Sweden

Eston

ia

Norway

Germany

Spain

Ne therland

s

Finland

Belgi

um

Denmark

Italy

Irelan

d

Czech R

epubli

c

Hungar

y

Tur ke

y

Greece

Icela

nd

Luxe

mbourg

Cypru

s

Lithu

ania

Switz

erland

Romania

Slovakia

Latvi

a

Poland

Bulga

ria

Online sophistication EU27+

Basic 20 Services

User Satisfaction5

Timetable EU eGovernment benchmark



PTX

March Kick OffApril Revision of measurements

Expert group peer review of approachMethodology & indicator definitions acceptedCountry list finalizedStart of execution of web surveys

May-June Execution of web surveysPilots

July-August Processing of PilotsQuality Control Findings

August-September Validation with member states

Meeting to review draft reportOct Completion of final reportNov: Presentation of report at e.g. Malmoe

Phase 1: Methodological preparations

Phase 2: Survey Benchmark Preparations

Phase 3: Conduct Benchmark study

Phase 4: Process results and reporting

March April May June July August September October

Methodology & Indicators

Agreed

Country DataObtained

Preliminary report

Final report

Phase planning & milestones

Basic 20 services: Sophistication / Online AvailabiityPolicy Goal: i2010 – Efficiency and Effectiveness, Service

Transformation



PTX

Commentary• A shift from a supply-side indicator towards a smart indicator: case-oriented view of e-government

services. Requires revision of: - List of services- Definition of services

• Value of International Expansion• Ongoing review of measurement procedures to ensure maximum burden reduction towards MS.

Current Position Indicator: 5-stages maturity modelStage 1 – InformationStage 2 - InteractionStage 3 - Two-way interactionStage 4 - TransactionStage 5 - Proactive and/or automatic delivery of service.Unit of analysis:List of URLs

2009Indicator: Indicator will remain stable in 2009 to allow for comparability.- Possible addition to review of 20 services: Pan-European interoperability, participation/consultationAction: - Approach requires rapid validation by Member State representatives.- Piloting in 2009 of alternative methods (e.g. webcrawlers)

Unit of analysis:.Action: Validation and Update of list of URLs is required.

Future Direction-To be evaluated post 2009 measurement-Gov 2.0 policy priorities, global policy priorities, life/customer events- Benchlearning exercise to measure connected government (Lot 2)

User Centricity (citizen satisfaction, business climate)Policy Goal: i2010 – No citizen left behind, Effectiveness of

online services



PTX

Commentary• Data fields subindicator failed• Accessibility testing (logo) was contested by many MS, Webcrawler methodology reviewing agreed

standards will provide much richer information• Extension of authentication subindicator to businesses, Investigation of relevance of STORCK to

Authentication• Benchlearning method for additional subindicators (link to Lot 2) beyond measurement Lot 1.

Basis: Capgemini measurements 2007

Current Position Indicator:- Authentication / eID- Compliance with

accessibility standards

- Number of data entry fields

- Multi-channel access

Unit of analysis:- Validated list of

URLs previously used

2009Indicator: shift to customer satisfactionReview of subindicators:- Data Field (AB Measure) will be dropped- Looking into feasibility of review of Accessibility of 20 services by Webcrawler- Identification of relevant maturity levels sub-transaction-Addition of user centricity review of national-level portalsAction: Agree on relevance of reporting on

citizen satisfaction / business climate

Future DirectionTo be re-evaluated post 2009 measurement

Strategic Alignment of methodology with user satisfaction measurement

eProcurement



PTX

20 public services rank distribution (2007) & (2009)



PTX

Online Public Service # ranks 2009 MS 1st 2009 # ranks 2007 MS 1st 2007

Job search services 2 29 out of 31 3 29 out of 31

VAT 2 30 out of 31 3 28 out of 31

Corporate tax 3 28 out of 31 4 26 out of 31

Customs declaration 3 29 out of 31 3 26 out of 31

Social contributions for employees 3 29 out of 31 4 25 out of 31

Income taxes 2 23 out of 31 3 20 out of 31

Declaration to the police 4 20 out of 31 4 18 out of 31

Public procurement 3 25 out of 31 3 16 out of 31

Submission of data to statistical offices 2 19 out of 31 4 15 out of 31

Registration of a new company 3 19 out of 31 4 15 out of 31

Car registration 5 21 out of 31 5 14 out of 31

Enrolment in higher education 15 15 out of 31 15 11 out of 31

Public libraries 8 14 out of 31 7 11 out of 31

Announcement of moving 9 13 out of 31 11 11 out of 31

Certificates 9 13 out of 31 13 9 out of 31

Environment-related permits 11 8 out of 31 10 5 out of 31

Health related services 11 7 out of 24 10 4 out of 23

Application for building permission 21 8 out of 31 21 4 out of 31

Personal documents 12 8 out of 31 11 2 out of 31

Social security benefits 18 9 out of 31 23 1 out of 31

Comparison of major eGovernment benchmarks / ranks



PTX

EUeGovBe (07/09)N: EU27+4

Brown/Brookings (2008)N: 198

UN (08/10)N: 184

Austria (1/4) Austria (65) Austria (16/24)

Slovenia (2/5) Slovenia (112) Slovenia (26/29)

Malta (2/1) Malta (56) Malta (29/30)

Portugal (3/2) Portugal (18) Portugal (31/39)

United Kingdom (4/8) United Kingdom (35) United Kingdom (10/4)

France (5/11) France (15) France (9/10)

Sweden (6/3) Sweden (72) Sweden (1/12)

Estonia (7/6) Estonia (59) Estonia (13/20)

Norway (8/15) Norway (60) Norway (3/6)

Germany (9/12) Germany (7) Germany (22/15)

Spain (9/13) Spain (14) Spain (20/9)

Netherlands (10/14) Netherlands (33) Netherlands (5/5)

Finland (11/7) Finland (28) Finland (15/19)

Belgium (12/12) Belgium (105) Belgium (24/16)

Denmark (12/10) Denmark (62) Denmark (2/7)

Italy (13/17) Italy (25) Italy (27/38)

Ireland (14/9) Ireland (8) Ireland (19/21)

Czech Republic (15/19) Czech Republic (55) Czech Republic (25/33)

Benchmark framework 1/3: Guiding principles



PTX

Dimensions DetailsPolicy Which policies should guide the design? e.g. i2011-15

Frameworks / Cause-and-effectrelationships

Which underlying frameworks guide the design? e.g. eGovernment,citizen satisfaction, government transformation, online sophisticationmodel, etc.

Priorities/Scope Which priorities? On a macro-level, what should be measured?Compared with what?

Governance & Responsibility e.g. How does the process of deciding on measures or guidingFrameworks look like? Which parties are involved (EC, MS,citizens,academia)?

Benchmark framework 2/3: design



PTX

Dimensions DetailsFramework e.g. Online sophistication model

Unit of analysis e.g. 20 public services and national portal in a country

Measures/variables/indicators Natural, proxy, constructed e.g. Service compliance with accessibilitystandard

Scoring and metric e.g. How are measures translated into values or ranks?

Frequency How often?

Data collection e.g. timeline, questions to ask, resources required on governemnt side

Data sources e.g. Who provides the data?; sample size

Data analysis e.g. Type qualitative and quantitative methods

Costs e.g. Value for money of the benchmarking activities

Limitations What are the weaknesses and gaps of the benchmark? Which data ismissing and how can it be made available?

Execution e.g. Who conducts data collection and analysis?

Benchmark framework 3/3: Learning and reporting



PTX

Dimensions DetailsResults e.g. Which results are presented? How are they presented?

Access e.g. Which data is accessible? Who has access? How can it beaccessed?

Overall Scope: Potential design

The future benchmark should have five focus areas

I. Government 2.0 / Open Government

II. European ICT projects

III. eGovernment services

IV. Information society & User opinion

V. Impact


EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX

Focus area I: Government 2.0 / Open Government

This area tracks the progress and innovations in five domains of Government 2.0 / Open government through a variety of indicators.

• Open data portals

• eParticipation (in policy; policy-cycle)

• Collaboration (co-production of service / public management support)

• Social media tools

• Transparency



Focus area II: European ICT projects

This area tracks the progress of European wide government ICT projects initiated by DGs or EU agencies through a variety of indicators.

• VIS (DG Home)

• Single Point of Contact 2.0 / Service Directive (DG Internal Market)

• IMI (DG Internal Market)

• INSPIRE (Environment)

• Safesea Net (EMSA)

• National alert platforms (Cyber Security; Digital Agenda)

• REACH (Environment)

• Prüm treaty data exchange



Focus area III: eGovernment services

This area introduces new eGovernment services that should be offered on a transactional basis as well as the implementation of EU Large Scale Pilot (e.g. eID/STORK) / European interoperability elements. Particular interest is on the life events of studying, working, retiring and receiving healthcare (Digital Agenda).

• eprocurement

• certificates

• building permits

• visa application

• enrolling in university

• application and management fo childcare / kindergarten



Focus area IV: Information society & User opinion



PTX

This area introduces a set of indicators that describe the state of information society and user opinions.

• user satisfaction surveys

• information society indicators (e.g. internet penetration, ICT 4 schools, ICT R&D investment)

Focus area V: Impact



PTX

This area introduces a comprehensive review of projects that are a case in point for X in a annually changing policy area.

• Significant budget savings

• eparticipation

• project failure

EU eGovernment index (Ranking)



PTX

NEWeGovernment index

Government 2.0

EU ICT Projects

eGov Services

INFSO / User

One eGovernment index with four subindexes that will compose the key eGovernment index.

5 sub indexes

8-10 sub indexes

5-10 sub indexes

5-10 sub indicators

Gov 2.0 pilot scope



PTX

Reference to previous measurement framework

Pillar in new measurement framework

Correspondence with eGovernment Action Plan 2012-2015 Priority

Group 1 of the Participation framework: Policy strategy and Monitoring

PARTICIPATION INDICATORInvolvement of citizens and businesses in policy-making processes

Group 3 of the Participation framework: Participation in policy making -- Openness of policy processGroup 5 of the Participation framework: Presence on social media

COLLABORATION INDICATOR Collaborative production of services

Group 1 of the Transparency framework: Transparency of Service delivery

TRANSPARENCY INDICATOR Improvement of TransparencyGroup 2 of the Participation framework: Passive participation -- Organizational transparency and accountability Group 3 of the Transparency framework: Personal data

Gov 2.0 pilot: collaboration indicators 1/3



PTX

Question Answer options Classification of indicator Data collection method

Does this administration involvecitizens in the co-production ofservices?

YesNo

Binary benchmark indicator0-100% availability score

Web survey of Institutional websites

If yes, for which services and how? YesNo

Binary benchmark indicator0-100% availability score (if a portfolio of services can be determined for the assessment)


Is it possible for citizens to track theirco-production inputs?

YesNo



Does the website provide evidence onhow the citizens’ input was used?

YesNo



Are the data and tools needed for coproduction up to date?

YesNo






PTX

Does the website include an an explicit social media policy by the administration?

YesNo

Binary benchmark indicator 0-100% availability score


In which of the following channels is the administration active?

• Social networking sites (like Facebook)• Media sharing sites (like Youtube)• Other (such as Second life)

YesNo

Binary benchmark, answers for each option0-100% availability Composite score (weighted by type of option)


Does the administration provide information and communication through the following tools?

• Tweeting• Blogging• Wikis• social bookmarking, tagging, canvasssing• polling / voting• petitioning• games• data visualization and/or analytics tools• other (please specify)

YesNo

Binary benchmark answers for each option0-100% availability Composite score (weighted by type of option)





PTX

Does the administration provide open data for mashing up new content, services, apps, etc.

YesNo

Binary benchmark answers 0-100% availability


Is the use of social media framed around:

• The institution• Specific topics/issues suggested

by the government• Specific topics/ issues suggested

by citizens or businesses• other (please specify)

YesNo

Binary benchmark answers for each option0-100% availability Composite score (weighted by type of option)


Open data portals measures



PTX

Question Variable Data provider

Is there a national OGD portal Yes / No Contractor

Is there a satistics portal Yes / no

Type of formats Formats (e.g. rtf, xls, csv)

SLF criteria: Completness MS

SLF criteria: primary contractor

SLF criteria: timeliness Not measurable

SLF criteria: Is there a registration contractor

SLF criteria: Is the registration fee of charge contractor

SLF criteria: Can disabled access the data contractor

SLF criteria: Tags and other navigation contractor

SLF criteria: machinereadable / API MS

Connected to social media platform Contractor

Statistics & metadata contractor

Open data portals measures



PTX

Question Variable Data provider

RSS feeds Yes / no contractor

Is there a forum? Yes / no contractor

Are users allowed to post apps? Yes / No Contractor

Are users allowed to rate user apps? Yes / no Contractor

Type of formats Formats (e.g. rtf, xls, csv) Contractor

Does the government have an open data policy Yes / no contractor

How should we measure?



PTX

• Open data

• eParticipation (policy)

• collaboration (service)

• Transparency

• Impact



PTX

Thank [email protected]

Blog: http://www.iq.harvard.edu/blog/netgov

Twitter: /schellong

http://www.iq.harvard.edu/blog/netgov

http://www.iq.harvard.edu/blog/netgov

eu egovernment benchmark

Government & Nonprofit