empowering citizens to transform european public ......d3.8 - final citadel methodology for...

D3.8 - Final CITADEL Methodology for co-creation Version 1.0– Final. Date: 31.03.2019

Project Title: CITADEL Contract No. GA 726755 www.citadel-h2020.eu of 125

Empowering Citizens to Transform European Public Administrations

Deliverable D3.8

Final CITADEL Methodology for co-creating a public service

Editor(s): Wim Vanobberghen (imec) Marisa Escalante (TECNALIA)

Responsible Partner: imec

Status-Version: Final v1.0

Date: 31/03/2019

Distribution level (CO, PU): Public

http://www.citadel-h2020.eu/



Project Number: GA 726755

Project Title: CITADEL

Title of Deliverable: D3.8 Final CITADEL Methodology for co-creating a public service

Due Date of Delivery to the EC: 31/03/2019

Workpackage responsible for the Deliverable:

WP3 - Co-create to transform

Editor(s): Imec

TECNALIA

Contributor(s):

Vanobberghen, Wim (imec)

Escalante Martinez, Marisa (TECNALIA)

Llamosa Gonzalo (Cantabria University)

Reviewer(s): Pieter Vermeyen (Antwerpen)

Approved by: All Partners

Recommended/mandatory readers:

WP4 and WP5

Abstract: This document will present the final version of the methodology to facilitate the co-creation of citizen-centric public services involving all the CITADEL Stakeholders.

Keyword List: Co-creation, methodology, activities, methods, tools

Licensing information: This work is licensed under Creative Commons Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0)

http://creativecommons.org/licenses/by-sa/3.0/

Disclaimer This document reflects only the author’s views and neither Agency nor the Commission are responsible for any use that may be made of the information contained therein


http://creativecommons.org/licenses/by-sa/3.0/



Document Description

Document Revision History

Version Date Modifications Introduced

Modification Reason Modified by

v0.0 12/02/2019 First draft version based on D3.7 and added new section

Imec, TECNALIA

v0.2 25/02/2019 Review of Phase 1 and Phase 3 TECNALIA

V0.3 26/02/2019 Review of Phase 2 and Phase 4 imec

V0.4 06/03/2019 Section 3.1 UC

V0.5 07/03/2019 Improvements in section 3 TECNALIA

V0.6 13/03/2019 Classification of the tools and methods TECNALIA & imec

V0.7 14/03/2019 Improvements in the classification TECNALIA & imec

V0.8 15/03/2019 First version sent to internal reviewer TECNALIA

V0.9 20/03/2019 Addition of new categorisation and inclusion of annex 3 and 4

TECNALIA & imec & UC

V0.10 21/03/2019 Final version to review TECNALIA &imec

V0.11 25/03/2019 Changes after internal review TECNALIA

V1.0 31/03/2019 Ready for submission TECNALIA




Table of Contents

Table of Contents .......................................................................................................................... 4

List of Figures ................................................................................................................................ 5

List of Tables .................................................................................................................................. 5

1 Introduction .......................................................................................................................... 8

1.1 Document Structure ...................................................................................................... 8

2 CITADEL Co-Creation methodology ..................................................................................... 10

2.0 Activity 0: Data Protection & Privacy .......................................................................... 10

2.1 Phase 1: Ideation and research ................................................................................... 11

2.1.1 Activity 1.1: Identification of Needs. ..................................................................... 11

2.1.2 Activity 1.2: Prioritise the citizens´ needs ............................................................. 12

2.1.3 Activity 1.3: Analyse potential users ..................................................................... 13

2.1.4 Activity 1.4: Launch the challenge to address citizens´ need ................................ 14

2.1.5 Activity 1.5: Creation and selection of the ideas to solve the challenges. ............ 15

2.2 Phase 2: Concept and Design ...................................................................................... 17

2.2.1 Activity 2.1: Environmental Analysis ..................................................................... 17

2.2.2 Activity 2.2: Define your Co-creation Trajectory ................................................... 18

2.2.3 Activity 2.3: Recruit Participants ........................................................................... 19

2.2.4 Activity 2.4 : Conceptualise ................................................................................... 20

2.2.5 Activity 2.5: What-to-prototype ............................................................................ 21

2.3 Phase 3: Development and implementation .............................................................. 23

2.3.1 Activity 3.1 Prototype ............................................................................................ 23

2.3.2 Activity 3.2: Develop .............................................................................................. 24

2.3.3 Activity 3.3: Implement ......................................................................................... 25

2.4 Phase 4: Production and maintenance ....................................................................... 26

2.4.1 Activity 4.1: Launch ............................................................................................... 27

2.4.2 Activity 4.2 Evaluate .............................................................................................. 28

2.4.3 Activity 4.3 Iterate ................................................................................................. 28

3 Rules for customisation ....................................................................................................... 30

3.1 Findings from the Systematic Literature Review ........................................................ 30

3.2 Rules to customise the methodology .......................................................................... 30

3.3 Questionnaire to customise the methodology ........................................................... 33

4 Conclusion ........................................................................................................................... 36

5 References ........................................................................................................................... 37

Annex 1: Supporting Methods .................................................................................................... 38

Annex 2: Supporting Tools .......................................................................................................... 80




Annex 3: ICT-based co-production of public services: Barrier and Enablers. A systematic review. ................................................................................................................................................... 118

Annex 4: Explaining citizens’ co-production through ICTs in health: A multi-level model analysis. ................................................................................................................................................... 121

Annex 5: Additional Resources.................................................................................................. 124

List of Figures

FIGURE 1. PHASE 1 WORKFLOW ........................................................................................................ 11 FIGURE 2. PHASE 2 WORKFLOW ........................................................................................................ 17 FIGURE 3. PHASE 3 WORKFLOW ........................................................................................................ 23 FIGURE 4. PHASE 4 WORKFLOW ........................................................................................................ 27

List of Tables

TABLE 1. CC-C TERMINOLOGY LAID DOWN IN D3.5 [1] ............................................................................ 8 TABLE 2. RULES FOR CUSTOMISATION ................................................................................................. 32 TABLE 3. CITADEL CO-CREATION METHODOLOGY – ADDITIONAL RESOURCES ........................................ 124




Terms and abbreviations

Cc-c CITADEL Co-Creation

GDPR General Data Protection Regulation

ICT Information and Communication Technology

PA Public Administration

DPO Data Protection Officer

DPIA Data Privacy Impact Analysis

DIGIMAT Digital Maturity Assessment Tool

KPI Key Performance Indicators

CIP CITADEL Innovation Platform

SILK Social Innovation Lab Kent

CTA Constructive Technology Assessment




Executive Summary

Co-creation in CITADEL is, as explained in D3.5 [1], a holistic approach to developing a (public) service together with all the stakeholders, a process starting with the conception of individual projects and continuing through to adoption, communication and evaluation. To help practitioners in public administrations with the challenging task of implementing co-creation in their realities, a methodology is being developed together with an IT-supported guide and toolbox.

This document D3.8 “Final CITADEL Methodology for co-creating a public service” is an update of D3.7 “Initial CITADEL Methodology for co-creating a public service” [2] and a continuation of the D3.6 “Final Requirements for Co-creation” [3] which were due in M24. This deliverable improves the co-creation methodology from the D3.7 but the most important advance in this deliverable is the section dedicated to the customisation of the methodology. The categorization of the methods and tools in six categories allow to select the most appropriate set of methods and tools for a specific co-creation trajectory project. To facilitate this customisation, a questionnaire has been created to gather context information of each co-creation trajectory project.

The final methodology, as laid down in detail in this document, is structured into 4 phases: Ideation & Research, Concept & Design, Development & Implementation, Production & Maintenance. Each phase is subdivided into practical co-creation activities, which need to be performed to move a project through the phases, linked to specific methods, techniques and tools classified in the following categories: time, budget, number of participants, technical complexity, project complexity.

The final methodology will be supported by a tool that it going to be part of the CITADEL Ecosystem (WP4).




Introduction

This document D3.8 “Final CITADEL Methodology for co-creating a public service” is a continuation of D3.7 “Initial CITADEL Methodology for co-creating a public service” and D3.6 “Final Requirements for Co-creation” which were due in M24. This deliverable improves the co-creation methodology from the D3.7 but the most important advance in this deliverable is the section dedicated to the customisation of the methodology. The categorization of the methods and tools in six categories allow to select the most appropriate set of methods and tools for a specific co-creation trajectory project. To facilitate this customisation a questionnaire has been created to gather context information of the co-creation project.

The CITADEL Co-creation methodology is structured into 4 phases: Ideation & Research, Concept & Design, Development & Implementation, Production & Maintenance. Each phase is subdivided into practical co-creation activities, linked to specific methods, techniques and tools, which move a project through the four phases. An explanation of the terminology can be found in the table below (see Table 1) and, in more detail, in D3.5 [1].

Table 1. Cc-c Terminology laid down in D3.5 [1]

APPROACH = Co-creation, the overall mind-set of the design process

PHASE = Describing the moment in the design process, phases are the highest abstraction level of the methodology

ACTIVITY = Each phase is linked to one or several activities

METHOD = Each activity consists of one or several methods, specific systematic and theoretical frameworks to address specific purposes and contexts

TECHNIQUE = How methods are put into action to achieve established objectives.

TOOLS = Instruments and devices (material components) useful for implementing a co-creation method and to achieve specific objectives

TOOLKIT = A collection of methods, tools and techniques that that are structured and made accessible to Pas

Document Structure

This document is structured as follows:

• Section 2 and each subsequent chapter outlines each of the four phases of co-creation. Phases are the highest abstraction level of the different steps the methodology consists of. Each phase is composed of several practical activities, which need to be performed to move a project through the phases. These aspects have been identified for the effective definition and execution of the holistic co-creation process. Before the activities of each phase are being laid down, Activity 0 “Data Protection & Privacy” leads the way to ensure that all activities involving users and citizens adhere to the regulations and requirements of the GDPR.

• Section 3 collects the information to support the customisation of the methodology: Rules and contextualisation questionnaire.

• Section 4 presents the conclusions extracted and also identified future work for the CITADEL Co-Creation Methodology

• Annex 1 and annex 2 present several tables with the different methods and tools suggested in the methodology. Each method and tool are explained and categorised to allow the customisation of the methodology. Also, links where obtain more information or examples are provided.




• Annex3 and annex 4 present two summaries the work done in the empirical studies carried out during this period: ICT-based co-production of public services: Barrier and Enablers. A systematic review and Explaining citizens’ co-production through ICTs in health: A multi-level model analysis.

• Annex 5 provides additional resources and knowledge bases.




CITADEL Co-Creation methodology

1.0 Activity 0: Data Protection & Privacy

● Activity id: PRIVACYCHECK ● Activity description: When PAs are working with citizens it is of utmost importance to

always ensure the protection of their personal data and privacy. With the implementation of the GDPR in May 2018, this aspect has become more formalised and more urgent yet. PRIVACY FIRST!!

● Activity execution: ○ Before starting with co-creation project in its whole, please take each of the

points in the imec.LivingLabs checklist into careful consideration and perform a privacy check.

○ For each separate activity implementing methods that deal with users and potentially their personal data, please also take each of the points in the checklist into careful consideration and perform a privacy check for the specific activity.

○ In addition, make sure that the Data Protection Officer of the PA is being involved. (In case there is no data protection officer, make sure to install one)

○ Check the CITADEL legal vademecum [4] for templates, forms and more information

○ Make sure to cover all contractual obligations (see e.g. data processing agreement) and to thoroughly inform all involved parts subjects about what it is going to be done (informed consent)

● Predecessors: Installation of Data Protection Officer ● Successors Activities: Each of the following phases and activities ● Input documentation/artefacts:

○ Mandatory: Data Protection Officer ○ Optional: Legal Vademecum

● Output documentation/artefacts: ○ Mandatory: Privacy Check ○ Optional: General Terms & Conditions, Data processing agreement/controller

agreement, informed consent form, DPIA ● Roles: PA, DPO, legal adviser ● Supporting techniques: ● Supporting tools and material: imec.LivingLabs checklist; https://www.cnil.fr/en/open-

source-pia-software-helps-carry-out-data-protection-impact-assesment


https://www.cnil.fr/en/open-source-pia-software-helps-carry-out-data-protection-impact-assesment

https://www.cnil.fr/en/open-source-pia-software-helps-carry-out-data-protection-impact-assesment



Phase 1: Ideation and research

The objective of this phase is to understand which the real needs of citizens are and analyse how these needs could be faced by the PA in terms of Digital Public services. Here, the preparatory work for phase 2, 3 and 4 is being conducted. Activity 1.1 is the first step, helping the PA to broadly identify what citizens need or want. Activity 1.2 helps the PA to align identified needs with its own strategy. Activity 1.3 helps the PA to analyse potential users of the potential service or solution. Activity 1.4 opens up the process by launching a challenge, i.e. a call for ideas addressing the identified needs. Activity 1.5 rounds up and links to phase 2 by collecting, analysing and selecting ideas.

NOTE: The activities are highly interdependent, often overlapping and might not be executable in a strictly chronological fashion.

Figure 1. Phase 1 Workflow

Activity 1.1: Identification of Needs.

● Activity id: 11ANALYSENEEDS. ● Activity description: In this task, the PA analyses the information available about

services provided to detect problems, challenges or discover potential needs inside and outside (i.e. citizens) the administration.

● Activity execution. This activity is executed by the PAs. The identification of the needs should be based on the detailed analysis of available information. The PA should be careful and consider all the information and try to align this with the PA digital strategy and priorities. The DIGIMAT, KPI report and CITADEL User assessment analysis can be utilised to gather additional insights. An additional output of this activity should be design principles, which help and inspire to improve (digital) services, to retain the perspective of the end-user and to form a general foundation for the work to come.

● Predecessors and Successors tasks ○ Predecessor: N/A ○ Successor: Activity 1.2

● Input documentation/artefacts: ○ Mandatory: Privacy Check ○ Optional: DIGIMAT Recommendations, KPI reports and the information

collected with the User Assessment tool ● Output documentation/artefacts:

○ Mandatory: First list of the identified needs ○ Optional: N/A

● Roles: PA analysts.




● Supporting methods and techniques: ○ Focus Group with civil servants. The objective of the focus groups in this activity

is to identify the expectations, needs for the citizens and to discuss if these needs are relevant for the PA.

○ Survey. Surveys collect data usually from a large number of participants. A survey may be undertaken to study objects or animals as well as people. Surveys may take the form of a questionnaire or a face to face interview.

○ Design Principles. Formulate these principles based on your organization’s requirements but taking into account the widely accepted best practices.

● Supporting tools and material: ○ DIGIMAT: The main objective of this tool is to assess the Digital Maturity of a

Public Administration to provide improvement recommendations. The result of the assessment will be a general level of maturity complemented with a set of improvement recommendations. This tool is based on a set of questionnaires to be able to analyse the answers in an automatic way and to provide recommendations to improve the digital maturity of the PA. These recommendations could be very useful in the identification of the needs.

○ KPI report. The main objective of this tool is to facilitate the process of harvesting data from various sources that can contain key performance indicators for the utility of e-Government services, curating datasets which are from a multitude of formats and allow focusing them into semantically enhanced data. The output can be further processed by analytics and/or recommender engines to suggest public service improvement needs. Also, this tool allows to visualise this output in a graphical view to facilitate the identification of future needs.

○ CITADEL User assessment analysis. CITADEL provides a tool that first allows citizens to assess the service that they have used through a ranking and a free text comment. Second, this tool provides the PAs with relevant information extracted from the sentiment analysis of the comments provided by the citizens.

○ Checklist. A checklist is a list of items that you need to verify, check or inspect. In the development of this activity the checklist could be used to identify different source of information to be analysed to identify needs. There are several on-line tools that facilitate the management of the check list.

Activity 1.2: Prioritise the citizens´ needs

● Activity id: 12PRIORITISENEEDS. ● Activity description: In this activity, the PA should prioritise those needs that are

relevant and are also aligned with its digital vision/strategy and organisational objectives.

● Activity execution. This activity could be carried out only by the PA, but also the PA can give the opportunity to rate the needs to citizens. The final decision on the prioritization is always taken by the PA and taking into account which are its main objectives.

● Predecessors and Successors tasks ○ Predecessor: Activity 1.1 ○ Successor: Activity 1.3

● Input documentation/artefacts: ○ Mandatory: Privacy Check, List of needs identify in the activity 1.1 ○ Optional: Feedback from citizens

● Output documentation/artefacts: ○ Mandatory: List of the needs prioritised ○ Optional: N/A




● Roles: PA analysts, PA manager, Citizen (Optional). ● Supporting methods and techniques:

○ Focus Group with citizens. The objective is to hear everyone’s voice, get diverse opinions. The important aspects here are to select the appropriate group, agree on a neutral space for the interview, and select one person to ask the questions and other to take notes.

○ Survey. Surveys collect data usually from a large number of participants. A survey may be undertaken to study objects or animals as well as people. Surveys may take the form of a questionnaire or a face to face interview.

○ Needs Analysis: The analysis of the needs of the target audience gives a prioritized list of the needs out of the user perspective.

● Supporting tools and material: ○ Brainstorm: It generates ideas with your team. ○ Card Sort: A Card Sort is a quick and easy way to spark conversations about what

matters most to the people you are designing for. ○ Issues cards: The issue cards are physical instruments used as a peg to induce

and feed interactive dynamics within a team. Each card may contain an insight, a picture, a drawing or a description; everything is able to suggest new interpretations of the problem and to induce the assumption of a different point of view. The result is the identification of aspects that are critical and opportunities.

○ KPI report. This report could provide useful information to understand the needs of the citizens based for example on the indicators about the use of particular services.

○ User assessment analysis. As this analysis is based on the feedback of the citizens regarding the actual services, it could be provided relevant information on the citizens´ needs.

Activity 1.3: Analyse potential users

● Activity id: 13ANALYSEUSERS. ● Activity description: The objective of this activity is to understand (future) end-users.

Once the needs to be solved are selected, this activity should analyse the characteristics, context and behaviour of (future) end-users. It is important to consider the broad spectrum of people who will be affected by the solution of each need. When researching, this includes users who have problems using existing services or getting the right outcome for them. This will help you create a simpler, clearer, faster service that more people can use.

● Activity execution. A complete analysis looks at: ○ Socio-demographic characteristics such as gender, age, education, language and

religion. ○ Geographic characteristics such as place of residence and how that information

might impact behaviour. ○ Psychographic characteristics such as needs, hopes, concerns and aspirations. ○ Enablers and barriers that prevent or encourage audience members to adopt

the desired behavioural change. ○ Effective communication channels for reaching the audience, for example

finding associations who are able to reach the audience. ○ Try to validate the analysis of the target audience, extend and adjust it by

qualitative methods. ● Predecessors and Successors tasks

○ Predecessor: Activity 1.2 ○ Successor: Activity 1.4




● Input documentation/artefacts: ○ Mandatory: Privacy check and needs to be solved ○ Optional: N/A

● Output documentation/artefacts: ○ Mandatory: Audience analysis correctly representing different segments of

target audience, containing (demographic) characteristics and use context and behaviour of the target audience.

○ Optional: Personas ● Roles: PA analysts, PA manager. ● Supporting methods and techniques:

○ Probes: A cultural probe is a method of collecting information about people, their context and their culture. The aim of this method is to record events, behaviours and interactions in their context.

○ Define Your Audience. The objective is to consider the broad spectrum of people who will be touched by the identified need.

○ Personas: Personas are fictional archetypes of the various segments of the target audience. They are the main characters in the scenarios.

● Supporting tools and material:

Activity 1.4: Launch the challenge to address citizens´ need

● Activity id: 14LAUNCHALLENGE ● Activity description: Once the needs are prioritized and the potential users are

identified, the PA should create the space to share with people involved the best ways to solve this need. In order to obtain the right ideas for solving the identified needs, an appropriate challenge needs to be formulated before it can be published. Getting the right definition of the challenge will get off on the right foot, organize how to think about the solution, and at moments of ambiguity, help clarify where the design should be pushed. This activity also specifies the conditions for the challenge and evaluation criteria of the ideas that will be presented.

● Activity execution. This activity is about formulating and launching a ‘challenge’: i.e. a call for ideas addressing the identified need and the corresponding challenge. The PA needs to ensure that the challenge is formulated in a way that all people addressed can understand it. The PA is responsible that the challenge is aligned with the PA objectives. The following steps can help PAs in this activity:

○ Ask yourself: Does the challenge drive toward ultimate impact, allow for a variety of solutions, and does it consider the context? Make sure to take these questions into account for the challenge and refine it until it is the challenge people are excited to tackle.

○ When the challenge is almost ready, this can be run a test to check if it can be come up with five possible solutions in just a few minutes. If so, it is likely on the right track

○ Start by taking a first stab at writing the design challenge. It should be short and easy to remember, a single sentence that conveys what the challenge wants to do.

○ Properly framed design challenges drive toward ultimate impact, allow for a variety of solutions, and take into account constraints and context. Another common pitfall when scoping a design challenge is going either too narrow or too broad. A narrowly scoped challenge won’t offer enough room to explore creative solutions. And a broadly scoped challenge won’t give any idea where to start.

Innovation Support Platforms are convenient tools for collecting the feedback on the challenge. Different options exist in this regard and should be chosen based on budget




considerations and personal preferences. While these online tools can be helpful, they are far from the only tool to use. When working with less digitally connected citizens, offline tools must be considered, too.

● Predecessors and Successors tasks ○ Predecessor: Activity 1.2 and Activity 1.3 ○ Successor: Activity 1.5

● Input documentation/artefacts: ○ Mandatory: Privacy Check, List of the needs prioritised ○ Optional: Potential audience identified

● Output documentation/artefacts: ○ Mandatory: Challenge to be solved, evaluation criteria ○ Optional: Innovation Platform up and running

● Roles: PA analyst. ● Supporting methods and techniques:

○ Design challenge. ○ How-might-we. To frame your ideation session, try this methodology, which

helps to explore different points of view, questions your assumptions or even explores possibilities of your assumptions being reversed.

● Supporting tools and material: ○ Welive Platform. This platform has been developed by WeLive project [4].

WeLive platform focuses on how to pass from innovation to adoption, by democratizing the creation process and fostering public-private partnership that will jointly exploit the outcomes of the innovation process. This platform tackles the whole innovation process phases (defined in WeLive project): a) conceptualization, b) voting and selection, c) funding, d) development and e) promotion and f) exploitation

○ CITADEL Innovation Platform. This tool supports the management of the co-creation process, providing a virtual space where citizens can collaborate proposing ideas, commenting and discussing other people ideas, expressing their appreciation for ideas, and allows the platform administrator to create challenges, thing that gives the possibility to guide people ideas towards the achievement of a specific objective.

○ World Café ○ Idea Dashboard: Idea dashboards help to collect ideas and to visualise them to

make sure that everyone understands them. ○ Social Media

Activity 1.5: Creation and selection of the ideas to solve the challenges.

● Activity id: 15CREATIONEVALUATIONIDEA ● Activity description: Once the design challenge is launched at the end of Activity 1.4,

citizens provide ideas to address the launched challenge, suggest potential solutions on how they would solve the challenge. The activity includes both the ideation itself and the evaluation of produced ideas.

● Activity execution: First, citizens as identified in the audience analysis in activity 1.3 generate ideas and solutions for the challenge. The suggested ideas should be shared with the rest of the citizens that are participating in the co-creation process. All the ideas can be ranking and comment by other citizens facilitating in this way the enrichment of the ideas. The second step consists in the evaluation of those ideas. This step is done by the PA and this evaluation should be undertaken based on criteria identified in activity 1.4, which needs to be communicated with the launch of the challenge to ensure transparency.




● Predecessors and Successors tasks ○ Predecessor: activity 1.4 and 1.3 ○ Successor: Phase 2

● Input documentation/artefacts: ○ Mandatory: Privacy Check, Challenges and Evaluation criteria ○ Optional: N/A

● Output documentation/artefacts: ○ Mandatory: Idea or ideas that solve the challenge ○ Optional: N/A

● Roles: Citizens and PA ● Supporting methods and techniques:

○ Brainstorming. The objective of this techniques is to generate as much ideas as possible. The basis for this technique is that no idea or solution is a bad solution per se. All the solution can contribute to the resolution of the challenge.

○ Interviews. Interviews are a way to connect with people; an opportunity to hear them to describe their experiences in their own words.

○ 101 Method. This is a brainstorming method focuses on creating volumes of ideas.

○ Focus Group See description above ● Supporting tools and material:

○ Welive Platform: See description above ○ CITADEL Innovation Platform: See description above




Phase 2: Concept and Design

The objective of this phase is to focus and further work on the decided needs upon in phase 1. To define the trajectory, to conceptualise potential solutions, and decide what will be experimented and prototyped in phase 3. Activity 2.1 is the decisive starting point for this phase and allows the PA to perform an environmental analysis of the problem. Activity 2.2 helps the PA to plan the co-creation trajectory before it is being carried out step by step. Activity 2.3 intends to help the PA with recruiting participants. Finally, Activity 2.4 helps the PA to actually carry out co-creative events to work further on the ideas and concepts from the previous phase. Before Phase 3 can start, Activity 2.5 helps the PA to choose what to prototype.

NOTE: The activities are highly interdependent, often overlapping and might not be executable in a strictly chronological fashion.


Activity 2.1: Environmental Analysis

● Activity id: 21ENVANAL ● Activity description: After needs have been investigated, prioritised and opened to

stakeholders in Phase 1, you start concentrating on the specific challenge you will work on throughout a particular co-creation trajectory. The Environmental Analysis is the first step. It is a process of gathering and analysing information to get an overview of trends and evolutions that (might) impact your need/problem, locating the challenge in a wider context. NOTE: It is not used for scoping. The problem needs to be defined before you start. It can, of course, be necessary to fine-tune your scope based on the results, if for example an unexpected or additional outcome is found. The Design Principles of your organisation, which you have defined in previous phase, should be the guiding, high-level principles for this phase.

● Activity execution. Combine desk research and (intuitive) observation. In-depth interviews can also help at this stage. An internal brainstorm session can help to gather insights and ensure that your organisation is behind your project. Stay open, gather as much info as you can, but keep the following questions in mind:

What are the major trends regarding your challenge? Take into account political, economic, social and technological, ecological, legal and ethical dimensions. The PESTEL framework can help to structure the trends.

Which stakeholders are (not) addressed (this includes non-users)?

Which solutions to the problem do already exist?

Which companies/organisations are offering these solutions?

What are interactions between stakeholders?




Which known barriers to the problem are already described? Collect your research notes and start thinking about the next steps to follow for the trajectory, the stakeholders you need to include in the co-creation process and the possible recruiting channels. The Design Brief in the following activity is a good method to structure these insights.

● Predecessors and Successors tasks ○ Predecessor: Phase 1 ○ Successor: 2.2

● Input documentation/artefacts: ○ Mandatory: Explicit challenge to work on. Target Audience Analysis ○ Optional: Privacy Check, Design Principles

● Output documentation/artefacts: ○ Mandatory: Environmental Analysis, Next Steps ○ Optional: Stakeholders to include and possible recruiting channels for

subsequent activities. Design Challenge ● Roles: PA, no direct user involvement here ● Supporting methods and techniques:

○ Brainstorming ○ Interview

● Supporting tools and material: ○ PESTEL: A strategic framework to understand the macro-environmental factors

in your environment ○ WWWWWH ○ Information Architecture

Activity 2.2: Define your Co-creation Trajectory

● Activity id: 22DEFINECOCR ● Activity description: The Environmental Analysis in Activity 2.1 has provided you with

the context of your challenge and the decisive trends affecting it. Now, the gathered insights need to be framed into a well-defined co-creation trajectory. It is important to break down the overwhelming challenge into smaller and, workable parts, create a good overview and a plan. Keep in mind that, instead of traditional-style meetings, workshops are a common feature of the design process. This format provides the opportunity to work more collaboratively and dynamically and will help to push your project forwards.

• Activity execution. The execution of this activity depends on your challenge, the outcome of the environmental analysis and your objectives but the most important output is a clear overview of timing, with layers of engagement and progress markers (the panel matrix is a convenient tool).

• Make sure that your team has the permissions, support and planning to succeed with your co-creation project, whether it is distilling high-level policy, creating citizen-centred design briefs, or working from ground-level initiatives rolling up into organisation-wide efforts.

• Formulate a design brief that converges all collected information and compresses them in the form of key insights and guides the co-creation trajectory to be executed.

• To lay down your planning, the panel matrix is a convenient tool. It helps to define each part of the trajectory in relation to the affected stakeholders. This can still be adapted at a later stage if necessary but must be laid down in detail here. Tentative personas can already help (see successor activity 2.3).

• If applicable, start building a community around your problem based on the Community Canvas




• If applicable, define a social media strategy to communicate the progress of the co-creation trajectory.

Also check the SILK method deck for additional inspiration. ● Predecessors and Successors tasks

○ Predecessor: Phase 1, Activity 2.1 ○ Successor: Activity 2.3

● Input documentation/artefacts: ○ Mandatory: Privacy Check, Environmental Analysis ○ Optional:

● Output documentation/artefacts: ○ Mandatory: Design Brief, Panel Circle, Panel Matrix ○ Optional: Personas, Customer Journey Map

● Roles: PA, no direct user involvement here ● Supporting methods and techniques:

○ Design brief: The design brief is formalizing findings of previous activities and possible design approaches, listing every important piece of information related to your service idea.

● Supporting tools and material: ○ Design Brief Template ○ Panel Circle ○ Panel Matrix ○ Community Canvas

Activity 2.3: Recruit Participants

● Activity id: 23RECRUIT ● Activity description: Participants are core and key to any successful co-creation

trajectory. They are the ones that provide the main input that you will work with. Their recruitment can be as difficult as it is important. And it is obviously very context-dependent. Several general aspects should be mentioned.

○ First, start early. Finding and recruiting the right participants takes time and effort. Second, be realistic. Do not set your KPIs too high.

○ Second, be personal. Personal mails are always better than mass mails. If you can, even calling might help; or be present on events, set up a counter on a market place in the city…

○ Third, make sure to not infringe personal data protection regulation when communicating with and recruiting the involved people. This last point is especially important.

● Activity execution. ○ The panel circle and panel matrix are convenient, hands-on tools to help with

selecting participants. ○ Depending on the objective, participants can be targeted directly (e.g. for a

predefined design challenge) or a broader selection (e.g. to find new use cases) can be made.

○ It is important to invite a mixture of participants with different expertise fields, i.e. users as experts of their own experiences and characters (e.g. realist, innovator, creator, etc.).

○ Target both the big broad mainstream and those on either extreme of the spectrum. An idea that suits an extreme user will nearly certainly work for the majority of others. Without understanding what people on the far reaches of your solution need, you will never arrive at solutions that can work for everyone. More importantly, talking to Extreme users can spark your creativity. The




involvement of local associations or initiatives could be a practice that helps to reach the extremes of the spectrum

○ In depth interviews and Focus groups (see annex 1 to get more information) can be a highly useful tool and also help you identify who you might like to speak more with in an individual Interview.

○ When selecting end-users and stakeholders, it is important to anticipate on the group dynamics by taking local political issues or sensitivities into account.

○ Preparing the participants before can be useful to establish a shared mind-set among the participants in the group or to quickly ‘break the ice’. There are multiple methods to prepare participants for a workshop. Cultural Probes, e.g. a diary study, can be given to the participants in advance of the workshop.

○ Ideally participants will be able to participate also in later phases of the project. Engaging participants throughout the project can provide participants with a sense of ownership, possibly becoming ambassadors of the co-created solutions. Ambassadors can help to increase acceptance of solutions and ease implementation.

○ Always inform the participants of outcomes of the workshop and related aspects to show that their input is valuable and important.

○ Make full use of Social Media and, if you are using one, Innovation Support Platform for communication

● Predecessors and Successors tasks ○ Predecessor: Activity 2.1 & 2.2 ○ Successor: Activity 2.4 & 2.5

● Input documentation/artefacts: ○ Mandatory: Privacy Check, Panel Matrix ○ Optional: Personas, Panel Circle

● Output documentation/artefacts: ○ Mandatory: Follow-up on all recruiting and engagement activities with the

panel matrix ○ Optional: Community Canvas, Social Media

● Roles: PA, citizens and stakeholders ● Supporting methods and techniques: ● Supporting tools and material:

○ Panel Matrix ○ Panel Circle ○ Community Canvas ○ Focus Group ○ Extremes & Mainstreams ○ Social Media

Activity 2.4 : Conceptualise

● Activity id: 24CONCEPTCOCR ● Activity description: Conceptualization is the process of concretising ideas, of

previewing the service solution, of making its future features more tangible by translating them in visual interpretations or representations. Making the ideas visible is

important to understand and explain them in a proper manner.The hardest thing concerning envisioning during a process of service design is that there is often the need to communicate both the inner mechanism of the process and the immaterial components of the experience (which are per se difficult to represent) to several actors who are not (supposed to be) familiar with any technical language or representation technique.




● Activity execution. Several ways of executing this activity can be followed depending on the work done in previous activities, especially the design brief and panel matrix. Following steps can serve as a non-exclusive guideline:

1. A customer journey exercise helps to become more concrete, not only regarding personas but also commercialisation related considerations if necessary.

2. Personas and Scenarios need to be created next. 3. Put your Panel matrix into practice, organising and executing the events you

have planned. 4. Co-creation workshops are the most common co-creation technique and many

methods and tools exists to adapt workshops to your context. They can be a major part of this activity. Here, you should combine various methods and tools (e.g. let your participants create a customer journey map, let them write scenarios, or even come up with personas, let them play with Lego)

5. Many other methods and techniques exist that help conceptualising and concretising ideas. Make sure to report on all your activities, the positive and negative outcomes and the insights gained. These reports are obviously important for your PA and your supervisors. But it is also very important that you communicate results back to participants, who need to see the value of their input.

● Predecessors and Successors tasks ○ Predecessor: Activity 2.3 ○ Successor: Activity 2.5

● Input documentation/artefacts: ○ Mandatory: Privacy Check, Filled Panel Matrix, Design Brief ○ Optional:

● Output documentation/artefacts: ○ Mandatory: Personas, Scenarios, Co-creation workshop report, Report on all

your activities and outcomes, Customer Journey Map ○ Optional: Design Challenge

● Roles: The PA is the owner of this activity, it organises and facilitates the different activities and methods. Citizens and other stakeholders are directly involved through methods and tools.

● Supporting methods and techniques: ○ Customer Journey: A customer journey is the total path an end-user takes from

the starting point of a need to the completion of it. ○ Persona: Persona are fictional archetypes of the various segments of the target

audience. They are the main characters in the scenarios. ○ Scenarios ○ Focus Group ○ Co-creation workshop ○ Lego Serious Play

● Supporting tools and material: ○ Customer Journey Map Template ○ Persona Template ○ Co-creation workshop guidelines ○ WWWWWH ○ Innovation Support Platform ○ Social Media ○ Dotmocracy

Activity 2.5: What-to-prototype

● Activity id: 25EXPERIMENT




● Activity description: In the previous activity, you will have built one but probably several concepts and solution ideas. One of these concepts will become an actual public service or a relation technology. The challenge is to decide which one to work on further and, is worth the efforts to be taken in following phases and activities? Your idea will have lots of testable components, so be clear about what you need to learn, and which components will give you the necessary answers. In this step of the methodology, you have to decide what you want to prototype. And remember: Prototyping isn’t about being precious. Make simple, scrappy prototypes to not only save time, but to focus on testing just the critical elements.

● Activity execution: You might be trying to learn something like, “How big should this be?”. At this stage you should have a lot of questions about how your idea should work

1. Write down the key elements of your idea. Think practically about what needs to be tested and write down your primary questions for each component. Pick a few questions to answer.

2. Think through what kind of prototypes make the most sense to answer these questions. You might consider holding a Brainstorm now.

3. If you want to prototype an interaction, consider putting this on a skit with your team. If you are testing a logo, print it out and stick it on a t-shirt to solicit feedback.

● Predecessors and Successors tasks ○ Predecessor: Activity 2.4 ○ Successor: Phase 3

● Input documentation/artefacts: ○ Mandatory: Concepts/concrete ideas Activity 2.4 ○ Optional: Privacy Check

● Output documentation/artefacts: ○ Mandatory: solutions/elements/components to prototype, Specific questions

to answer with prototypes ○ Optional: prototype drafts (MVP, paper, wireframe, Lego…)

● Roles: PA ● Supporting methods and techniques:

○ Brainstorm ● Supporting tools and material:

○ dotmocracy




Phase 3: Development and implementation

This phase is the development period. The objective is to, first, prototype the concept or solution coming from previous phases, test and iterate them. Second, to validate the prototyped solution and, third, to implement its final version. The process of trial and error supports the improvement and refinement of the ideas before they are being developed and implemented. First, activity 3.1 serves to prototype the identified solution. Second, activity 3.2 serves to validate the prototyped solution. Third, activity 3.3 helps to finally implement the co-created solution, or in other words to pre-launch it.

NOTE: The activities are highly interdependent, overlapping and are not to be executed in a strictly chronological fashion.


Activity 3.1 Prototype

● Activity id: 31PROTOTYPE ● Activity description: Having built a concept in the previous phase, it is now time to put

it to the test and experiment! At this stage, there is probably a lot of questions about how the chosen solution should work. This activity is still based on the panel matrix filled out (although possibly adapted in previous activity). Before getting to the costly development of a new or improved service, some aspects of the service idea can be experienced together with customers, stakeholders or professionals in order to improve the solutions. This is usually called prototyping.

● Activity execution. Prototyping is not about being precious. Make simple, scrappy prototypes to not only save time, but to focus testing on just the critical elements. The purpose of building a prototype is to find answers, discover new insights and ideas, and to filter and measure the assumptions made. Therefore, this activity is iterative, often repeated numerous times, debunking the assumptions or validating insights and might also bring back a few steps in the co-creation process. It´s important to keep in mind that all inputs are important and it is better to realise that it is e not on the right path and change than end up with a worthless service.




Prototyping is the easy translation of concepts and ideas into tangible artefacts. They help the design team with the development and testing of ideas. Prototypes may differ from a sketch on paper to an online and interactive clickable mock-up (level of ‘fidelity’): Low-fidelity (Lo-Fi) prototypes are created early in the innovation process. Very common is paper prototyping, where users are presented with pages representing for example the interface screen. The participant indicates what he or she would do on each screen page, while the researcher swaps subsequent pages to simulate the interface response. High-fidelity prototypes (Hi-Fi) are more refined and should resemble the to-be-developed product or services and its key functionalities and design. They are characterised by a high-tech representation of the design concepts, resulting in partial to complete functionality. A prototype testing plan can help in keeping the overview. This process is about learning, not getting it right the first time. Better to test a failure and learn from it, rather than take ages making a beautiful, highly refined prototype. Last but not least, a hackathon can be great way of testing a concept, too, depending on the context, of course.

● Predecessors and Successors tasks ○ Predecessor: Activity 2.4 / 2.5 ○ Successor: Activity 3.2

● Input documentation/artefacts: ○ Mandatory: Privacy Check, Panel Matrix, ideas to prototype ○ Optional: Prototype Testing Plan, IDEO Prototyping Course material

● Output documentation/artefacts: ○ Mandatory: well-defined, tested and validated prototype ○ Optional: Pre-launch prototype

● Roles: PA analysts, citizens, stakeholders or professionals are directly involved in the testing of prototypes

● Supporting methods and techniques: ○ Prototyping ○ Rapid Prototyping ○ Lo-fidelity prototype ○ MVP ○ Lego Serious Play ○ Wireframes ○ Hackathon

● Supporting tools and material: ○ IDEO Prototyping Course ○ Prototype Testing Plan

Activity 3.2: Develop

Activity id: 32DEVELOP

● Activity description: While the prototyping activity is about answering questions, testing and experimenting with the ideas, concepts and (potential) solutions or their components, this activity is the next step: validating the prototyped concept.

● Activity execution: The validation at this point of the project should be conducted in different ways. On the one hand, by professionals and experts, and, on the other hand by actual users in the form of usability tests and use cases. A test on the usability of a service may encompass anything from a quick hallway test to an organized lab set-up. In addition, and depending on the solution you are working on, a hackathon can provide you with additional validation by the participants of the hackathon (developers, entrepreneurs, domain experts…), which can certainly be regarded as experts, too.




The SCRUM methodology or framework [5] for development can be a very beneficial way to organise this activity. This is the case at this stage of the process because the end goal of SCRUM is the launch of the developed, which is something that is not always addressed by other methods. The outcome of this activity will thus be a final solution, well-working, validated and ready to be implemented in the next activity. At this stage, it also is recommended to consider the alignment of technological advances, which your solution should provide, with values, needs and expectations of society, i.e. your target group. The CTA toolbox is a good starting point for assessing societal and ethical aspects.

● Predecessors and Successors tasks ○ Predecessor: Activity 3.1 / 2.5 ○ Successor: Activity 3.3, phase 4

● Input documentation/artefacts: ○ Mandatory: tested prototypes ○ Optional: N/A

● Output documentation/artefacts: ○ Mandatory: Expert Review, Usability Tests ○ Optional: Solutions developed during a hackathon, Scrum Framework

● Roles: PA analysts, experts, citizens, developers, entrepreneurs ● Supporting techniques:

○ Top Task Analysis ○ Expert (peer) Review ○ Hackathon ○ Usability Test ○ Use Case

● Supporting tools and material: ○ Scrum and Sprint framework ○ CTA toolbox

Activity 3.3: Implement

● Activity id: 32IMPLEMENT ● Activity description: Once a well-defined, tested and validated solution exists,

implementation can start. Note that this activity overlaps with previous ones and that some of the following points are best executed in parallel with activity 3.2. and even 3.1. The process of “pre-launch” has to do with analysing, validating, distilling and orchestrating the upcoming launch of the prototype. While previous phase’s validation was mainly revolving around usability, here the steps towards commercialisation are most important.

● Activity execution: Make sure that your solution is not only technically sound but also has a sound value proposition, a stable business model and if it is required a realistic pricing model. Osterwalder’s canvases are the most common tools here.

○ The business model canvas is a widely used template to ensure you have covered the most vital aspects of creating a successful business. It is important to pay special attention to the value proposition, to make sure you are creating real value for your end-users/citizens. The citizens relationships and customer segments fields must be completed with true and validated information Without assumptions.

○ The Value Proposition Canvas complements the Business Model Canvas. It allows you to focus on the value proposition you intend to offer to your users/citizens. Paying special attention to the wants and needs of your users is most significant.




○ The lean validation board is another helpful tool that can help to stay focused on acting while implementing your project. It functions most of all to turn assumptions into facts.

○ Also check the SILK method deck and the CTA toolbox for additional inspiration. Once you completed some of these steps, your prototype is ready to be launched. Start thinking about a (social) media strategy to accompany your launch.

● Predecessors and Successors tasks ○ Predecessors: Activity 3.2 ○ Successors: Phase 4

● Input documentation/artefacts: ○ Mandatory: Well-defined, tested and validated solution ○ Optional: N/A

● Output documentation/artefacts: ○ Mandatory: Business Model Canvas, Value Proposition Canvas ○ Optional: Lean Validation Board, Innovation value chain, Social Media strategy

● Roles: PA analysts, PA manager, target groups indirectly involved (input from previous activities)

● Supporting techniques: ○ (intention) Survey

● Supporting tools and material: ○ CTA toolbox ○ Business Model Canvas ○ Value Proposition Canvas ○ Lean Validation Board ○ Social Media

Phase 4: Production and maintenance

The final quarter is where you finalise your project (i.e. the service or product), launch it and evaluate it. At the end of the process, you should evaluate all information and consider whether you should start all over again or to start from any point in the co-creation process. It is important to keep in mind that co-creation is an iterative process that ideally never truly ends. First, activity 4.1. helps the PA to launch the service based on all the insights gathered previously. Second, activity 4.2 serves to evaluate the service while it is running. Third, activity 4.3 serves as a reminder that co-creation is an iterative and a circular process.





Activity 4.1: Launch

● Activity id: 41LAUNCH ● Activity description: Finally, the solution you have been working on is ready to be

launched. Based on all previous steps and all insights you have gathered and implemented, you can now deliver it to the stakeholders. This implementing process includes explaining its importance and impact for the context.

● Activity execution: While the launch of your solution obviously causes much work internally in your organisation (technical and organisational), the launch as described here in this methodology is mainly oriented towards the stakeholders.

○ A sound (social) media strategy should accompany the launch. It is relevant to make sure to spend sufficient time on the marketing. If you developed a community canvas in phase 1 or phase 2, link your strategy to it.

○ A test market, a geographical area selected for a small-scale introduction of your solution, can be a good starting point. It serves as a field laboratory that simulates some or all factors associated with a full-scale launch without the users being aware. Multiple test market locations allow evaluation of different strategies etc.

○ A valuable addition to this activity can be crowdfunding. Crowdfunding is not only a manner of getting funding in an open platform manner, mainly from users. But it is also a great way to assess, without too much effort, whether people are willing to pay for your product. And, even better, crowdsourcing can also be set up as a form of competition (see hackathon). This should be linked to a sustainable community of users (see community canvas).

● Predecessors and Successors tasks ○ Predecessors: Activity 3.3 ○ Successors: Activity 4.2

● Input documentation/artefacts: ○ Mandatory: Business Model / Value Proposition Canvas ○ Optional:

● Output documentation/artefacts: ○ Mandatory: (social) media strategy ○ Optional: Test Market Analysis, crowdsourcing campaign, willingness-to-pay

analysis




● Roles: PA, users and customers ● Supporting techniques:

○ Social media strategy ○ Test market ○ Crowdsourcing ○ Interview

● Supporting tools and material: ○ Community canvas

Activity 4.2 Evaluate

● Activity id: 42EVALUATE ● Activity description: Once your solution has been launched, it is time to evaluate it. This

is done to ensure future sustainability of the product/service. Many of the existing methodologies and toolkits focus on the earlier phases of the process and leave this part out, even though evaluation is equally important. Evaluation is a medium- to long-term process that might lead to improvements or even cancellation of your product/service.

● Activity execution: Evaluation, and iteration based on the evaluation, are key to the co-creation process. There are several ways of doing so. Try the Growth Hacking method, analyse web statistics (e.g. with Google Analytics), analyse conversion (i.e. the rate of users who succeeded with a task) and conduct a survey. This can all help you to gain insights into the sustainability of your solutions, to evaluate the components that are satisfactory and those that need to be iterated.

● Predecessors and Successors tasks ○ Predecessors: Activity 3.3 ○ Successors: Activity 4.1

● Input documentation/artefacts: ○ Mandatory: Business Model / Value Proposition Canvas ○ Optional:

● Output documentation/artefacts: ○ Mandatory: web statistics analysis, customer satisfaction ○ Optional: Growth Hacking canvas, conversion insights, other surveys

● Roles: PA, users and customers ● Supporting techniques:

○ Top Task Analysis ○ Social media strategy ○ Growth Hacking ○ Web statistics Analysis ○ Customer Satisfaction ○ Conversion insights ○ Survey

● Supporting tools and material: ○ Growth Hacking canvas ○ Google Analytics ○ CITADEL User Assessment ○ KPI Report

Activity 4.3 Iterate

● Activity id: 43ITERATE ● Activity description: Co-creation is iterative, a circular process. Once you are done, start

again ● Predecessors and Successors tasks




○ Predecessors: Phase 1 to phase 3 ○ Successors: Phase 1




Rules for customisation

Findings from the Systematic Literature Review

The systematic review showed a range of structural and cultural factors that can act as a barrier to or enabler of ICT-based public services co-creation by government and citizens. As regards to the government, key barriers were associated with a lack of financial capacity and autonomy, inadequate technical skills of staff, complex regulation and lags in adequate regulatory reform, as well as professional resistance to using ICTs to co-produce. Enablers included governments’ selection of lower cost ICT solutions, reducing the cost of financial failure, and situations where governments enjoyed greater financial autonomy. Other enablers included staff training, government support to adapt legislation to ICT deployment and efforts to overcome staff reluctance to use ICTs.

As regards to citizens, the systematic review detects significant structural and cultural barriers to and enablers of their take-up of ICT-based co-creation. A lack of technical skills, or a fear of being humiliated when using ICTs, were major barriers to citizens ICT use. The relationship between demographic, such as age, gender and ICT use are not straightforward, but, generally, a pattern is detected whereby older people and females use ICT-based co-creation less than their younger and male counterparts. Although it can therefore be concluded that a second digital divide affects the ICT-based co-creation processes, these findings should be interpreted with caution given the low number of studies reviewed in the systematic review. A number of cultural factors also influenced ICT use: a lack of trust in government, traditional form of social interaction and specific ethnic, social and language differences dampened citizen ICT co-creation. As regards enablers, involving citizens in the design of ICT systems and running tailored technical training of citizens can enhance ICT use. It is observed a positive relationship between greater citizen engagement in public service delivery and greater citizen trust in the government. Finally, it is also found that citizens are more likely to engage in ICT based co-creation when this process is associated with higher citizen collaboration.

Too often, policy-makers see technology as a solution in itself. The results of this systematic review show which factors are associated with enabling ICT use in co-creation, but also tend to pose barriers to these activities. Clearly, the potential of ICTs in the co-creation of public services is immense. A better understanding of the factors that make co-creation work, and which tend to slow it down, or prevent it, should be considered by policy-makers when deciding about new policy initiatives to pubic services’ co-creation.

The complete information used to derive the rules presented in the following section is based on the D3,2 and D3.4 and the work explained in Annex 3: ICT-based co-production of public services: Barrier and Enablers. A systematic review.

Rules to customise the methodology

As mentioned before, the following list of rules for the customisation of the CITADEL co-creation methodology is derived from the findings of the systematic literature review carried out in the task 3.1 and task 3.2 and collected in D3.2 [7] and D3.4 [8] . Furthermore, the requirements of the use cases and aspects that are deemed as generally important underlie this list.

These rules have been divided according to several categories. Whether these categories remain necessary for the next step in the process of rule definition remains to be decided.

- Practitioner (“U”)




Practitioners are the main target user group of the Cc-c (CITADEL Co-creation), mainly PAs or Civil servants. From hereon they are referred to as user (of the support tool). Experience, available time, budget or internal support are decisive factors here.

- Participants (“P”) Participants are citizens and other stakeholders who are addressed through the co-creation methods and tools. Factors such as role, technological capacity, age, socio-cultural background are decisive.

- Project (“Pr”) Project factors relate to the co-creation project itself, such as the phase it is in, its objectives, KPIs, timeframe among others.

- Organisation (O) Organisation refers to the public organisation in which the practitioner is trying to realise a project. The factors thus relate to the characteristics of the PA, such as government level (national, regional, local), language, used to co-creation, backing of policy documents.

- Technical Rules (“T”) Technical rules concern the technology of the tool itself, not the technical knowledge of the practitioner or the participants.

In order to allow the customisation of the methodology, it has been decided that the tools and the supported methods are categorisation based on 6 categories defined based on the rules detailed in the following table. Three levels are defined for each category (Red, orange and green). Each tool is classified in any of the following categories:

• Time: o Green: flexible, low time-involvement (not much time - one/two days; flexible

in outline) o Orange: semi flexible, medium time-involvement (one week) o Red: fixed procedure, high time involvement (so it is needed to follow every

step and it takes you more than one week; on-going action over long time period)

• Budget o Green: flexible, no or very low cost; o Orange: semi-flexible, the cost incurred should be moderated o Red: Cost unknown but pay attention!

• Number of participants to recruit: o Green: <10 o Orange: Between 10 and 20 o Red: large group of participants: more than 20

• Technical complexity: o Green: easy to understand and use, no technical tool needed o Orange: small instructions required to be used o Red: It implies to have technical training

• Experience and knowledge: o Method or tool can be used without background knowledge (green) o Method or tool needs a basic introduction (orange) o Method or tool needs a thorough training and background knowledge (red)

• Project Complexity • Green: Participants belong to one user group and have the same background

characteristics • Orange: Participants belong to one user group but have different backgrounds

characteristics




• Red: Participants belong to different user groups and have different backgrounds (from different sectors, many players (PA, citizens, private sector.), socio-demographics, …)

Table 2. Rules for customisation

# Rule Category Classification Criteria for the tool and methods

Question of the questionnaire (see section 3.3)

1. Depending on the experience of the user, the tool customises the methodology so that suggested methods and techniques correspond with the user’s level of experience.

U Experience and knowledge

6 & 8

2. Depending on the time the user has, the tool customises the methodology so that suggested methods and techniques do not consume too much time.

U Time 2

3. Depending on the budget the user has at her disposal, the tool customises the methodology so that methods and tools are chosen accordingly.

U Budget 3

4. Depending on the project, context and organisation, the tool suggests in which phase a user should start with its co-creation project.

Pr Status of the co-creation

1

5. If the participants of the co-creation activity have no or little technical knowledge, the tool customises the methodology so that suggested methods and techniques do not depend on technological solutions.

P Technical Complexity

6 & 7

6. Depending on the time frame of the project, the tool suggests short-term effective methods and tools or those that are more effective and sustainable when implemented over a longer time.

Pr Time 2

7. Depending on the context of the project, the tool customises the methodology so that both digital and non-digital communication and interaction channels are utilised.

Pr Technical complexity, Project complexity

5 & 6 & 7 &8

8. Depending on the background of the participants, the tool customises the methodology so that methods and tools are combined in a fashion that allows to gather complimenting insights from all target groups

P Project Complexity Participants

4 & 5 &8 &9




# Rule Category Classification Criteria for the tool and methods

Question of the questionnaire (see section 3.3)

9. Depending on the context of the project and the technological capacity of participants, the tool customises the methodology so that participation is as simple as possible.

P Project complexity Technical complexity

5 &6 & 8

10. Depending on the complexity of the stakeholder environment of a project, the tool customises the methodology so that communication is fostered not only between PA and stakeholders, but also between stakeholders.

Pr Project Complexity

5 & 8

11. Depending on the age of the target groups, the tool customises the methodology so that utilised methods and tools correspond with their characteristics and knowledge.

P Participants/ Technical Complexity

4&5

12. Depending on the characteristics of the participants, the tool customises the methodology so that digital methods are combined with non-digital ones to ensure that everyone can provide insights…

P Participants

4 & 5 & 6 & 8

13. Depending on the knowledge of participants, the tool customises the methodology so that an adequate mix of methods and tools is suggested so that all stakeholder groups are empowered to provide (complimenting!) input.

P Experience and knowledge

8

14. Depending on the language spoken in the project’s environment, the tool supports translation of information…

O 9

15. Depending on the cultural context of the co-creation project, the tool customises the methodology so that different culturally embedded knowledge practices are being taken into account.

O Participants Technical complexity Experience and knowledge

5 & 6 & 8

Questionnaire to customise the methodology

This questionnaire is used to collect some information that allows to contextualise the co-creation project.

This collected information together with the classification mechanism explained in the previous section will be used by the co-creation supported tool to customise the CITADEL co-creation methodology.




This questionnaire should be answered taking into account a specific co-creation trajectory project.

Following the questions that are listed and based on the responses, the most appropriate methods and tools will be selected.

1. Which statement describes properly the situation of the co-creation project in the best way:

a. There is no clear idea yet about what the main needs from the PA are (Phase 1) b. There is a clear idea on what the PA needs, but it still needs to be designed the

way to do so should be decided upon. (Phase 2) c. There is already a service available, but we want to improve it (Phase 3 and

Phase 4). 2. Is there any time constraint in the co-creation process?

a. Yes, it is required to do it in short period time. (Category Time= green) b. No, the deadlines for completion are quite flexible (Category Time= green,

orange and red) 3. Is there a specific budget for this project?

a. No budget available (Category Budget= Green) b. Yes, there is a budget but very restricted and no means to increase it (Category

Budget = Green) c. No, the budget is flexible enough (Category Budget= green, orange and red)

4. , Do you need/want to involve a large group of participants in this co-creation trajectory project?

a. Yes, the approach is to involve as much people as possible (more than 20) (Category Participants = Red)

b. Yes, but the group should have a medium size (between 10-20) (Category Participants = Orange)

c. No, just a small and compact group (<10) (Category Participants = green) 5. The participants to be involved in this co-creation trajectory process are:

a. From the same sector/user group and have common background characteristics (age and experience) (Category Project Complexity = green)

b. From the same user group/sector but have different background characteristics (Category Project Complexity = orange)

c. From Different sectors/user groups and different background characteristics (Category Project Complexity = red)

6. Participants to be involved must have: a. High experience on working with ICT Tools (Category Technical = red) b. Some experience on working with ICT Tools (Category Technical = orange) c. No experience on working with ICT Tools (Category Technical = green)

7. Based on the participants, what kind of tools do you prefer? a. Only digital tools b. Non-digital tools c. Both digital and non-digital

8. Participants are used to express their opinion in a co-creative context’: a. They do not have any experience and knowledge with co-creation processes

(Category Technical = green) b. They have a certain experience and knowledge with co-creation processes

(category orange) c. The have a long experience and knowledge on co-creation processes (Category

Technical = red) 9. Is there any concern regarding the language to be used?

a. Yes




b. No




Conclusion

The final CITADEL Co-Creation methodology, as laid down in detail in this document, is structured into 4 phases: Ideation & Research, Concept & Design, Development & Implementation, Production & Maintenance. Each phase is subdivided into practical co-creation activities, which need to be performed to move a project through the phases, linked to specific methods, techniques and tools classified in the following categories: time, budget, number of participants, technical complexity, and project complexity.

The methodology is highly structured and provides a process-based and holistic guide to co-creation. This is innovative and highly relevant to the project as well as other public administrations intending to provide better and more citizen-centric public services. The logic of the methodology and the decisive interlinking of often separate activities, methods and tools is based on common practice in the field and additional expertise by the project partners. The step by step guide is practical and hands-on enough to be used in practice.

The objective of phase 1 and its activities is to understand needs of citizens, and to crowdsource how these needs could be faced by the PA in terms of Digital Public services. The outcomes present the PA with clear ideas on what to further work on. The objective of phase 2 and its activities is to work on a concrete project, plan it and execute diverse co-creative happenings. The outcomes are concepts and ideas that are ready to be prototyped. Phase 3 is the development period where the concept of previous phases is prototyped, tested and iterated. The outcome is a solution that is ready to be launched, including business considerations. In phase 4, the project is finally being finalised. The outcome is an up-and-running service or solution.

One added value of this methodology is the set of methods and tools collected. These methods and tools are classified in six different categories: Budget, time, participants, technical complexity, participant’s experience and knowledge and project complexity. This classification will allow selecting the most appropriate set of methods and tools depending on the context of each co-creation project.

The methodology as presented in this document will be supported by an IT tool developed in WP4.




References

[1] CITADEL, “D3.5 Initial requirements for co-creation,” 30 September 2017. [Online]. Available: https://www.citadel-h2020.eu/sites/citadel.drupal.pulsartecnalia.com/files/documents/D3.5_Initial%20Requirements%20for%20Co-creation_v1.0.pdf. [Accessed September 2018].

[2] CITADEL, “D3.7 “Initial CITADEL Methodology for co-creating a public service”,” February 2019. [Online]. Available: https://citadel-h2020.eu/sites/citadel.drupal.pulsartecnalia.com/files/documents/D3.7_Initial%20CITADEL%20Methodology%20for%20co-creating%20a%20public%20service_v1.0_20180930%20.pdf. [Accessed February 2019].

[3] CITADEL, “D3.6 Final requirements for Co-creation,” 2018.

[4] CITADEL Consortium, “D2.4 Legal "vademecum" for PAs,” 2019.

[5] WeLive, “WeLive Project,” 2017. [Online]. Available: https://www.welive.eu/. [Accessed May 2018].

[6] SCRUM.org, "The home of SCRUM," [Online]. Available: https://www.scrum.org/. [Accessed February 2019].

[7] CITADEL, “D3.2 Final scientific study of co-creation and citizens´ participation,” September 2018. [Online]. Available: https://citadel-h2020.eu/sites/citadel.drupal.pulsartecnalia.com/files/documents/D3.2%20Final%20scientific%20study%20of%20co-creation%20and%20citizens%E2%80%99%20participation_v1.0_20180930.pdf. [Accessed March 2019].

[8] CITADEL project, “D3.4 Final report on how to involve most effectively the private sector in public service co-creation,” September 2018. [Online]. Available: https://citadel-h2020.eu/sites/citadel.drupal.pulsartecnalia.com/files/documents/D3.4%20Final%20report%20on%20how%20to%20involve%20most%20effectively%20the%20private%20sector%20in%20public%20service%20cocreation_v1.0_20180930.pdf. [Accessed March 2019].

[9] C. W. K. M. M. G. G. a. E. N. N. Mack, “Qualitative research methods: a data collector’s field guide,” FLI USAID, North Carolina, 2005..

[10] M. Sandelowski, “Focus on qualitative methods: sample size in qualitative research revisited,” Res. Nurs. Health, no. 18, p. 179–183 , 1995.

[11] P. B. a. E. Plumley, “Co-Creation: Designing With the User,” UX Booth, 2014.




Annex 1: Supporting Methods

1.1. Design Principles (p. 38) 1.2. Focus group (p.39) 1.3. Interview (p. 41) 1.4. Survey (p. 42) 1.5. Needs Analysis (p.44) 1.6. Card Sorting (p. 45) 1.7. Probes (p.47) 1.8. Define Your Audience (p. 48) 1.9. Extremes and Mainstreams (p. 49) 1.10. Brainstorm (p. 50) 1.11. Personas (p. 52) 1.12. Scenarios (p. 53) 1.13. Customer Journey (p. 54) 1.14. Co-Creation Workshop (p. 55) 1.15. Rapid Prototyping (p.57) 1.16. Lo-fidelity Prototype (p. 57) 1.17. LEGO® SERIOUS PLAY® (p. 61) 1.18. Storytelling (p. 62)

1.19. MVP (Minimum Viable Product) (p. 63) 1.20. SWOT Analysis (p. 63) 1.21. How Might We (p. 64) 1.22. Design Challenge (p. 66) 1.23. WWWWWH (p. 67) 1.24. World Café (p. 68) 1.25. Reverse Brainstorm (p. 69) 1.26. Usability Test (p. 70) 1.27. Expert (peer) Review (p.71) 1.28. Top Task Analysis (p. 72) 1.29. Hackathon (p. 73) 1.30. Crowdsourcing (p. 74) 1.31. Use Case (p. 74) 1.32. Customer Satisfaction (p. 76) 1.33. Scrum and Sprint (p. 77) 1.34. Growth Hacking (p. 78)

Title 1. Design Principles

Method ID DESIGNP

Interlinking Activity 1.1

Activity 2.1

Description Design principles are not specific to one co-creation project but should be formulated generally to inform all service design.

Why Help and inspire to improve (digital) services, to retain the perspective of the end-user.

Use them as common starting point for all related activities to ensure consistent implementation

Tips • Start with needs: user needs, not government needs.

• Do less.

• Design with data.

• Do the hard work to make it simple.

• Iterate. Then iterate again.

• This is for everyone.

• Understand context.

• Build digital services, not websites.

• Be consistent, not uniform.

• Make things open: it makes things better.

Execution Formulate these principles based on your organization’s requirements.

Most important, start with needs.




Title 1. Design Principles

The websites of the UK and Australian governments give great instructions and helpful examples (see link in the Resources section).

Supporting tools NA

Resources Design Principles by the UK Government Digital Service

Design Principles by the Australian Government Digital Transformation Agency

Categorization Time Budget Participants Technical Complexity

Experience and knowledge

Project Complexity

ALL

Title 2. Focus group

Method ID FOCUS GROUP


Activity 1.3

Activity 2.4

Interview

User Panel

SWOT

Description A focus group is a small, but demographically diverse group of people and whose reactions are studied especially in market research or political analysis in guided or open discussions about a new product or something else to determine the reactions that can be expected from a larger population. They are usually composed of 6 to 12 participants, led by a moderator.

It is a form of qualitative research consisting of interviews in which a group of people are asked about their perceptions, opinions, beliefs, and attitudes towards a product, service, concept, advertisement, idea, or packaging. Questions are asked in an interactive group setting where participants are free to talk with other group members.

Why Focus groups do not offer the depth of an individual Interview but can give a look at how a larger set of the people you are designing for operates

Identify expectations, needs and desires of users

Gain several different perspectives about a (design) problem

Good to identify participants whom you might want to invite to other, more in-depth methods


https://www.gov.uk/design-principles

https://www.dta.gov.au/standard/design-principles/




Tips • Seek to hear everyone’s voice, get diverse opinions, and are strategic about group makeup.

• Select the appropriate group, agree on a neutral space for the interview, and select one person to ask the questions and other to take notes

• Focus group study results may not be generalizable.

• Focus group participants can influence each other.

• Another disadvantage of this method is that it removes the subjects from their context.

Execution Preparation:

• Prepare thoroughly. Think about the required outcome (major objective) and make use of adequate methods Prepare a screening questionnaire.

• Select facility. Neutral ground is best, for example a shared community space that people of all ages, races, and genders will have access to

• Recruit participants. Identify the sort of group you need. If you are trying to learn something specific, organize the group so that you have the best chance at hearing it

• Invite participants to your session well in advance and get firm commitments to attend. Remind participants the date of the event.

• (Before you do the actual focus group, it is wise to practice with your team to get a sense of how to frame the questions for better response)

Facilitation

• Participants should sit around a large table.

• Describe rules.

• Be certain to have one person asking the questions and other team members taking notes and capturing what the group is saying.

• First question should encourage talking and participation.

• The moderator manages responses and asks important questions:

Make sure to prompt participants to be specific in clarifying their preferences and motivations

Come prepared with a strategy to engage the quieter members of the group. This can mean asking them questions directly or finding ways to make the more vocal members of the group recede for the moment.

• Moderator collects forms and debriefs focus group.

Analysis

• Analyse results while still fresh.

• Summarize key points.

• Run additional focus groups or one on one interviews to deepen analysis.

Supporting tools https://www.focusgroupit.com/

User Panel

Resources NA


https://www.focusgroupit.com/






Project Complexity

ALL

Title 3. Interview

Method ID INTERVIEW


Activity 4.2

Probes

Five why’s

Focus Group

User Panel

Test Market

Description Interviews are a way to connect with people; an opportunity to hear them describe their experiences in their own words. Speaking to people about their everyday lives can help you define and describe problems they face, understand the environment they’re part of, and even start to picture the ways in which you can reach them.

The data comes from what people tell the researcher, focus on self-reports: what people say they do, what they say they believe and what opinions they say they have. Interviews and the data gathered thereby must thus be considered in the light of subjectivity and should be combined with other methods and tools.

Why To grasp meaning and gain a deep understanding of the social/living environment

To answer questions that start with 'how' and 'why' (not: 'who', 'what' or 'how many')

To collect rich, unstructured data from a limited number of participants

Tips • Conduct interviews early in your co-creation trajectory and consider another round to validate and re-assess outcomes in the Implementation and Production phases

• When possible, conduct your interviews in the interviewee’s space.

• Your questionnaire is a guideline, not a strict tool. However, stick to the subject! Always keep your research question in mind.

• In multi-method designs: Use interviews to prepare (before) or enrich (after) quantitative methods

Execution No more than three research team members should attend any single interview so as to not overwhelm the participant or crowd the location.




Title 3. Interview

Each team member should have a clear role (i.e. interviewer, note-taker, and photographer).

Prepare a semi-structured question protocol or topic list. Start by asking broad questions about the person’s life, values, and habits, before asking more specific questions that relate directly to your challenge.

Listen closely to your participants, observe non-verbal cues and elaborate on their answers

Make sure to write down exactly what the person says, not what you think they might mean. This process is all about hearing exactly what people are saying. If you are relying on a translator, make sure he or she understands that you want direct quotes, not the gist of what the interviewee says.

What the person says is only one data point. Be sure to observe your interviewee’s body language and the context in which you are talking.

Supporting tools Five Why’s

User Panel

Resources http://www.designkit.org/methods/66



Project Complexity

ALL

Title 4. Survey

Method ID SURVEY


Activity 1.2

Activity 4.2

User Panel

Customer Satisfaction

Description Surveys are a method of collecting information. Surveys collect data usually from many participants. Surveys may take the form of a (online or offline) questionnaire or a face to face interview.

A valuable specification of the method is “intention Surveys” which help understand whether users would use/buy a product or service and thus leads to extrapolations regarding market potential or the distribution of innovators, early adopters, majority and laggards.


http://www.designkit.org/methods/66



Title 4. Survey

Why Easy to analyse

Large sample at relatively low cost

Simple to manage

Familiar format

Quick to complete

Can be used for sensitive topics

Respondents have flexibility in time to complete

Tips • Have a standard procedure

• Ask one information at a time

• Be as simple as possible

• Adjust the style of the questions to the target audience

• Avoid complex questions

• Avoid leading questions

• Avoid closed questions with an incomplete set of possible answers

• Avoid long questionnaires (>35 questions)

• Make a good of mix of open and closed answers

Disadvantage of surveys: although a survey is sent to a representative group of citizens, the non-response group includes more socio-demographical vulnerable groups. Therefore, it may be hard to gather results that are representative to the entire population, which may lead to incorrect conclusions.

Execution Define topics for research

Define the participants

Prepare the questions

Use closed questions with multiple predefined choices or open questions to allow respondents to respond in their own words

Two common closed formats are:

• Likert 5-point format: strongly agree, agree undecided, disagree, strongly disagree.

• 4 point forced choice format: Strongly agree, agree, disagree, strongly disagree.

Present the questionnaire

Refine the questionnaire

Questions should flow logically

Intention Survey:

• Likert 5-point format: “1 – I would by it immediately”, to “5 – I would certainly never buy this”

• Question: To which degree would you buy/use the solution if it was available?

• Confront respondent with the solution (can be a description, a picture of an idea or concept as well as a tangible product.)

Supporting tools User Panel




Title 4. Survey

Community Canvas

Resources https://freeonlinesurveys.com/#!/

https://qualaroo.com/home

https://usabilla.com/

https://www.hotjar.com/

https://surveymonkey.com/

https://www.google.com/forms/about/

https://www.typeform.com/



Project Complexity

Title 5. Needs Analysis

Method ID NEEDANAL


Activity 2.1

Description The analysis of the needs of the target audience gives a prioritized list of the needs out of the user perspective.

Why Services designed around users and their needs:

• are more likely to be used

• help more people get the right outcome for them - and so achieve their policy intent

• cost less to operate by reducing time and money spent on resolving problems

Tips • Understand the needs of all kinds of users, not just ‘typical’ users.

• Consider the needs of people who provide the service or support other users (for example, caseworkers, call centre agents, inspectors, lawyers and charity workers).

• Focus on users who have problems using existing services or getting the right outcome for them. This will help create a simpler, clearer, faster service that more people can use.

Execution You can learn about users and their needs by:

• reviewing existing evidence (for example, analytics, search logs, call centre data, previous research reports etc)

• interviewing and observing actual or likely users


https://freeonlinesurveys.com/#!/









Title 5. Needs Analysis

• talking to people inside and outside your organisation who work with actual or likely users

• Treat any opinions or suggestions that don’t come from users as assumptions that have to be proven by doing research.

In phase 1 of your project, you should find out:

• who your likely users are and what they’re trying to do

• how they currently do it (for example, what services or channels they use)

• the problems or frustrations they experience

• what users need from your service to achieve their goal In phase 2 of your project, you should:

• improve your understanding of your users and their needs

• test design ideas and new features with likely users

• assess users’ experience of your service, to make sure it meets their needs Once you have a good understanding of your users’ needs, write them down. User needs are usually written in the format:

• I need/want/expect to… [what does the user want to do?]

• So that… [why does the user want to do this?]

• (If it is helpful, you can add) As a… [which type of user has this need?]

• When… [what triggers the user’s need?]

• Because… [is the user constrained by any circumstances?]

• Prioritize needs according to frequency and importance for the end-user Prioritize needs per target audience segment

Supporting tools NA

Resources https://www.gov.uk/service-manual/user-research/start-by-learning-user-needs



Project Complexity

All

Title 6. Card Sorting

Method ID CARDSORT


Interview

Workshop

Focus Group

Description A Card Sort is a quick and easy way to spark conversation about what matters most to the people you are designing for. By sorting a series of cards, each labeled




Title 6. Card Sorting

with a piece of content, into groups that make sense to users, people’s subjectivity can be studied.

It requires the active participation of 2 to 10 participants.

Why Card sorting is a simple, reliable and inexpensive technique to build taxonomies or category trees

To find out how people think your content should be organized and make informed information architecture decisions/workflows or menu structures

Especially useful when audiences differ significantly in how they view similarities among items and appropriate groupings

Tips • Card sorting works well for small, homogeneous sites or site parts

• In an open card sorting session, users can label groups of topics themselves. In a closed session, users must use the labels that are provided

• Card sorting can be done manually or remotely with software

• When tailoring your deck of cards to your precise research objectives, be sure that you are mixing concrete ideas with more abstract ones. You can learn a lot about how the person you are designing for understands the world by making this exercise more than just a simple ranking.

Execution Make your deck of cards for the card sort:

• Collect all 'content' items that need to be sorted and print them on small cards

• Use either a word or a picture on each card, but whatever you select, make sure that it is easy to understand.

• Pictures are a better choice if the person doing the Card Sort speaks another language or cannot read.

Ask users to group and label items that belong together or according to what’s most important, individually or in small groups.

There are a couple variations on this method:

• Instead of asking the person you are designing for to rank the cards in order of preference, ask her to arrange them as she sees fit.

• Another tweak is to pose different scenarios. Ask the person you are designing for how she would sort the cards if she had more money, if she were old, if she lived in a big city.

Supporting tools Pre-made cards with a relevant word or image on them


https://www.usability.gov/how-to-and-tools/methods/card-sorting.html



Project Complexity



https://www.usability.gov/how-to-and-tools/methods/card-sorting.html



Title 7. Probes

Method ID PROBES


Affinity Diagram

Interview

Mobile Contribution Apps

Description A cultural probe is a method of collecting information about people, their context and their culture. The aim of this method is to record events, behaviours and interactions in their context. This method involves the participants to record and collect the data themselves.

Why This is a useful method when the participants that are being studied are hard to reach for example if they are travelling.

It is a useful technique if the activities being studied take place over an extended period or at irregular intervals.

The information collected can be used to build personas.

Tips • Select the participants carefully

• Give them support during the study

• Combine with one in-depth interview afterwards to make sure that you understand the collected information correctly

Execution Define the objective of your study.

Recruit your participants.

Brief the participants

Supply participants with a kit. The items in the kit are selected to collect the type of information you want to gather and can include items such as notebooks, diary, camera, voice recorder or post cards.

You can use an affinity diagram to analyse the data collected

Supporting tools Mobile contribution (app)

Resources Diary Notebooks Pens Post-it notes Voice recorder Post cards Digital Camera



Project Complexity




Title 7. Probes

Title 8. Define Your Audience

Method ID AUDIENCE


Description Consider the broad spectrum of people who will be touched by your design solution. Before digging into the in-context research, it is critical to know who we’re designing for. We are bound to learn more once we’re in the field, but having an idea of the target audience’s needs, contexts, and history will help ensure that the research starts by asking smart questions.

Why Priorities the user from the very beginning

Make sure to understand in-depth for whom you are designing and delivering your service

Tips • Do not limit thinking to the people the service is for

• consider governments, NGOs, other businesses, or competitors

Execution Write down the people or groups that are directly involved in or reached by the challenge “Are you designing for children? For farmers?”

Write all the groups down on Post-its and put them on a wall so they can be visualized to your audience

Add people or groups who are peripherally relevant or are associated with your direct audience

Think about the connections these people have with your topic. Add them to the wall.

Arrange these Post-its into a map of the people involved in the challenge

Supporting tools NA




Project Complexity





Title 9. Extremes and Mainstreams

METHOD ID EXTREME

Interlinkages Activity 2.3

Interview

Focus Group

Description When recruiting people to interview, target both the big broad mainstream and those on either extreme of the spectrum. An idea that suits an extreme user will nearly certainly work for most others.

And without understanding what people on the far reaches of your solution need, you’ll never arrive at solutions that can work for everyone. More importantly, talking to Extreme users can spark your creativity by exposing you to use cases, hacks, and design opportunities that you’d never have imagined.

Why Designing a solution that will work for everyone means talking to both extreme users and those squarely in the middle of your target audience.

Tips Suggested Time: 30-60 Minutes

Execution Think about all the different people who might use your solution. Extreme users can fall on a number of spectrums and you’ll want variety. Maybe you’ll want to talk to someone who lives alone and someone who lives with a large extended family. Maybe you’ll want to talk to both the elderly and children. Each will offer a take on your idea that can spur new thinking.

When you talk to an extreme, ask them how they would use your solution. Ask them if they use something similar now and how it does or does not suit their needs.

Select appropriate community contacts to help arrange meetings and individual Interviews. Make sure you are talking to men and women. You might even stumble across an extreme user in another context and want to talk to them there.

Be sensitive to certain extremes when you Interview them. They may often be left out of discussions like these so make them feel welcome and let them know that their voices are critical to your research.

Supporting tools




Project Complexity





Title 10. Brainstorm

Method ID BRAINSTORM


Activity 2.1

Idea Dashboard

101Method

COCD-box

GPS

Description Brainstorming generates lots of ideas. From the large number of ideas, you can identify and extend better ideas, distinguish between predictable ones and maybe crazy ones. It is important to realise that no ideas are bad per se. You should not discard any without going in-depth in trying to understand their value.

The goal isn’t a perfect idea, it is to generate lots of ideas, collaboration, and openness to wild solutions. The last thing you want in a brainstorm is someone who, instead of coming up with ideas, only talks about why the ones already mentioned won’t work.

Brainstorming can be valuable both inside your team and when involving users, citizens or other external stakeholders.

Why To explore new, ‘outside the box’ possibilities and solutions

To define a wide range of opportunities in the fuzzy front-end of the new product development process

Tips • Welcome odd, crazy and unfeasible ideas and provide an open, funny and loose atmosphere

• Steal like an artist: combine and remix ideas of others (1+1=3)

• Make sure the central question or problem is specific and clear

• Combine with COCD-box to distil the best 'outside the box' ideas from a long list of ideas (>40)




Title 10. Brainstorm

Execution Start with a ‘divergent’ phase (withholding criticism) and focus on generating as many ideas as possible.

• In a good session, up to 100 ideas are generated in 60 minutes. Crank the ideas out quickly and build on the best ones.

Defer judgement. You never know where a good idea is going to come from.

• The key is making everyone feel like they can say the idea on their mind and allow others to build on it.

Encourage wild ideas. Wild ideas can often give rise to creative leaps.

• In thinking about ideas that are wacky or out there we tend to think about what we really want without the constraints of technology or materials.

Build on the ideas of others.

• Being positive and building on the ideas of others take some skill. In conversation, we try to use “and” instead of “but.”

Stay focused on the topic. Try to keep the discussion on target, otherwise you can diverge beyond the scope of what you are trying to design for.

• One conversation at a time. Your team is far more likely to build on an idea and make a creative leap if everyone is paying full attention to whoever is sharing a new idea.

Be visual. In live brainstorms we write down on Post-its and then put them on a wall.

• Nothing gets an idea across faster than drawing it. Doesn't matter if you are not Rembrandt!

End with a ‘convergent’ phase to structure and evaluate all relevant ideas

Supporting tools • Visual collaboration maps

• mind maps

• collages

• How-might-we

• Idea Dashboards

• Voting and Rating

• Affinity Diagram

• COCD-box

• 101Method

• Negative Brainstorm




Project Complexity

All





Title 11. Personas

Method ID PERSONA


Activity 2.2

Activity 2.3

Scenario

Workshop

Target Audience Analysis

Customer Journey Mapping

Needs Analysis

Design brief

Description Personas are fictional archetypes of the various segments of target audience. They are a detailed description of a (fictive) user and his/her behaviour. Personas are also the main characters in a scenario.

Ideally, they are based on insights gathered during interviews and focus groups and should inform all subsequent steps (especially the journey mapping). That said, they are useful tools to refer to at any given point in your creation process. In this way, they keep the perspective of the user present, not only during development but in communication with stakeholders.

Why To keep the perspective of the user present

To avoid using yourself as a reference during the design phase and be inspired by the personas

To communicate in a consistent way about your customer segments with your project’s stakeholders

Tips • Use personas throughout the entire design process

• Develop personas together with your project’s stakeholders

• Use a quote or a picture of the product's use context to enrich your personas

• Make sure to validate your personas

Execution Start with a broad identification of customer segments and collect data from each segment

Develop skeletons and prioritize them

Enrich the skeletons (names, tags, visuals, figures, etc.), and validate your personas

The description of the persona (maximum one page) includes a name, relevant demographic information, a (stock) picture or sketch/cartoon and behavioural practices relevant to the research goal. Personas are created based on research done in phase 1 and Activity 2.1, 2.2 and 2.3.

Personas are often created in workshops, or closed-door team meetings, but it is important to remember that as important as it is to base your project around personas, it is equally important to validate these personas. Personas created in workshops for example are based on assumptions which do not always




Title 11. Personas

correspond to the real-world situation. Personas should be based on your qualitative and quantitative research results.

Supporting tools Persona Template 1

Persona Template 2

Design Brief


Needs Analysis

Resources http://www.ux-lady.com/diy-user-personas/

http://designingwithpeople.rca.ac.uk/people



Project Complexity

All

Title 12. Scenarios

Method ID SCENARIO


Persona

Workshop

Environmental Analysis

Affinity Diagram

Description A process that enables you to envision a diversity of futures for your market by distinguishing likely vs less likely transition scenarios and identifying the most important determinants towards those futures

Scenarios are basically a storyline that describe possible interactions of different types of end-users with the product or service. They describe a believable

narrative, often set in the future, of a person’s experience as he orshe engages with a product or a service.

The use of scenarios in a workshop helps the participants in getting an idea of the possibilities of the product or service.

Why To create visions of possible futures or aspects of possible futures and to make grounded projections about the future of your market

To underpin strategic choices regarding long term focus and positioning

To identify 'cardinal points' (most important determinants for your envisioned scenarios), that require close monitoring


http://www.ux-lady.com/diy-user-personas/

https://github.com/open-design-kit/opendesignkit/blob/master/resource-materials/persona-worksheet.png

http://www.ux-lady.com/diy-user-personas/

http://designingwithpeople.rca.ac.uk/people



Title 12. Scenarios

To highlight discontinuities from present and to reveal choices available and their potential consequences

Tips • Environmental scanning is an ideal starting point for scenario analysis

• Link your scenarios based on personas

• Use affinity diagramming to make grounded choices to limit the set of scenario variables

• You can build scenarios yourself, in your team, or also in a workshop with other stakeholders. This will make your scenarios stronger and more realistic.

Execution Determine a set of scenario variables and distinguish certain vs uncertain trends

Based on your design challenge and your personas, you build out scenarios in terms of stakeholders involved (who), objective (what), location (where), time of day / month / year (when) and benefits (why). Identify the most important stakeholders to take into account and choose a maximum of three direct / indirect stakeholders, describe their goals and map their actions. This can be linked to the customer journey map. Build conceptual relation models between sets of scenario variables, and build variations on the 'uncertainties' Name scenarios and make narrative descriptions of them The building of scenarios can be done in a workshop setting. In this case, ask the participants to build them according to the categories mentioned above. You can let them represent the stakeholders in a form of role play.

Supporting tools Affinity Diagram

Resources U4IoT Co-creation Workshop Handbook



Project Complexity

All

Title 13. Customer Journey

Method ID CUSTOMERJOURNEY


Persona

Scenario


Workshop


https://u4iot.eu/pdf/U4IoT_CoCreativeWorkshopMethodology_Handbook.pdf



Title 13. Customer Journey

Description A customer journey is the total path an end-user takes from the starting point of a need to the completion of it. It describes the total process, and the different touchpoint with the service or system itself.

A customer journey map is the tool for building an educational narrative that follows customer interactions with your brand.

Why Customer Journey Maps help you empathize with how customers feel when they interact with your products and services

Tips • Base your customer journey map on the research done in previous activities

• Link your customer journeys to your personas

• Customer journeys can be done in an interactive workshop setting

• To pinpoint specific customer journey touchpoints that cause pain or delight

• To break down silos to create one shared, organization-wide understanding of the customer journey

• To assign ownership of key touchpoints in the journey to internal departments

Execution A common starting point for customer journey mapping is a good market / customer / user segmentation developed into a set of personas.

On the basis of these personas the entire process regarding the service is followed. The journey is usually displayed visually in a 'journey map'.

The results of customer journey mapping can provide relevant input for optimizing the customer process (e.g. where is it going wrong? Which part process is not user-friendly for which types of users?).

Supporting tools Customer Journey Map

Resources Customer Journey map template



Project Complexity

All

Title 14. Co-Creation Workshop

Method ID WORKSHOP


Co-creation workshop guidelines

Personas

Scenario

Customer Journey


https://www.mightybytes.com/wp-content/uploads/2017/01/customer-journey-template.pdf



Title 14. Co-Creation Workshop

Panel Circle

Conversation Starters

Lego Serious Play

Description The most common and basic technique for co-creating with users or citizens is a co-creation workshop or session. In activity 2.4, such workshops should be at the core of your trajectory.

In a co-creation workshop, voices can be made heard and needs, values and expectations explored. While a focus group is a guided group discussion only, a co-creation workshop is more interactive and usually more productive in terms of tangible output, based on a wide variety of methods and tools (which can include a group discussion but should not be limited to it).

Why The main advantage of organizing co-creation workshops is that a lot of qualitative data can be collected from multiple respondents during a short time period.

Tips • Be creative in adapting to context, content and research questions.

• Use the co-creation workshop guidelines for the practical considerations

• Make the most of a Co-Creation Session with Conversation Starters, a Brainstorm, Role Plays, Rapid Prototyping, or other activities to get your group engaged around the problem you are looking to solve.

• Disadvantages of the method are that group dynamics might influence some respondents’ willingness to share their opinion, particularly when their opinion is conflicting with the opinion of the group.

• An important risk to take into consideration is that the generalization of results could be overestimated.

Execution The preparation and set-up of the workshops and the analysis of the workshops results require substantial effort. It is also highly context dependent.

Building further upon this exploration of the context of the user, generative methods and tools are used to co-create together with the workshop participants.

Usually six to ten participants take part in the workshop, led by a moderator who introduces the discussion topics and a facilitator who takes notes.

Supporting tools Workshop Guidelines

Resources U4IoT Co-creation Workshop Handbook



Project Complexity

All





Title 15. Rapid Prototyping

Method ID RAPIDPRO


Activity 3.1

Lo-fidelity Prototype

Description Because prototypes are meant only to convey an idea—not to be perfect—you can quickly move through a variety of iterations, building on what you’ve learned from the people you are designing for. Rapid Prototyping makes sure that you are building only enough to test your idea, and that you are right back in there making it better once you’ve gotten the feedback you need.

Why Quickly get key feedback from the people you are designing for

Tips • Make sure to get feedback.

• Make sure to integrate feedback and iterate

Execution Once you've Determined What to Prototype, build it.

Make any number of types of prototypes: Storyboards, Role Plays, models, mock-ups. The goal here is to make something tangible that conveys the idea you want to test. No need to make it perfect, just make it good enough to get the idea across.

Now take your prototype out and test it with people you are designing for. Put it in their hands and ask them what they make of it.

Here is where you can now Integrate Feedback and Iterate. Once you've quickly built another prototype you'll do it all over again until it is just right.

Supporting tools List of possible tools: https://www.poweredbysearch.com/blog/13-rapid-prototyping-tools-for-designers/




Project Complexity

Orange/Red

Title 16. Lo-fidelity Prototype

Method ID LOFIPROTOTYPE


Activity 3.1


https://www.poweredbysearch.com/blog/13-rapid-prototyping-tools-for-designers/

https://www.poweredbysearch.com/blog/13-rapid-prototyping-tools-for-designers/





Lego Serious Play

Rapid Prototyping

MVP

Wireframes

Description A Lo-fidelity Prototype (or mock-up) is a scale or full-size model or replica of a technology or a product/service providing mainly information on its structure, used for instructional or experimental purposes. They are 'very early prototypes' made of cardboard or otherwise low-fidelity materials. For designers, they mainly function to acquire feedback.

Note that the terms "mock-ups", "low-fidelity prototypes", or "paper prototypes" are usually used as synonyms. See “Wireframes” to define the information hierarchy of a visual interface.

Why Quick to create

Helpful in enabling early visualization of alternative design solutions

Help provoke innovation and improvement

Users may feel more comfortable suggesting changes

To test prototypes without a single line of code (therefore saving time and money!)

Tips • Focus on content and functionality rather than design

• Use mock-ups to alter your ideas at an early stage

• Use mock-ups as a discussion medium between designer and user but also between the members of the design team

• Think about utilising Lego’s Serious Play method

Execution Bring together a team of diverse and relevant stakeholders (e.g. designers, end-users, programmers)

Provide templates and supporting tools (e.g. stickers of buttons, screens, pop-ups) and start the design process

See “Wireframes” to define the information hierarchy of a visual interface.

Rather than strict guidelines for execution of Lo-Fi prototypes, simple examples seem most helpful. Anything is possible, and the practitioner can become creative in choosing materials and format of the mock-up.

A mock-up of a calendar application for school kids made of cardboard and plastic. The mock-up was used to test early design ideas with school children and thus obtain early feedback in the development process:





After a couple of tests with the mock-up, it was made into a prototype programmed in Macromedia Flash. The prototype was more 'polished' and allowed for more interactivity. Using prototype, various scenarios of use (use cases) were tested with the users, after which the Flash prototype was built into a real application:

Paper mock-ups of applications and websites. One should use one piece of paper for each Web page and then users try them out in a usability test. Users indicate where they want to click to find the information and you change the page to show that screen.





A mock-up of a webpage being tested by a user:

Supporting tools E.g. Paper, polystyrene, clay, Lego, Velcro…

Resources https://www.usability.gov/how-to-and-tools/methods/prototyping.html

https://www.gov.uk/service-manual/design/making-prototypes



Project Complexity

Orange/Red


https://www.usability.gov/how-to-and-tools/methods/prototyping.html

https://www.gov.uk/service-manual/design/making-prototypes



Title 17. LEGO® SERIOUS PLAY®

Method ID LEGOSERIOUS


Activity 2.5

Activity 3.1

Mock-up

Storytelling

Scenario

Persona

Workshop

Description LEGO® SERIOUS PLAY® is a facilitation methodology created by the Lego Group and since 2010 is available under an open source model. Participants work through imaginary scenarios using visual three-dimensional Lego constructions.

It requires the active participation of 2 to 6 participants.

Why Storytelling in a cheap and fast way

Roleplay the design with internal stakeholders

Test a design that has a variety of physical aspects and involves multiple parties

Tips • Engage with experienced LEGO® SERIOUS PLAY® facilitators

• Also using LEGO elements for building mock-ups and lo-fi prototypes without following this trademarked (and expensive) method will provide you with valuable insights!

• The use of LEGO® elements enables you to take a speedy shortcut to the core. The LEGO elements work as a catalyst – and when used for building metaphors, they trigger processes that you probably were previously unaware of.

Execution During a meeting, you will be building models with LEGO elements, giving them meaning through story-making, and playing out various possible.

Start by writing the outline of the story

Prepare the physical set-up with Lego and other means

Ask workshop participants to play the scenario

“Do not play the persona, be the persona”

If you play the scenario in detail, it will quickly become clear what is missing or what parts of the design need more work

Supporting tools NA

Resources https://www.lego.com/en-us/seriousplay

http://www.artlab.org.uk/lego/slide00.htm


https://www.lego.com/en-us/seriousplay

http://www.artlab.org.uk/lego/slide00.htm



Title 17. LEGO® SERIOUS PLAY®



Project Complexity

Orange/Red

Title 18. Storytelling

Method ID STORYTELLING


Activity 2.5

Activity 3.1

Lego Serious Play

Scenario

Persona

Workshop

Description Storytelling is a creative method to communicate how users use a product. It requires the active participation of 2 to 5 participants.

Why To summarize user needs and ideas from previous project phases

To explore how a product is used from the point of view of the user, through time and its context

To reach a shared understanding of the product's functionalities

Tips • Stories can be used to communicate the current or future state of a product

• Use your personas as protagonists in the stories

Execution Every story has a plot, characters and a context. Stories can be made using:

• Text (see scenarios)

• Images (see storyboard)

• Role playing (see Lego serious play)

Supporting tools Storyboard

Resources



Project Complexity

Orange/Green




Title 19. MVP (Minimum Viable Product)

Method ID MVP


Activity 3.1

Workshop

Mock-up

Rapid Prototyping

Description Minimum Viable Product is a prototype which requires the least amount of resources to fulfil the purpose of being able to test what you are looking to test. It should be regarded as an advanced mock-up, a hi-fidelity prototype with a commercial, customer-oriented perspective.

Why Get the biggest “bang for your buck”: maximum learning with minimal effort

Tips Use the template to check whether your MVPs fulfil all defined requirements

Execution In order to learn how to create your MVP, you should check Alex Cowan’s approach on this website. Based on several case studies, he explains what’s most important for an MVP. This website offers a guide to MVPs. This link lets you download a template to check whether your MVPs fulfil all defined requirements

Supporting tools template

Resources Alex Cowan's approach to assessing MVPs

A guide to MVPs



Project Complexity

Title 20. SWOT Analysis

METHOD ID SWOT


Affinity Diagram

Focus Group


https://www.forbes.com/sites/darden/2016/06/07/how-to-design-your-next-minimum-viable-product-3-case-studies/#3762248b733b

https://scalexl.com/the-ultimate-guide-to-minimum-viable-products/

https://www.creatlr.com/template/Bw4Rbc8I4kKm0dFcdXoTje/MVP/download/

https://www.creatlr.com/template/Bw4Rbc8I4kKm0dFcdXoTje/MVP/download/

https://www.forbes.com/sites/darden/2016/06/07/how-to-design-your-next-minimum-viable-product-3-case-studies/#3762248b733b

https://scalexl.com/the-ultimate-guide-to-minimum-viable-products/



Title 20. SWOT Analysis

Description A framework to analyse the strengths, weaknesses, opportunities and threats of an idea, concept, product, service or organization. It requires the active participation of 1 to 12 participants.

Why To identify strengths, weaknesses, opportunities and threats

To discover opportunities to innovate

To facilitate focus group discussions

Tips • The SWOT analysis can be performed by individuals, but it is also helpful for group discussions

• It can also be used as a container to collect and organize information from different sources (e.g. research data)

• Make sure to translate the SWOT outcomes to actionable information (e.g. an action plan)

Execution Provide a grid with four quadrants: (S)trengths, (W)eaknesses, (O)pportunities and (T)hreats

Start with a focus on internal aspects: (S) –e.g. “what is it good at”, and (W) – e.g. “what can be improved”

Continue with a focus on external aspects: (O) – e.g. “what are the current market trends or gaps”, and (T) – e.g. “what are competitors doing”

Supporting tools

Resources http://thetoolkitproject.webflow.io/tool/swot-analysis



Project Complexity

Title 21. How Might We

Method ID HOWMIGHTWE


http://thetoolkitproject.webflow.io/tool/swot-analysis



Title 21. How Might We


Brainstorm

Workshop

Description The “how might we” approach uses “how might we...” sentences to describe ideas, starting with an open-ended problem statement or insight. This helps to shed light on different points of view and to question assumptions.

You have identified the problem you are designing for. Reframe your insight statements as “How Might We…” questions to turn those challenges into opportunities. The format suggests that a solution is possible and offers the chance to answer them in a variety of ways.

Why To frame your ideation session or design challenge,

To turn challenges into opportunities

We use the How Might We format because it suggests that a solution is possible and because they offer you the chance to answer them in a variety of ways

Tips • This method doesn’t suggest a particular solution, but gives you the perfect frame for innovative thinking

• Make sure that your How Might We’s aren’t too broad. It’s a tricky process but a good How Might We should give you both a narrow enough frame to let you know where to start your Brainstorm, but also enough breadth to give you room to explore wild ideas

Execution Start by looking at the insight statements that you’ve created. Try rephrasing them as questions by adding “How might we” at the beginning.

The goal is to find opportunities for design, so if your insights suggest several How Might We questions that’s great.

Now take a look at your How Might We question and ask yourself if it allows for a variety of solutions. If it doesn’t, broaden it. Your How Might We should generate a number of possible answers and will become a launchpad for your Brainstorms.

Supporting tools

Resources worksheet to prepare your “how might we” session



Project Complexity


https://static1.squarespace.com/static/57c6b79629687fde090a0fdd/t/589cc8b8d2b85721b37d3efe/1486670008488/HMW-Worksheet.pdf



Title 22. Design Challenge

Method ID DESIGNCHA


Activity 2.1

Activity 2.4

How might we

Workshop

Innovation Support Platform

Description A technique that helps you to explore a context and define your problem: your design challenge. Analysing a problem means obtaining a thorough understanding of the problem, its stakeholders and the facts and values involved. Ask yourself: Does my challenge drive toward ultimate impact, allow for a variety of solutions, and take into account context?

A design challenge can also be an open-ended project that encourages citizens to provide ideas and think creatively.

Why Getting the right frame on your design challenge (your “problem”) will get you off on the right foot, organize how you think about your solution and at moments of ambiguity, and help clarify where you should push your design.

Tips • A design challenge can be defined by yourself and your team.

• A design challenge is also an interesting step in a participatory co-creation workshop.

Execution A design challenge should be short and easy to remember, a single sentence that conveys what you want to do. You can phrase these as questions which set you and your team up to be solution-oriented and to generate lots of ideas along the way.

Properly framed design challenges drive toward ultimate impact, allow for a variety of solutions, and take into account constraints and context. Rethink your phrases with this in mind.

Watch out, do not scope too narrow or too broad: Too narrow won’t offer enough room to explore creative solutions, too broad won’t give you any idea where to start.

Now that you’ve run your challenge through these filters, do it again. It may seem repetitive, but the right question is key to arriving at a good solution. Test your design challenge: A quick test we often run on a design challenge is to see if we can come up with five possible solutions in just a few minutes.

Supporting tools






Title 22. Design Challenge



Project Complexity

Orange/Red

Title 23. WWWWWH

Method ID WWWWWH


Activity 2.4

Design Challenge

Design Brief

Workshop

Description Analysing a problem means obtaining a thorough understanding of the problem, its stakeholders and the facts and values involved. An important notion in problem analysis is deconstruction of the problem: by asking yourself a multitude of questions (about the stakeholders, facts, etc.), you are able to deconstruct the problem systematically. Consequently, you can review the problem and set priorities.

Here are several methods available for analysing a problem systematically, one of which is WWWWWH (who, what, where, when, why, and how?).

Why • Get greater clarity about the problem situation (the problem context)

• Gain a better understanding of the stakeholders, facts and values of the problem, and more insight into problems underlying the initial problem.

Tips • mention as many people as possible that are involved with the problem

• think also about the problems behind the problem. Try to find the essence of the problem.

• You can also ask “What for”

• Use WWWWWH in your workshop.

Execution WWWWWH (who, what, where, when, why, and how?) helps to analyse a problem by deconstructing it: asking a multitude of questions (about the stakeholders, facts, etc.) enables to deconstruct the problem systematically. Consequently, you can review the problem and set priorities. This also helps to frame your design challenge.

Write down the initial design problem in brief statements.

Ask yourself the following who, what, where, when, why, how (WWWWWH) questions to analyse the initial design problem.




Title 23. WWWWWH

Review the answers to the questions. Indicate where you need more information.

In an interactive workshop, use WWWWWH to help the participants with building scenarios.

Supporting tools NA

Resources http://wikid.io.tudelft.nl/WikID/index.php/WWWWH



Project Complexity

All

Title 24. World Café

Method ID WORLDCAFE


Workshop

Description The World Café is a simple, effective, and flexible format for hosting large group dialogue. The basic process is simple and simple to learn, but complexities and nuances of context, numbers can make it more complex.

World Café is an interactive knowledge sharing approach across mixed groups that enables participants to have creative conversations around different tables.

Why Simple, effective, flexible

Large groups

Enables creative conversations

Tips • The right questions are the core to a successful World Café session

• Create a “special” environment, most often modelled after a café ( i.e. small round tables …)

• There should be four chairs at each table (optimally) – and no more than five

• begins with a warm welcome and an introduction to the World Café process, setting the context, sharing the Cafe Etiquette, and putting participants at ease

Execution World Café needs at least twelve participants, with theoretically no upper limit.

Groups of about four to six participants sit around tables, together with a "table host", and discuss questions which have been agreed upon at the beginning of the event or defined by the organisers in advance.


http://wikid.io.tudelft.nl/WikID/index.php/WWWWH



Title 24. World Café

Each table has a different set of questions belonging to a comprehensive theme.

After approximately 20 minutes participants move to a next table where another topic - which ideally is built upon the previous one - is discussed.

Discussion results are directly noted down on a makeshift paper table-cloth or a nearby flip chart.

The "table host" welcomes new participants and informs them about the results of the previous discussion at the table.

Finally, the results of all groups will be reflected on in a common plenum session.

Strategies for further actions and opportunities for further cooperation of participants are identified.

Supporting tools

Resources http://www.theworldcafe.com/key-concepts-resources/world-cafe-method/



Project Complexity

Orange/red

Title 25. Reverse Brainstorm

Method ID REVERSEBRAIN

Interlinking Brainstorm

Workshop

Description A reverse brainstorm starts from the negative, destructive perspective in order to stipulate creativity and discussion.

Why To use the human tendency to see the negative side of things.

When a topic or problem is put forward, it is often easier to come up with reasons why it won’t work or why the problem cannot be solved rather than looking for what’s good about the idea or solutions to the problem.

Tips • Reverse the problem into solution ideas once all ideas are collected

Execution Clearly identify the problem or challenge and write it down.

Reverse the problem or challenge by asking, "How could I possibly cause the problem?" or "How could I possibly achieve the opposite effect?" For example, in the case of tourism in Puglia, ask what the participants can think of in order to completely stop (“kill”) tourism in the region.


http://www.theworldcafe.com/key-concepts-resources/world-cafe-method/



Title 25. Reverse Brainstorm

Brainstorm the reverse problem to generate reverse solution ideas. Instead of brainstorming for possible solutions the team brainstorms for everything that could make the problem worse. Allow the brainstorm ideas to flow freely. Do not reject anything at this stage.

Once you have brainstormed all the ideas to solve the reverse problem, now reverse these into solution ideas for the original problem or challenge.

Evaluate these solution ideas. Can you see a potential solution? Can you see attributes of a potential solution?

Supporting tools

Resources



Project Complexity

Orange/red

Title 26. Usability Test

Tool ID USABILITYTEST


Description Usability testing refers to evaluating a product or service by testing it with representative users. Typically, during a test, participants will try to complete typical tasks while observers watch, listen and takes notes. A test on the usability of a service may encompass anything from a quick hallway test to an organized lab set-up.

Why Usability testing lets the design and development teams identify problems before they are coded.

The earlier issues are identified and fixed, the less expensive the fixes will be in terms of both staff time and possible impact to the schedule.

Tips • To run an effective usability test, you need to develop a solid test plan, recruit participants, and then analyse and report your findings.

• Align the test with your KPIs, design principles and customer journey

Execution Usability.gov provides detailed descriptions on planning, recruiting, running and analyzing usability tests. Check the websites below.

Supporting Tree testing

Site map testing




Title 26. Usability Test

Resources https://www.usability.gov/how-to-and-tools/methods/usability-testing.html

https://www.usability.gov/how-to-and-tools/methods/planning-usability-testing.html

https://www.usability.gov/how-to-and-tools/methods/recruiting-usability-test-participants.html

https://www.usability.gov/how-to-and-tools/methods/running-usability-tests.html

https://www.usability.gov/how-to-and-tools/methods/reporting-usability-test-results.html

Categorization

Time Budget Participants Technical Complexity


Project Complexity

Orange/red

Title 27. Expert (peer) Review

Tool ID EXPERTREVIEW


Description An expert review (or website audit) is where a usability expert uses his/her knowledge and experience of testing websites with users to walk through a website in the shoes of a typical user.

Why To spot problems and recommend changes to improve usability when budgets and timescales don’t allow for user research.

Tips

Execution Make sure your experts pay full attention to the context, the nature of the system being evaluated, and the goals of the business.

Make sure your experts fully understand who the users of the system are and what the business goals are, so the initial brief is crucial.

By placing themselves in the shoes of the potential user, the expert will examine the interface in context to make recommendations that help the customer achieve their user goals

Supporting

Resources

Categorization



Project Complexity


https://www.usability.gov/how-to-and-tools/methods/usability-testing.html





https://www.usability.gov/how-to-and-tools/methods/running-usability-tests.html





Title 27. Expert (peer) Review

Orange/red

Title 28. Top Task Analysis

Tool ID TOPTASK

Interlinking Phase 3

Card Sorting

Top Task Analysis

Description A quantitative method to discover what content is popular on a website. It requires the active participation of 250 to 1000 participants.

Why • To help you optimise the design of a website

• To understand the “top tasks” the users perform on a website

• The “top tasks” are a small number of user tasks which account for a disproportionally large percentage of visits (the so-called long neck)

Tips • Make sure all major stakeholders are contributing to the list of potential top tasks

• If the list is too long, you can group tasks. A list of 100+ items is OK.

Execution Interview internal stakeholders to collect tasks they think are popular

Use a top task survey to prioritize tasks

The survey will show that 4 to 8 of these tasks account for at least 25% of the traffic. These tasks are called top tasks

Supporting

Resources https://measuringu.com/top-tasks/

Categorization



Project Complexity


https://measuringu.com/top-tasks/



Title 28. Top Task Analysis

Title 29. Hackathon

Tool ID HACKATHON


Crowdsourcing

Description Hacking is creative problem solving. (It does not have to involve technology.) A hackathon is any event of any duration where people come together to solve problems. However, it is mostly a design sprint-like event in which computer programmers and others involved in software development and subject-matter-experts collaborate intensively on projects. The goal of a hackathon is to create usable software or hardware with the goal of creating a functioning product by the end of the event.

Why To crowdsource your design or development

Tips • Don’t be too broad: have a specific and concrete focus for your hackathon.

• Organise parallel tracks for workshops and other formats together with your hackathon. Training workshops are, for instance, a great parallel track especially for newcomers but also for all participants.

Execution Hackathons typically start with one or more presentations about the event, as well as about the specific subject, if any.

Then participants suggest ideas and form teams, based on individual interests and skills.

Participants typically form groups of about 2-5 individuals, take out their laptops (if the event is technology themed), and dive into problems

Then the main work of the hackathon begins, which can last anywhere from several hours to several days.

At the end of hackathons, there is usually a series of demonstrations in which each group presents their results.

There can be a contest element.

Supporting

Resources https://hackathon.guide/

Categorization



Project Complexity


https://hackathon.guide/



Title 30. Crowdsourcing

Tool ID CROWDSOURCING

Interlinking Hackathon

Activity 4.1

Community Canvas

Description Crowdsourcing is used to describe the procedure of outsourcing to a large body of independent workers rather than through a single entity or company. Crowdsourcing allows a company to connect with a widespread network of talent, but foregoes many of the quality controls that usually exist when outsourcing to a single company.

Crowdsourcing can be a manner of getting funding in an open platform manner (so called crowdfunding), mainly from users. It’s also a great way to assess, without too much effort, whether people are willing to pay for your product. Crowd voting or polling is often used as a method for getting information from crowds about their opinions, while creative crowdsourcing involves numerous individuals in a creative work. And, even better, crowdsourcing can also be set up as a form of competition (see hackathon). This should be linked to a sustainable community of users (see community canvas).

Why Connect with a widespread network of talent

Tackle a difficult problem in short time

Tips

Execution

Supporting

Resources https://www.herox.com/crowdsourcing-news/772-a-total-beginners-guide-to-crowdsourcing

Categorization



Project Complexity

Title 31. Use Case

Tool ID USECASE



https://www.herox.com/crowdsourcing-news/772-a-total-beginners-guide-to-crowdsourcing

https://www.herox.com/crowdsourcing-news/772-a-total-beginners-guide-to-crowdsourcing



Title 31. Use Case

Description A use case is a written description of how users will perform tasks on the application. It outlines, from a user’s point of view, a system’s behaviour as it responds to a request. Each use case is represented as a sequence of simple steps, beginning with a user's goal and ending when that goal is fulfilled.

Why Use cases help explain how the system should behave and, in the process, they also help brainstorm what could go wrong.

Use cases provide a list of goals and this list can be used to establish the cost and complexity of the system.

Use cases help the team to decide which functions will be built.

Tips • Depending on how in depth and complex you want or need to get, use cases describe a combination of the following elements:

o Actor – anyone or anything that performs a behaviour (who is using the system)

o Stakeholder – someone or something with vested interests in the behaviour of the system under discussion

o Primary Actor – stakeholder who initiates an interaction with the system to achieve a goal

o Preconditions – what must be true or happen before and after the use case runs.

o Triggers – this is the event that causes the use case to be initiated. o Main success scenarios [Basic Flow] – use case in which nothing goes

wrong. o Alternative paths [Alternative Flow] – these paths are a variation on the

main theme. These exceptions are what happen when things go wrong at the system level.

Execution Identify who is going to be using the application or system.

Pick one of those users.

Define what that user wants to do with the application. Each thing the use does with the application becomes a use case.

For each use case, decide on the normal course of events when that user is using it.

Describe the basic course in the description for the use case. Describe it in terms of what the user does and what the system does in response that the user should be aware of.

When the basic course is described, consider alternate courses of events and add those to "extend" the use case.

Look for commonalities among the use cases. Extract these and note them as common course use cases.

Repeat the steps 2 through 7 for all other users.

Supporting

Resources https://www.usability.gov/how-to-and-tools/methods/use-cases.html


https://www.usability.gov/how-to-and-tools/methods/use-cases.html



Title 31. Use Case

Categorization



Project Complexity

Title 32. Customer Satisfaction

Tool ID CUSTOMERSATISFACTION

Interlinking Activity

Survey

Description Customer satisfaction is simply a measurement we use to quantify the degree to which a customer is satisfied with a product, service, or experience. What can be used for customers can also be used for your users. With digital analytics, you can determine if users are researching a goal, how they are interacting with a feature, or even their relative struggle completing a given task. But you can’t gauge their emotional response to any of that.

Why Getting insights in the satisfaction of end-users helps to define improvements.

Tips

Execution Every method of collecting data on customer satisfaction comes down to a customer survey. The secret is that with measuring customer satisfaction, you want to understand the customers’ emotional responses.

In practice, this can be executed in different survey design tactics, such as differing questions, survey response scales, and collection methods.

Sometimes it is best to collect these measures right after a transaction, or sometimes at a fixed date in the customer relationship.

Sometimes it is best to use a five-point Likert scale, and sometimes we use a Net Promoter Score (NPS) survey.

Supporting https://qualaroo.com/home






Resources https://www.hubspot.com/customer-satisfaction#how-to-measure-customer-satisfaction








https://www.hubspot.com/customer-satisfaction#how-to-measure-customer-satisfaction

https://www.hubspot.com/customer-satisfaction#how-to-measure-customer-satisfaction



Title 32. Customer Satisfaction

Categorization



Project Complexity

All

Title 33. Scrum and Sprint

Method ID SCRUMSPRINT


Description Scrum is an agile framework for managing work with an emphasis on software development.

The Scrum is included in this phase of the process, as the end goal of it is launch, which is something that most of the tools don’t address.

Why Helps to focus on a launch as the end goal during the agile development of your solution

Tips

Execution In going through this product development process, each team member is given specific roles and the process is followed thoroughly, ticking off tasks in going forward.

It is designed for teams of three to nine developers who break their work into actions that can be completed within timeboxed iterations, called sprints (30 days or less, most commonly two weeks) and track progress and re-plan in 15-minute stand-up meetings, called daily scrums.

Check for more information on the websites below

Supporting /

Resources http://www.scrumguides.org/

https://www.scrum.org/

https://www.scrum.org/resources/webcasts

Categorization



Project Complexity

All


http://www.scrumguides.org/

https://www.scrum.org/

https://www.scrum.org/resources/webcasts



Title 34. Growth Hacking

Method ID GROWTHHACKING


Business Model Canvas

Description The Growth Hacking Canvas is a working canvas, similar to the Business Model Canvas. Its main job is to describe the key elements of a growth program and their focal relationships in a way that is highly transparent.

Why Provides considerations decisive for the purpose of bringing your business, idea, project to the next level.

Tips

Execution Download the canvas from the website

Consult the website for detailed guidelines to each of the following steps

Segments & Personas: Who are the customers? What do they think? See? Feel? Do?

Value Propositions: What’s compelling about the product to these personas? Why do they buy it, use it?

Brand Experiences: What are the key experiences customers have with the product? How do they find it? Buy it? Use it? How does this differ across actors? If a CIO or parent buys it and then a support person or child uses it, how does that work?

Branding: What is the personality of the brand? Its positioning? How does it talk about itself? How do is that executed?

Lexicon: What words and phrases do customers use to talk about the area? What do they type into Google?

Assets: What are the products most important brand assets?

Activities: What are the most important growth activities?

Organic Channels: What organic (unpaid) channels are most important to the product’s branding and growth?

Paid Channels: What paid channels are most important to the product’s branding and growth?

Promotional Infrastructure: What promotional infrastructure (email lists, in store displays, social media accounts) is working for the brand?

Supporting Tool

Growth Hacking Canvas (https://www.alexandercowan.com/growth-hacking-canvas/#What_is_it)


https://www.alexandercowan.com/growth-hacking-canvas/#What_is_it

https://www.alexandercowan.com/growth-hacking-canvas/#What_is_it



Title 34. Growth Hacking

Resources

Categorization



Project Complexity

All




Annex 2: Supporting Tools

2.1 Privacy Checklist (p.80) 2.2 Open Source PIA software (p.82) 2.3 Storyboard (p.82) 2.4 Dotmocracy (p.83) 2.5 Five Why’s (p.84) 2.6 User Panel (p.85) 2.7 Conversation Starters (p.86) 2.8 Design Brief (p.87) 2.9 Idea Dashboard (p. 88) 2.10 Community Canvas (p.89) 2.11 Panel Circle (p.91) 2.12 Panel Matrix (p.91) 2.13 PESTEL (p.92 ) 2.14 COCD-Box (p. 95) 2.15 Information Architecture (p. 95) 2.16 Affinity Diagram (p.97) 2.17 GPS-brainstorm kit (p.98)

2.18 Wireframes (p. 99) 2.19 Prototype Testing Plan (p.99) 2.20 IDEO Prototyping Course (p.100) 2.21 DIGIMAT (p.101) 2.22 KPI Report (p.102) 2.23 CITADEL User Assessment Analysis (p.103) 2.24 Innovation Support Platforms (p.104) 2.25 Social Media (p.105) 2.26 Business Model Canvas (p.106) 2.28 Value Proposition Canvas (p.107) 2.29 Lean Validation Board (p.108) 2.30 CTA toolbox (p.110) 2.31 Web statistics (p.111) 2.32 Co-creation workshop guidelines (p.112)

Title 1. Privacy Checklist

Tool ID PRIVACYLIST

Interlinking Activity 0

Description The flowchart helps you to consider the most important aspects regarding privacy.

Supporting Legal Vademecum

Categorisation

Time Budget Participants Technical complexity


Project Complexity

ALL




Title 2. Open Source PIA software

Tool ID OSPIASW

Interlinking Activity 0

Description The PIA software aims to help data controllers build and demonstrate compliance to the GDPR. This tool also intends to ease the use of the PIA guides published by the CNIL.

Why To facilitate carrying out a data protection impact assessment, which will become mandatory for some processing operations as of 25 May 2018

The tool is available in French and in English.

The tool relies on a user-friendly interface to allow for a simple management of your PIAs. It clearly unfolds the privacy impact assessment methodology step by step. Several visualisation tools offer ways to quickly understand the risks.

Tips •

Execution A video tutorial can be found be found here: https://www.youtube.com/watch?v=-SdA9L4j0a8&feature=youtu.be

Resources https://www.cnil.fr/en/open-source-pia-software-helps-carry-out-data-protection-impact-assesment

Categorisation



Project complexity

Green/Orange

Title 3. Storyboard

Tool ID STORYBOARD


Storytelling

Workshop

Description A storyboard is a sequence of images to visualize how a persona uses a product or service. Use it for storytelling.

Why To help communicate the product's or service's concepts with images

To convince project stakeholders via an appealing and visual deliverable


https://www.youtube.com/watch?v=-SdA9L4j0a8&feature=youtu.be



Title 3. Storyboard

Tips • Basic drawing skills can come in handy. But also pictures and drawings from a simple google search will work fine.

• Some images can be re-used later to explain concepts to real users

Execution Start by writing the outline of the story

Draw the images on paper

Scan the images and add text and colour digitally

Supporting Storytelling

Resources https://www.storyboardthat.com

Categorisation



Project complexity

Green/Orange

Title 4. Dotmocracy

Tool ID DOTMOCRACY


Activity 2.5

Workshop

Brainstorm

COCD-box

Description Also known as do-voting, Dotmocracy is a method used to describe voting with dot stickers or marks with a marker pen. When using this approach, participants vote of their chosen option using a limited number of stickers or marks with pens, being this voting approach a form of cumulative voting.

Technically this methodology is known as “cumulative voting “. It’s a one question multiple-choice survey done with stickers.

Why A simple group activity for recognizing preferences among limited options

Tips • A variation of the method can be created by using dots of different colours to express like (e.g. green) and dislike (e.g. red). This will allow you to see which ideas have opposition.

• Unfortunately, it has a few serious weaknesses, like vote-splitting (https://en.wikipedia.org/wiki/Vote_splitting) and sticker cheating, that can cause false or misleading results.


https://www.storyboardthat.com/

https://en.wikipedia.org/wiki/Vote_splitting



Title 4. Dotmocracy

Execution Participants are each given a set number of dot stickers.

They place dot stickers next to options presented that they like.

Options with the most dots “win”.

• Keep the number of options to about a dozen or less. Participants should be expected to review, consider and compare all options before sticking their dots, and too many can be overwhelming.

• New options cannot be added once dotting has started, as this would not be fair to the new additions.

• Avoid similar or related options, as these can cause vote-splitting. This may require you to combine options to be less specific.

Have someone monitor the process to ensure no one cheats by adding extra dots, peeling off dots or moving dots.

Conduct the process more than once with the options presented in a different order, to see if the resulting pattern persists. Often people will just ‘get on the bandwagon’ and dot where everyone else has dotted.

Supporting /

Resources https://dotmocracy.org/

Categorisation


Experience and Knowledge

Project complexity

Green/Orange

Title 5. Five Why’s

Tool ID FIVEWHYS

Interlinking Interviews

Description The five why’s (or sometimes also called “Laddering”) is an interview technique leading the researcher and participant to a deeper level of explanation and therefore understanding in their interview.

It’s simple: to get to the core of the problem, you should ask “why” to every answer provided by the interviewee, five times in a row.

Why By asking why five times you’ll get to some essential answers to complicated problems.

This can be a great method to use if you are trying to get at the human and emotional roots of a problem.


https://dotmocracy.org/



Title 5. Five Why’s

Tips • Use this method while you are conducting an Interview

• Start with really broad questions like “Do you save much money?” or “How was your harvest this year?”

• Suggested Time: 15 minutes

Execution Start by asking a broad question about your Interview participant’s habits or behaviours then ask “why” to their response five times in a row.

Remember that you are not asking a horizontal question, (i.e. “Why else didn’t you get a good harvest this year?”) you are actually going for depth (i.e. “Why weren’t you able to buy the fertilizer you needed?”).

Write down what you hear, paying special attention to moments when it feels like you’ve moved a level deeper into understanding why the person does what she does.

Keep in mind that you might not get to the core stuff until the fourth or fifth “Why.”

Supporting /


Categorisation



Project complexity

Green/orange

Title 6. User Panel

Tool ID USERPAN

Interlinkages Survey

Focus Group

Interview

Community Canvas

Description A user panel is a group of individuals selected by a business or organization to provide input and opinion on products and services for research on consumer behaviour.

Panel members are chosen to be representative of the general population or a target group. If you have built a community based on the community canvas, you should find the panel members here.

Why Test public opinion in a controlled method for advertisements, products and services





Title 6. User Panel

Test products for response with less risk than a full product launch, which requires the investment of considerably more resources

Unlike focus groups, consumer panels are usually ongoing and can be reconvened quickly for input. This continuous nature allows opinions to be tracked and updated through various changes and efforts

Tips • Target a specific demographic for detailed insight on their behaviour or select a balanced demographic makeup to ensure that more users are represented

Execution Identify your audience: What types of users you want to recruit for your panel. Is it your key target audience or do you also need information from influencers?

Set goals and program parameters: What types of information do you need to gather from your participants? How many users do you need? Determine how often to connect with your panel.

Build a relationship with your users: Offering an incentive motivates your panel members to continue to respond and provide insightful feedback

Supporting Community Canvas

Resources /

Categorisation

Time Budget Participants

Technical complexity


Project complexity

ALL, especially orange/red

Title 7. Conversation Starters

Tool ID CONSTAR


Focus Group

Interview

Workshop

Description The objective is to put a bunch of ideas in front of citizens to spark their reactions. The goal here is to encourage creativity and outside-the-box thinking from the people you are designing for.

Why The idea here is to suggest a bunch of ideas around a central theme to the people you are designing for and then see how they react. The ideas you generate for your




Title 7. Conversation Starters

Conversation Starters are totally sacrificial, so if they don’t work, drop them and move on.

Tips • Suggested Time: 30-60 Minutes

Execution Determine what you want the people you are designing for to react to.

Come up with many ideas that could get the conversation started.

Start by telling them that you are interested in their reactions to these Conversation Starters. Some may be silly, some may be absurd, and you are only looking to get their opinions.

When one of them strikes them, ask them follow-up questions. You can learn a lot about how they think and what she might want out of your solution.

Supporting /

Resources Pens, notebook


Categorisation


Experience and Knowledge

Project complexity

Green/orange

Title 8. Design Brief

Tool ID DESIGNBRIEF


Design Challenge

Description A design brief is a document that you create after you’ve done some exploration, but before you start conceptualising and experimenting. It formalises findings and possible design approaches. It lists every important piece of information related to your service idea. It is interlinked with your design challenges

This is done by the PAs. Once research is conducted and insights identified and classed, the PA can formulate a design brief that converges all collected information and compresses it in the form of key insights, something that you have decided, based on your experiences, that is worth exploring further.

This is what a client would give, e.g. to a designer. It can also serve as a useful tool to ensure that the whole team is on the same page.

Why Communicate the solution to a problem





Title 8. Design Brief

Help the team to align on deliverable(s) and factors that might contribute to the satisfactory conclusion of work.

Tips • 1 hour to write, 2-5 hours to combine all relevant insights from previous activities

• Never apologize in a design brief

• Keep it succinct

• Only talk about (design) features that have a direct correlation with your problem

• Write as if your audience has no idea about the project since this is the best way to tell the whole story.

Execution A design brief may consist of different aspects according to the service in question and the context. Follow following structure and broad categories for your brief to make sure that everything is covered:

• Insight: Share your most relevant observation from previous steps.

• Therefore: Explain the conclusions you’ve come to based on the insights.

• Concept: Articulate the concept by revealing your actual idea in a few sentences.

• Execution: Communicate how the concept will convey in the project/design you are creating. (this can be linked to the panel matrix)

• Benefit: Reveal the reason why you are executing the project in this way and how it will relay the perks to the consumer.

• Message: State the takeaway for your organisation based on the project you’ve described.

• Objective: Reiterate the goal that was outlined in the initial project assignment.

Supporting tools

One of many Design Brief templates to be found online

Resources http://opendesignkit.org/methods/design-brief/

Categorisation




Project complexity

ALL

Title 9. Idea Dashboard

Tool ID DASHBOARD


Design Brief


https://docs.google.com/document/d/1HWWb39_3S4PX6mxKg5u394QKrh_UtbsnDideciWMHzU/edit

http://opendesignkit.org/methods/design-brief/



Title 9. Idea Dashboard

Activity 2.4

Activity 2.5

Workshop

Prototype

Brainstorm

Description Idea dashboards help to collect ideas, but especially to make sure that everyone understands the ideas. Explaining something in words can form different images in different minds.

Why Using material objects, drawings of low-fi prototypes helps all members of a team to see the same discussed solution in the same way.

Tips • Use an idea dashboard early in your project

Execution Fill in the dashboard, i.e. a worksheet (see template). This should be linked to your design brief.

This helps you to articulate what your (early stage) idea or solution is.

Capturing a few details on this sheet can help you think through some of the basic details of your idea.

Creating a dashboard puts the team on the same page.

Supporting Idea Dashboard template

Resources https://dschool.stanford.edu/resources/idea-dashboard

Categorisation




Project complexity

ALL

Title 10. Community Canvas

Tool ID COMCAN


Activity 2.2

Activity 2.3

Description The community Canvas is a framework that will help you build a community, analyse a community or improve an existing community. It has 3 sections: Identity, Experience, and Structure. These sections are divided into 17 themes. For each of the 17 themes, we have identified a set of questions that will help find the right answers for your own community.


https://dschool.stanford.edu/resources/idea-dashboard




Why Creating a community around your project and its management can be helpful or even required.

The community can contribute to the design, validation and implementation, bring new ideas, together with the stakeholders you are targeting.

A well-established and run community can help reach impact and sustainability.

Tips • It is not a framework that provides you with the actual answers.

• Every community is as unique as the humans in it. Therefore, even though most communities share a similar structure, the answers will be wildly different across different kinds of communities.

• All Community Canvas documents are released under Creative Commons license.

Execution Identity: The first part of the Canvas focuses on questions of belief. These aspects of the community are less tangible than the points covered in later parts of the Canvas. (Themes: Purpose, Member Identity, Value, Success Definition, Brand) Experience: Explore the community from the perspective of the members. What does actually happen in the community and how does it translate its purpose into activities that create tangible value for the members? (Themes: Selection, Transition, Shared Experiences, Rituals, Content, Rules, Roles) Structure: The third part of the Canvas focuses on the operational elements of running a community, focussing especially on sustainability, that is stability in the longer term. Organizational aspects are often neglected, and the necessary structures aren’t in place to deal with challenging situations, as they eventually and often suddenly come up. (Themes: Organization, Governance, Financing, Channels & Platforms, Data Management)

Supporting tools

Resources with downloadable material:

summary of the Community Canvas

pdf of the Community Canvas

Community Canvas template


http://bit.ly/community-canvas-summary1

http://bit.ly/community-canvas-summary1

http://bit.ly/community-canvas-guidebook1

http://bit.ly/community-canvas-guidebook1

http://bit.ly/community-canvas-worksheet-summary




Community Canvas Worksheet Doc

Categorisation




Project complexity

ALL

Title 11. Panel Circle

Tool ID PANCIRCLE


Activity 2.2

Activity 2.3

Panel Matrix

Workshop

Description Over the last years imec.livinglabs developed a two-step method to keep a project-overview of the defined feedback steps of the panel (segments): first to map all involved stakeholders in a Living Lab panel and then to organize that panel in a panel matrix. The panel circle is the tool for the first step, used to identify, cluster, and organise stakeholders (=users) in relation to their active involvement within the project and its planned user activities.

Organizing at least one workshop with the different stakeholders offers the opportunity to broaden the views of the (sub)groups of ‘users’ who are involved in relation to the problem definition(s). By mapping all (newly) identified (sub)groups into a panel circle, the relationships between these groups will become clearer and more structured/clustered.

The next step is the panel matrix.

Why Broaden the view on earlier identified stakeholders

Creating insights regarding the level of involvement of each stakeholder (group)

Generate discussion around involving stakeholders as users

Tips • The Panel Circle can be used in an interactive workshop setting if you want to integrate input of other people in your organisation or external stakeholders

• Use panel matrix to link the stakeholders’ (=users) of the panel circle to the planned user activities within the project in order


https://docs.google.com/document/d/1fyuwwqX911i9CK1BjY8jXkk3v2m8ZCHeV3dVqYuDiGM/edit



Title 11. Panel Circle

Execution Write down each individual stakeholder(group) on a post it (1 stakeholder/post-it)

Brainstorm which of this stakeholder(groups) need to be clustered/deepened, e.g. when there’s a post-it with ‘Citizens’:

• deepen by dividing this group further into male, female, young, old,

• cluster by taking together post-its with different descriptions about the same stakeholder(groups)

Decide which stakeholder (groups) is the most important to be involved during the project and put this one in the middle.

!!! Attention!!! There can only be one stakeholder (group) placed in the centre of the circle

Next, place other stakeholders on the circle based on how closely they are related to the one in the middle.

People most closely related to the central stakeholder will be in the second shell, and so on…

Supporting tools

Resources http://dev.u4iot.eu/pdf/U4IoT-toolkit-Panel-management.pdf

Categorisation




Project complexity

ALL

Title 12. Panel Matrix

Tool ID PANMA

65+Health care

provider

Carer

Family

Volunteer

Pharmacist

General Practicioner

City

Government

CPAS

GP Circuit

Hospital

Specialist


http://dev.u4iot.eu/pdf/U4IoT-toolkit-Panel-management.pdf



Title 12. Panel Matrix


Activity 2.3

Panel Circle

Description Over the last years imec.livinglabs developed a two-step method to keep a project-overview of the defined feedback steps of the panel (segments): first to map all involved stakeholders in a Living Lab panel and then to organize that panel in a panel matrix. The panel matrix is the tool for the second step, to link identified stakeholders (=users) with organised activities within a project.

After the Panel Circles have been created, and all (sub)groups are identified and clustered, these (sub)groups are transferred to a panel matrix. This matrix combines the groups with the defined research/user activities of the project/initiative. It offers the possibility to create an overview on which groups would have to be involved in which activities.

Why Creating awareness about the importance of user involvement within the project

A project-overview of the defined feedback-steps of the user panel (segments) and the impact of the undertaken steps to achieve the goals set within the project

Increasing the possibility to adjust strategies if necessary without losing the initial planning out of sight

Follow-up of successes & failures

One transparent overview of all panel activities offering the necessary insights to evaluate and facilitate their efforts

Tips • Use panel circles to map & cluster the identified stakeholders’ (=users) relation to the project in order to discover their level of involvement

Execution Map all the stakeholders (and their subgroups) in the left column of the matrix

Map all identified activities (phase by phase) in the upper row of the matrix

Link stakeholders & activities with an ‘x’

Replace (if relevant) ‘x’ with numbers

Supporting tools

Matrix Excel Sheet (imec.livinglabs)

Resources http://dev.u4iot.eu/pdf/U4IoT-toolkit-Panel-management.pdf

Categorisation




Project complexity

ALL


http://dev.u4iot.eu/pdf/U4IoT-toolkit-Panel-management.pdf



Title 13. PESTEL

TOOL ID PESTEL


Description The PESTEL analysis is a strategic framework to understand the macro-environmental factors in your environment.

Why To understand the wider environment in which your company operates

To identify threats and opportunities in the environment

Tips • Don’t fall into the trap of generating an unstructured list of random trends

• Focus on your business branch and find relevant trends

• Select the most important trends, then structure and prioritize them

Execution Identify major trends in the political, economic, social and technologic environment (PEST)

Alternatively include ecological, legal and ethical dimensions of the environment (STEEPLE)

Build a strategy responding to these environmental trends

Supporting tools

Resources Kew, J. & Stredwick, J. (2005). Business Environment: Managing in a Strategic Context. London: CIPD House.

Categorisation



Project complexity

ALL




Title 14. COCD-Box

Tool ID COCD


Dotmocracy

Description A technique to prioritize ideas after a creative brainstorm.

Why A successful creative brainstorm can generate a lot of ideas (>40) and you want to select the best out of them: in terms of feasibility, originality and added value to your project To reveal the real 'out of the box' ideas via quick, democratic and simple selection process

Tips • This technique is only useful if you have more than 40 distinct ideas

• Let participants choose their top five individually in advance

• Start sticking dots only after everyone has made his/her shortlist

Execution N (ideas) / N (participants) = N (votes) for each participant Every respondent selects top 5 ideas Use coloured dots to vote on selected ideas. Every colour represents a category of ideas (see image): NOW (feasible and good), HOW (not yet feasible but original), and WOW (ideas that are both feasible and original). WOW ideas have priority.

Supporting tools

Resources https://www.cocd.org/

Categorisation



Project complexity

ALL

Title 15. Information Architecture

Tool ID INFOARCH


https://www.cocd.org/





Card Sorting

Top Task Analysis

Description A discipline that focuses on structuring and disclosing information in a user-centric way

Why To help in structuring content

To facilitate the design of taxonomies, navigation and search for websites and applications

To inspire the design of web pages

Tips • Start with the design of the taxonomy. Page designs will come later

• Focus on real content right from the start

• Design mobile first and desktop later

Execution Use card sorting and collaborative mind mapping for building taxonomies (see image as example) and metadata.

Use top task analysis to discover popular content

Use content modelling and the cores & paths technique for designing pages and links between pages or content blocks

Supporting /

Resources /

Categorisation



Project complexity





ALL

Title 16. Affinity Diagram

Tool ID AFFINITYDIAGRAM


Focus Group

Scenario

Persona

SWOT

Description Affinity Diagraming is a very simple but powerful (focus group) tool for grouping and understanding (large amounts of) information. It requires the active participation of 6 to 12 participants.

Why To converge and bring structure in large and divergent amounts of information

To identify mental and emotional associations between a set of concepts, ideas, objects, facts, figures, etc.

To cluster similar items (e.g. ideas) based on perceived relationships.

Also, useful to group a long list of ideas coming from brainstorms

Tips • Ideal preparation to group information into dimensions for scenario analysis, persona creation or opportunity identification

Execution Everyone prepares a set of cards/sticky notes that contain concepts, facts and figures, based on desk research or previous research

Randomly spread these cards on the table, the floor or the wall

Cluster these cards step by step, then discuss on the meaning of each cluster, and name each group

It is important to reach group consensus

Start with a silent phase (grouping) and end with a discussion (group meanings)

If a card belongs to two groups, make a duplicate

Supporting

Resources http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html

Categorisation



Project complexity

ALL


http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html



Title 17. GPS-brainstormkit

Tool ID GPS


COCD-box

Description A structured method for ideation that leverages on existing trends. It requires the active participation of 8 to 15 participants.

Why The GPS-brainstorm kit is a tool for creative brainstorming based on existing trends, resulting in a lot of ideas

Tips • Start your central question with “How can we …”

• Participants should be a mix of the organization’s stakeholders and users

• Work out the selected ideas' advantages, threats, and prerequisites

Execution Identify five trends and determine a central question

Generate ideas within trends (post-its). Do this with one to three persons per trend. Rotate trends every five to ten minutes.

Select ideas to further expound via voting

Supporting

Resources https://www.flandersdc.be/nl/gids/tools/gps

Categorisation




Project complexity

Orange/red


https://www.flandersdc.be/nl/gids/tools/gps



Title 18. Wireframes

Tool ID WIREFRAMES


Lo-fidelity prototyping

Rapid Prototyping

Description A prototyping method to define the information hierarchy of a visual interface, while stripping out aesthetic elements.

Why To develop the architecture, functionalities and the logic of the innovation in a schematic way (skeleton)

To design the lay-out of the interface with a clear focus on what the elements do and not what they look like

Tips • Collect inspiring examples to enhance your creativity

• Raw wireframes can be gradually enhanced by adding typography, greyscales, adding visual details

• Software allows the creation of clickable mock-ups without a single line of code

Execution Choose the most appropriate tool (can be on paper, but is often software, e.g. Axure, Fireworks)

Define the grid (framework in which you place the different interface components) and fill it with labelled boxes (blank squares with a short description of the content)

Create multiple ‘pages’ and define the interaction between them (wire flows, a set of wireframes which are related through interactive components)

Supporting

Resources https://www.usability.gov/sites/default/files/creating-wireframes.pdf

Categorisation



Project complexity

Title 19. Prototype Testing Plan

Tool ID PROTOTYPEPLAN


Activity 3.1

Rapid Prototyping

Prototyping

Wireframes


https://www.usability.gov/sites/default/files/creating-wireframes.pdf



Title 19. Prototype Testing Plan

Description The Prototype Testing Plan gives a basic, but useful overview of the different ways in which you can test your work, as well as when to test it. You can build a prototype using various materials, or simply draw or act out your idea. The website, including a free template, provides simple and clear instructions for planning your prototype. There is also a video about the why’s & how’s behind this method.

Why basic, but useful overview Clear instructions

Tips requires some dialogue with colleagues/peers so plan for some time to interact and fill out in collaboration

Execution The Prototype Testing Plan also helps structure the testing process. It is most efficient if you go through a structured series of steps. This way you can continually improve your work, while avoiding getting lost once the feedback you collect starts piling up.

The worksheet indicates two periods when it is usually beneficial to test your idea: in the early stage of development and in the later stages just before full implementation.

Use the worksheet as a basic guide to help plan your prototype tests. Always clearly specify the main idea you want to test out through your prototype. And make sure to note down any learnings on how to improve your work by reallocating activities, resources, people or materials.

Supporting Prototype testing plan template

Resources http://diytoolkit.org/tools/prototype-testing-plan/

Categorisation



Project Complexity

Title 20. IDEO Prototyping Course

Tool ID IDEOPROTOTYPING COURSE


Activity 3.1

Prototyping

Description This is an intensive, hands-on learning experience that will equip you to prototype products, services, interactions, and environments. You’ll learn methods that IDEO.org designers use to quickly build prototypes, learn best practices for testing those ideas in the field, and collecting real user feedback to iterate efficiently.

The course lasts four weeks and is focusses on prototyping. The course starts at set intervals, so check when the next course is starting.


http://diytoolkit.org/tools/prototype-testing-plan/



Title 20. IDEO Prototyping Course

Why Practice the mindsets and methods of human-centred design to produce tangible results quickly

Experiment with methods to prototype products, services, interactions, and environments

Gain valuable feedback from the people you are designing for and incorporating that feedback into new iterations

Demonstrate how the cycle of prototyping, feedback, and iteration leads to concept refinement

Tips

Execution Team-Based Course: We strongly encourage (but do not require) that you form a team of 2-8 people to take the course with. It works best to find friends or co-workers who can meet in person. Need to find team members? You can post to the discussion forums a month before the course starts.

Platform: NovoEd (+Acumen and IDEO.org developed the course. We host this course on the NovoEd platform.)

Timing: This course will be open for 5 weeks. You can work through the materials at your own pace during the time the course is open. There are no required times for you to log in. There will be suggested deadlines for you to complete each assignment. The final deadline is the final day of the course.

Supporting

Resources https://www.plusacumen.org/courses/prototyping

Categorisation



Project complexity

Title 21. Digimat

Tool ID DIGIMAT


Description The main objective of this tool is to assess the Digital Maturity of a Public Administration to provide improvement recommendations. The result of the assessment will be a general level of maturity complemented with a set of improvement recommendations. This tool is based on a set of questionnaires in order to be able to analyse the answers in an automatic way and to provide recommendations to improve the digital maturity of the PA. These recommendations could be very useful in the identification of the needs


https://www.plusacumen.org/courses/prototyping



Title 21. Digimat

Why Get recommendations to help PAs in the identification of possible needs

Tips • The questionnaire should be filled in by different roles to collect the most appropriate answers in each dimension.

Execution Access to the questionnaire

Create a user to fill in the questionnaire

Select which roles in the organisation should fill in the questions

Collect the results and recommendations

Analysis the recommendations and identify future needs

Supporting tools

DIGIMAT is a web tool

Resources https://integration.citadel.esilab.org/digimat/

Categorisation



Project complexity

Title 22. KPI Report

Tool ID KPI Report


Activity 1.2

Activity 4.2

Description The main focus of this tool is to facilitate the process of harvesting data from various sources that can contain key performance indicators for the utility of e-Government services, curating datasets that originate from a multitude of formats and allow fusing them into semantically enhanced data. The output can be further processed by analytics and/or recommender engines to suggest public service improvement needs. Also, this tool allows to visualise this output in a graphical view to facilitate the identification of future needs

Why Setting clear KPI’s sets clear expectations for all parties on what needs to be achieved within the project lifespan. Working with KPI’s negates any discussion after the facts, both between project partners but also as a form of transparency towards other stakeholders.


https://integration.citadel.esilab.org/digimat/



Title 22. KPI Report

Tips Setting ‘good’ KPI’s is paramount. Look at existing KPI’s rather than creating your own. If no model exists, look at the available data and how that can relate to KPI’s. Setting up additional data gathering will be an extra effort. KPI’s should be chosen with the relevant stakeholders of that KPI (i.e. those who will gather the data, perform the activities related to the data, those who will process and evaluate the data).

Execution Choose KPI’s based on clear desired project outcomes.

Choose evaluation framework for KPI’s: both quantifiable and qualitative data can be used. Scores can be used to harmonise quanti and quali data.

Do not choose too many KPI’s. Make sure indicators clearly relate to what you want measured.

Data gathering from all relevant parties

Data scoring or processing into KPI framework

Communicating KPI outcomes

Supporting tools

Resources https://gitlab.ilabt.imec.be/citadel-2/tutorial-201901/wikis/home

Categorisation



Project complexity

Title 23. CITADEL User Assessment Analysis

Tool ID USERASSESSMENT


Activity 1.2

Activity 4.2

Description CITADEL provides a tool that on one hand allows citizens to assess the service that they have used through a ranking and a free text comment. And on the other hand, this tool provides the PAs with relevant information extracted from the sentiment analysis of the comments provided by the citizens.

Why CITADEL User Assessment tool provides to PA a system where to check the punctuation received by each service and the analysis of other sources where the users have expressed their experience with digital public services (i.e. social networks).

Tips


https://gitlab.ilabt.imec.be/citadel-2/tutorial-201901/wikis/home



Title 23. CITADEL User Assessment Analysis

Execution From the User perspective:

Once the citizens have been used the services, they could rate the services considering several aspects using a “star” rating and could provide comments about the services

From the PA Perspective:

At a first step, PAs can create their own questionnaires with those questions that are more relevant for them. Just one questionnaire is valid at each time.

Once citizens have assessed the service, in one hand, PAs can check the ratings of the services and which service is the best rated and in the other hand PAs can analyse the comments of the citizens because this tool provides information based on a sentiment analysis.

Supporting tools

https://integration.citadel.esilab.org/assessment (Analysis)

https://integration.citadel.esilab.org/rating (Rating)

Resources https://integration.citadel.esilab.org/assessment (Analysis)

https://integration.citadel.esilab.org/rating (Rating)

Categorisation



Project complexity

Title 24. Innovation Support Platforms

Tool ID SUPPORTPLATFORM


Activity 1.5

Activity 2.4

Description Innovation Support Platforms are convenient tools for collecting the feedback on your challenge. Different options exist in this regard and should be chosen based on budget considerations and personal preferences.

Why To have a one stop shop for interacting with your community

Tips • Many such platforms exist, so have a look around to find the most suitable one for your specific context, including the language you need.

Execution The CITADEL Innovation Support Platform supports the management of the co-creation process, providing a virtual space where citizens can collaborate proposing ideas, commenting and discussing other people ideas, expressing their appreciation for ideas,


https://integration.citadel.esilab.org/assessment

https://integration.citadel.esilab.org/rating

https://integration.citadel.esilab.org/assessment

https://integration.citadel.esilab.org/rating



Title 24. Innovation Support Platforms

and allows the platform administrator to create challenges, thing that gives the possibility to guide people ideas towards the achievement of a specific objective.

The Welive platform has been developed by WeLive project [4]. WeLive platform focuses on how to pass from innovation to adoption, by democratizing the creation process and fostering public-private partnership that will jointly exploit the outcomes of the innovation process. This platform tackles the whole innovation process phases (defined in WeLive project): a) conceptualization, b) voting and selection, c) funding, d) development and e) promotion and f) exploitation

Supporting

Resources https://dev.welive.eu/

Categorisation




Project complexity

Title 25. Social Media

Tool ID SOCIALMEDIA


Activity 2.3

Activity 2.4

Activity 4.1

Activity 4.2

Description Within the co-creation process social media are relevant for disseminating results and for gathering feedback of the stakeholders. Social media also contributes to generate public values on services that have been yet defined and are entering the market.

Why Social media can be a strong tool to reach a large in a short amount of time.

Disseminating and explaining your outcomes is a part of implementation - spreading the word, getting your idea out there.

Tips • Creating a documented strategy focused on processes and goals will improve your results

Execution To create a social media strategy to communicate your outcome, take a look at the CoSchedule blog which provides the best way to plan a social media social media strategy in 5 steps with a free template.

Supporting /


https://dev.welive.eu/



Title 25. Social Media

Resources https://coschedule.com/blog/social-media-marketing-strategy-template/

Categorisation



Project complexity

Title 26. Business Model Canvas

Tool ID BUSINESSMODELCANVAS


Value Proposition Canvas

Growth Hacking

Description Business Model Canvas is a strategic management and lean start-up template for developing new or documenting existing business models. It is a visual chart with elements describing a firm's or product's value proposition, infrastructure, customers, and finances. It assists firms in aligning their activities by illustrating potential trade-offs. It is a widely used.

The Business Model Canvas was initially proposed by Osterwalder. Since the release of Osterwalder's work in 2008, new canvases for specific niches have appeared (see e.g. Value Proposition Canvas).

Why To make sure to cover all vital aspects to make your solution also commercially sustainable

Tips • Pay special attention to the value proposition, to make sure you are creating real value for your end-users.

• The customer relationships and customer segments fields must be completed with true and validated information, not assumptions!

• The Business Model Canvas is also available in web-based software format.

Execution The Business Model Canvas can be printed out on a large surface so groups of people can jointly start sketching and discussing business model elements with post-it note notes or board markers.

It is a hands-on tool that fosters understanding, discussion, creativity, and analysis.

It is distributed under a Creative Commons license from Strategyzer AG and can be used without any restrictions for modelling businesses.

Supporting Value Proposition Canvas

Resources https://strategyzer.com/canvas/business-model-canvas


https://coschedule.com/blog/social-media-marketing-strategy-template/

https://strategyzer.com/canvas/business-model-canvas



Title 26. Business Model Canvas

Categorisation



Project complexity

Title 27. Value Proposition Canvas

Tool ID VALUEPROPOSITIONCANVAS


Business Model Canvas

Description The Value Proposition Canvas complements the Business Model Canvas. It allows you to focus on the value proposition you intend to offer to your users/customers. Paying special attention to the wants and needs of your users is most significant.

Why • To pay special attention to the wants and needs of your users

• Detailed look at the relationship between two parts of the Business Model Canvas; customer segments and value propositions

• Refine an existing product or service offering or where a new offering is being developed from scratch

Tips Identifying the value proposition on paper is only the first stage. It is then necessary to validate what is important to customers and get their feedback on the value proposition.




Title 27. Value Proposition Canvas

Execution After listing gain creators, pain relievers and products and services, each point identified can be ranked from nice to have to essential in terms of value to the customer.

A fit is achieved when the products and services offered as part of the value proposition address the most significant pains and gains from the customer profile.

These insights can then be used to go back and continually refine the proposition.

Supporting /

Resources https://www.b2binternational.com/research/methods/faq/what-is-the-value-proposition-canvas/ https://strategyzer.com/canvas/value-proposition-canvas

Categorisation



Project complexity

Title 28. Lean Validation Board

Tool ID LEANVALIDATION

Interlinking Focus Group

Survey

Brainstorm

Workshop

Social Media


https://www.b2binternational.com/research/methods/faq/what-is-the-value-proposition-canvas/

https://www.b2binternational.com/research/methods/faq/what-is-the-value-proposition-canvas/

https://strategyzer.com/canvas/value-proposition-canvas




Description The Lean Start-up Validation Board is a one-page tool created by Lean Start-up Machine. It's a way to test early stage ideas, to make sure that they are valuable enough to continue working on them.

Why To turn assumptions into facts

To not lose track in the complicated process

Make sure to solve a problem that your users actually care about

Tips • You don't have to be planning a start-up to use this methodology or to think in terms of customers. It can also help you and your team for general strategy etc.

Execution Customer Hypothesis: create a profile of your target customer. For the hypothesis, a customer is someone who gives you something. This could be their time, information, money, or anything else of value. Some ideas have two customers. Importantly, this could change dramatically as more is learned. Problem Hypothesis: your theory about a problem that your customer (previously identified) has that you can solve. Assumptions: These are assumptions about the customer and problem hypothesis that require testing. Solution Hypothesis: Since this was the first time through the board. There was no solution hypothesis developed. Riskiest Assumption: Of the assumptions generated, one is the riskiest and if it fails your test, then the whole idea needs to be revisited - the pivot. Method (Validation Approach): These are the activities you'll pursue to test the riskiest assumption of your hypothesis in the fastest way possible. It's time to get creative. Minimum Success Criterion: This is the smallest or least response that validates your idea. In other words, if at least "N" persons won't say "yes" to whatever you are asking. Then you have to decide if you didn't find the right customers - and then you need to change your Customer Hypothesis - or if you haven't identified an important (as in it matches your time/effort/money criteria) enough problem to solve.

Supporting Focus Group

Survey

Brainstorm

Workshop

Social Media

Resources https://www.leanstartupmachine.com/validationboard/


https://www.leanstartupmachine.com/validationboard/




Categorisation



Project complexity

Title 29. CTA toolbox

Tool ID CTATOOLBOX


Activity 3.3

Description CTA stands for Constructive Technology Assessment. If you are interested to look at societal and ethical aspects in your research this 'toolbox' is a good starting point. It provides an overview and description of several approaches that support you to consider economic, societal, ethical and legal aspects in technical research projects.

Why Consider the alignment of technological advances, which your solution should provide, with values, needs and expectations of society, i.e. your target group.

Tips • Socio-technical configuration can also be applied in the beginning of the project if an application is already envisioned, to understand what is needed for a technology to work




Title 29. CTA toolbox

Execution The tools have been selected because of their suitability for socio-technical integration approaches and range from desk-top analyses to highly interactive settings such as a workshop.

The ‘toolbox’ provides practical hints for the implementation of the described approaches

It is a starting point to get (practical) information about methods integrating economic, societal, ethical and legal aspects in technical research projects

An overview contains a short description and key characteristics of each tool, which support you to find the most suitable tools for you.

Supporting /

Resources https://cta-toolbox.nl/

Categorisation



Project complexity

Title 30. Web statistics

Tool ID WEBSTATISTICS


Description Web analytics is the collection, reporting, and analysis of website data. The focus is on identifying measures based on your organizational and user goals and using the website data to determine the success or failure of those goals and to drive strategy and improve the user’s experience.

Why Drive strategy and improve the user’s experience

Tips • Analyse web statistics according KPIs. Simplify and focus on the KPIs that you need to measure your organizational and user goals

• Encourage a data-driven environment for decision making

• Avoid only providing traffic reports. Reporting about visits, pageviews, top sources, or top pages only skims the surface.

• Always provide insights with the data. Reporting metrics to your stakeholders with no insights or tie-ins to your business or user goals misses the point.

• Avoid being snapshot-focused in reporting. Focusing on visits or looking only within a specific time period doesn’t capture the richer and more complex web experiences that are happening online now.

Execution Critical to developing relevant and effective web analysis is creating objectives and calls-to-action from your organizational and site visitors’ goals, and identifying key


https://cta-toolbox.nl/



Title 30. Web statistics

performance indicators (KPIs) to measure the success or failures for those objectives and calls-to-action.

Check the website for examples and more information.

Supporting https://marketingplatform.google.com/about/analytics/#?modal_active=none

https://matomo.org/

https://snowplowanalytics.com/

https://www.zapbi.com/

Resources https://www.usability.gov/what-and-why/web-analytics.html

Categorisation



Project complexity

Title 31. Co-creation workshop guidelines

Tool ID CCCWORKSHOP


Workshop

Description The most common and basic technique for co-creating with users or citizens is a co-creation workshop or session. Here, voices can be made heard and needs, values and expectations explored. Building further upon this exploration of the context of the user, generative methods and tools are used to co-create together with the workshop participants. Usually six to ten participants take part in the workshop, led by a moderator who introduces the discussion topics and a facilitator who takes notes. The group interaction can trigger discussion amongst the respondents and contradictory responses or creations can be discussed immediately.

Co-creation workshops are part of the qualitative research tradition, which generally leads to a deeper level of meaning and sense making than quantitative methods. The main advantage of organizing co-creation workshops is that a lot of qualitative data can be collected from multiple respondents during a short time period. However, the preparation and set-up of the workshops and the analysis of the workshops results require substantial effort. Disadvantages of the method are that group dynamics might influence some respondents’ willingness to share their opinion, particularly when their opinion is conflicting with the opinion of the group. An important risk to take into consideration is that the generalization of results could be overestimated. As it concerns qualitative research, the results are only valid for the respondents that take part in the study and the results are not meant to be generalized to the wider population.

Although workshops are but one among many techniques that will be proposed in the Cc-c methodology, it is regarded as worthwhile for this report to go into depths in this


https://marketingplatform.google.com/about/analytics/#?modal_active=none

https://matomo.org/

https://snowplowanalytics.com/

https://www.zapbi.com/

https://www.usability.gov/what-and-why/web-analytics.html




regard. Not only are workshops indeed a main tool of the co-creation practitioner. The detailed description that follows will also serve to illustrate that co-creation is often a down to earth approach that relies on activities that are far from spectacular.

The following subsection will be structured according to the organisation of a co-creation workshop and the factors that may make it a successful endeavour. It starts by considering the preparation of a workshop. It then describes some practical arrangements and critical factors for running it. Finally, it considers the work to be done after the workshop.

Execution 1. Preparation of a co-creation workshop Before starting with the organization of a workshop, it is important to explore problem and context, to be able to define the research goal and research question and to identify the possible end-users of the to-be-developed product or service.

Exploration of the problem and the context: An important first step is to explore the current context of users, the context of use and the state of the art of existing technologies.

Definition of the research goal and the research questions: The exploration of the problem and the context should lead to a clear vision of the goal of the innovation. Make sure that a collaborative vision is created about the innovation that is shared by all stakeholders. A clear goal should also be defined for the organization of a co-creation workshop. These goals differ based on the stage in the service life cycle. Once the goal is defined, specific research question can be formulated.

Identification of the end-user: An end-user is the person (i.e. citizen) who ultimately uses the end product or service. It is thus important to identify for whom the product or service is developed. Once the end-users are identified, it should be delineated which end-users will be invited to participate in the workshop. As the limited set-up of a co-creation workshop usually doesn’t allow inviting all identified end-users, one or a few subsamples should be selected based on the research questions. Here, CITADEL’S processes will support the PAs. A very common sampling strategy is purposive sampling, where recruitment happens based on a set of criteria relevant to the research question (e.g. people with asthma) [7] . Sandelowski [8] also calls this ‘criterion sampling’. Bringing in a different persona might be useful “to help bring different perspectives to the process” [9]. A difference here can be made between ‘maximum variation’ and a ‘phenomenal variation’ [8]. Maximum variation implies obtaining a maximal spread in respondents regarding some meaningful characteristics, e.g. socio-demographic characteristics (spread in age, household composition, work situation etc.). Phenomenal variation stands for a variation in certain phenomena that are relevant for the research topics (e.g. respiratory diseases).

2. Practical organization of a co-creation workshop Once key research goal and end-users are defined, the preparations can start for the organization of the co-creation workshop. Many practical arrangements are to be made:

Set a date and time: Take into account the possible availabilities of the research sample. If the co-creation workshop is for example targeted towards working people, it is better to organize the workshop in the evening.

Find a good location: Look for a convenient to reach location with the necessary facilities. Make sure a large table and enough chairs are present in the room.





Incentives: Specify if you will give the workshop participant an incentive for their participation, and what incentive this will be (e.g. an online coupon).

Recruit the workshop participants: Make an invitation to recruit the workshop participants. This can be a letter, an email, a social media post, a (online) poster… Specify in the invitation the goal of the workshop and the innovation, possible selection criteria (e.g. people with asthma), the date and time, the location, the incentive (optionally) and how interested people can sign up for the workshop. In order to purposefully select the participants based on predefined selection criteria, the invitation can include a link to a short online survey with some extra questions (e.g. demographical information).

Select the workshop participants: Send a confirmation of their participation with some more practical information: detailed directions, parking facilities, duration of the workshop, state if drinks and/or food will be provided, contact information (in case they don’t find the way, will be delayed or not be able to make it). Ask the selected participants to confirm their attendance. Don’t forget to send a reminder the day before the start of the workshop.

Always invite two people more to participate than the required minimum number of participants. It is very common that people cancel or just don’t show up, so prepare for this.

Make the topic list: Some topic lists already have a complete overview of all specific questions which should be asked, whereas others just include a list of topics to be covered. Also describe the co-creation exercises and techniques in this topic list. It is very important to include in this topic guide a strict timing for every topic and exercise.

Informed consent: Informed consent means that the research participants “understand what it means to participate in a particular research study, so they can decide in a conscious, deliberate way whether they want to participate” [7]. The informed consent usually is a form that is signed by the research participant. The informed consent should describe [7][: The research goal; The research method; What is expected of the research participants; Risks and benefits (if present); A statement that participation is voluntary and everyone can withdraw at any time; A confidentiality/privacy statement; Name and contact information of the researcher in case of further questions. Print two informed consents for every participant: one they can keep and one for the researcher.

Catering: Provide drinks and optionally food.

Arrange the necessary supplies (e.g. sticky notes, pens, scissors, paper…)

Recording devices: During the co-creation workshops, somebody should be present to take notes. However, to make sure that all information is captured, it is highly advised to (video) record the co-creation workshop. Make sure to select a recording device with a sufficient battery that will last for the duration of the workshop. Set up the recording equipment in the room before the start of the workshop. If possible, also record the workshop also with a second device (e.g. a smartphone), in case something would happen to the first recording.

Make a PowerPoint presentation: It is convenient to make a presentation to use during the workshop with basic information (e.g. the agenda, what is a co-creation session?) and optionally with documentation for the exercises.

Provide a data projector or look for a room with an installed projector. This is necessary to show the PowerPoint presentation.





Provide name tags: Make these nametags in advance and give them to the attendees when they arrive. These name tags are important, since the help the researcher and the attendees in addressing each other personally.

Set-up of the room: Make sure the table and chairs set up in such a way that everybody can see each other, the moderator and the presentation screen. Put the camera at a spot in the room or on the table where all the participants will appear in the image.

3. Running a co-creation workshop A co-creation workshop usually lasts 1,5 to 2 hours. At least two people should be present to run a co-creation workshop: a moderator and a note-taker/facilitator. Besides the people who run the workshop and the workshop participants, it is advised that no extra people are present in the room without a specific role as this might give the participants an uncomfortable feeling. Workshops usually follow these steps:

I. Start the workshop with presenting yourself and other researchers present. II. Ask the research participants to sign the informed consent. First orally explain to

them what an informed consent is and what is in the document, and then give the participants the necessary time to carefully read the document.

III. Refer to the recording device(s) in the room. Mention that this will only be used for the processing of the workshop, and the recording will never be used for another purpose. Only after this explanation, switch the recording devices on.

IV. Start with giving an overview of the agenda and the topics to be discussed. Also, shortly explain what a co-creation workshop is and what is expected of the participants (e.g. there are no ‘wrong’ answers). State some ground rules (e.g. please don’t interrupt others while speaking). Use a PowerPoint presentation to present this information.

V. Presentation of the workshop participants: Do a round-table in which all the attendees can present themselves. Ask them to state their name, age, occupation. This should last maximum 5 minutes.

VI. Start the actual workshop using the topic guide. Try to strictly follow the pre-defined timings of every topic and co-creation exercise.

VII. At the end of the workshop, do a round-table again. This gives all attendees the opportunity to give a last comments or additional remarks on the discussed topics.

VIII. If an incentive for participation was promised, give everybody the incentive. a. The role of the moderator

The moderator is the person who leads the workshop discussions and exercises. She or he asks the questions, guides the conversations, explains and wraps-up the exercises. It is important for the moderator to [7]:

. Create a relaxed and positive atmosphere, among others by being friendly, making eye contact and using a relaxed body language.

. Create an atmosphere of mutual respect between the participants and leaders of the workshop, among others by stating some basic rules at the start of the focus group (e.g. don’t interrupt others) and emphasizing that every contribution is meaningful. Everybody should feel comfortable to contribute.

. Limit his/her own participation to the discussion. “A good moderator should be skilled at creating a discussion in which he or she participates very little” [7]

. Be very familiar with the topic guide. To be able to fluently lead the discussion, it is important that the moderator is very familiar with the topic guide. This enables the moderator to ask follow-up questions related to what is being





discussed without losing sight of what is in the topic list. It is not necessary to follow the exact order of the topic list. If a participant starts discussing a topic which is only addressed at a later moment in the topic list, it is good to immediately take advantage of this for the natural flow of the conversation. However, make sure that an overview is kept at all times of which topics still have to be discussed. It might help to check of questions in the topic guide once they have been discussed. It is also okay to say to the participant that a certain topic will be discussed at a later moment.

. Involve everybody. When a certain participant doesn’t contribute, the moderator should try to encourage this person to contribute. Politely ask a question directly to that person.

. Never ask leading or biased questions. It is very important for the moderator not to ask leading or biased questions that might make the participants answer in a certain way. “Leading questions are questions worded in such as way as to influence participants’ responses – in other words, questions that lead participants along a particular line of thinking” [40, p. 63]. An example:

. Unbiased question: Why do you want to retrieve air quality information about your city?

. Biased question: Do you want to retrieve air quality information about your city because you are afraid air pollution might affect your health?

. Never add your own opinion or vision to what is being discussed. The moderator should always be neutral.

. Respect silences. Don’t be afraid of silences after a question is asked. Give the attendees time to think. Somebody will answer eventually.

b. The role of the note-taker/facilitator Besides the moderator, it is important that an extra person is present to facilitate the smooth running of the focus group and to assist the moderator when necessary. This way the moderator can fully focus upon leading the workshop. Besides facilitating, another very important role for this person is taking notes. The note-taker takes notes of the discussions and writes down observations. The workshop should be recorded as well but taking notes real-time can save you a lot of work. The recordings can be used in case the note-taker was not able to write down all information real-time or for further exploration. As a facilitator of the workshop, this person should:

. Help the moderator in passing around papers, pens, co-creation materials…

. Check the recording devices regularly to see if they’re still recording.

. Track the time. Inform the moderator if certain topics discussion or exercises are taking too much time.

. Assist people who arrive late or have to leave early. When taking notes, it is important to [7]:

. Also observe the non-verbal behaviour. Write down observations as well, but clearly indicate in your notes that these are observations and not actual things somebody said.

. Be very familiar with the topic guide. Also, for the note-taker it is useful to know the topic guide very well. If the discussions do not strictly follow the topic guide, this helps in identifying with which question the discussion is associated. Also, it gives you a better insight into the purpose of each question and which





answers or quotes are necessary to write down. If the moderator would forget a crucial topic, you can inform him/her on this.

. Identify the participants. If you write down something a person had said or done, always mention who that participant was.

It is not always possible to write down everything, since people might talk faster than you are able to write. Some tips for taking faster notes [7]:

. Only write down key phrases and use key words for the rest (expand your notes after the workshop, see step 4.

. Use abbreviations.

. If an important conversation goes too fast and you are not able to write down everything, write down the exact timing (since the start of the workshop, e.g. 00:45) so you can easily find that moment in the recording to listen to it again afterwards.

. Create a list beforehand in which every participant is assigned a number. This way you only have to write the number of the participant instead of their complete name when you write down something about this participant.

. Make sure every question or discussion topic in the topic guide is numbered. This way you only have to write down the question number and not the complete question. To avoid confusion with the numbers that act as the participant identifiers, put a Q before every question number (e.g. Q1, Q2 …).

4. After the workshop After the workshop has finished, take pictures of all the outputs (e.g. collections of sticky notes, created wireframes …). When the workshop is finished, the note-taker should expand the notes as soon as possible (within 24 hours). The sooner you do this, the more the discussions are still fresh in mind and the higher the chances are that you will still remember things which you have not written down. The moderator should also go through these notes as soon as possible, so he/she can expand these notes with possible extra insights and observations.

Supporting See also U4IoT Co-creation Workshop Handbook


https://dschool.stanford.edu/resources/gear-up-how-to-kick-off-a-crash-course

Resources

Categorisation



Project complexity







Annex 3: ICT-based co-production of public services: Barrier and Enablers. A systematic review.

Governments and international organizations are heralding Information and communication technologies (ICTs) as a new panacea in their quest to augment the co-production of public services by their administrations and citizens (European Commission, 2018; OECD, 2018a). ICTs fundamentally consist of the convergence of telephone networks, audio-visual systems and computers, and facilitate the capture, transmission and display of data and information electronically (OECD, 2018a). Public service co-production is the (voluntary or involuntary) involvement of public service users in the design, management, delivery and/or evaluation of public services (Osborne et al., 2016). The deployment of ICTs for the co-production process is seen as a powerful means of boosting the opportunities to co-produce public services, a particularly attractive goal in the era of budget-strapped governments seeking to innovate and improve public value delivery (Linders, 2012). Crucially, ICTs are thought to facilitate the potential for a two-way, continuous, mass communication between citizens and governments. For the first time, mass collaboration, dissemination and production now become imaginable (Linders, 2012). Potentially, ICTs could be used to promote the development of a “user-generated approach” to public service co-production (Meijer, 2011). Citizens thus can express their varied points of view on public services, encouraging debate and exchange of data and other information (Granier and Kudo, 2016; Uppström and Lönn, 2017; West, 2004), which could be channeled towards improved public service quality, engagement and relevance (Szkuta et al., 2014). Despite their rhetorical enthusiasm, governments have been slower than expected to adopt ICTs with a view to enhancing co-production (Meijer, 2015, OECD, 2018a). However, in recent years, governments have started to roll out ambitious programs to promote citizen participation. Examples include the Australian government’s Artificial Intelligence (AI) assistant avatar to facilitate access to government services of the disabled (OECD, 2018a), or the United Nations’ partnership with Microsoft to launch a social innovation hub to enable young women to start up their own businesses, by supporting them with ICT training and resources (OECD, 2018b).

In response to the rhetoric around ICTs’ potential for public service co-production, and governments’ emerging efforts to promote it at a large scale in practice, some scholars have expressed skepticism about the effect’s ICTs will have on co-production processes. For example, Criado and Villodre (2018) have pointed out that, just as ICTs created a “digital divide” as regards telecommunications have/have nots, ICT-based co-production may do a similar thing, enabling co-production in some scenarios whilst posing a barrier in other contexts, potentially even raising a “double digital divide”. This double digital divide could mean, for example, that particular countries, regions or groups of citizens, could successfully and fruitfully adopt ICT-based co-production, leaving others behind.

Before the introduction of ICTs into the co-production process, a significant body of research had already found traditional forms of public service co-production to be highly uneven. Commonly cited enablers and barriers to co-production by government include funding availability and professional skills (Bovaird and Loeffler, 2012), public professional culture (Tuurnas, 2015) and legal and institutional structures (Torvinen and Ulkuniemi, 2016; Williams et al., 2016). As regards citizens, enablers and barriers to public service co-production have been associated with motivation to engage (Fledderus and Honingh, 2016), demographic factors (Alonso et al., forthcoming) and social capital (Thijssen and Van Dooren, 2015). The impact of ICTs on both already existing public service co-production, as well as new initiatives to co-produce, is likely to be heterogenous. Whilst, in some cases, ICTs may well facilitate greater and deeper co-production, there will likely be other cases where ICTs act as a barrier to these processes.




But what do we actually know about the ways in which ICTs enable or pose barriers to public service co-production? A body of research has emerged, starting slowly from 2000 onwards, but accelerating from the decade starting in 2010, in parallel with the spread of ICT-based co-production initiatives by governments (Bonsón et al., 2012; Ho, 2002; Porumbescu, 2016; Tursunbayeva et al., 2017; Uppström and Lönn, 2017; West, 2004). Within this literature, a sub-strand has examined evidence on the ways in which ICTs enable or pose a barrier to co-production (see, for example, Castelnovo, 2016; Da Silva and Albano, 2017; Lecluijze et al., 2015; Meijer, 2012). To date, there has been no systematic review of the literature on the impact of ICTs on public service co-production, hence, the contribution of this paper is to fill this gap, by performing, to the best of our knowledge, the first systematic review on the topic.

Findings identify structural and cultural factors that act as barriers to, or enablers of, ICT-based public service co-production in the cases of both governments and citizens. As regards government, the most important factors include financial and technical capacities, legal issues and organizational culture. For citizens, we identify technical skills, demographic factors (particular age and gender), social dynamics, as well as a number of cultural factors associated with citizen trust to be relevant.

The rest of the paper is organized as follows. The second section presents the organizational framework which is adapted from Meijer (2015). The third section provides information on the methodology used for the systematic review as well as data collection. The fourth section discusses the results of the publication characteristics as well as the literature analysis. The final section concludes with our main findings and describes the limitations of this research and future lines for research on this topic.

References

Alonso, J. M., Andrews, R., Clifton, J., & Díaz-Fuentes, D. (Forthcoming). Factors influencing citizens’ co-production of environmental services: a multi-level analysis. Public Management Review.

Bonsón, E., Torres, L., Royo, S., & Flores, F. (2012). Local e-government 2.0: Social media and corporate transparency in municipalities. Government Information Quarterly, 29(2), 123–132.

Bovaird, T., & Loeffler, E. (2012). From Engagement to Co-production: The Contribution of Users and Communities to Outcomes and Public Value. Voluntas, 23(4), 1119–1138.

Castelnovo, W. (2016). Co-production makes cities smarter: Citizens’ participation in smart city initiatives. SpringerBriefs in Applied Sciences and Technology, 97–117.

Criado, I., & Villodre, J. (2018). Public employees in social media communities: Exploring factors for internal collaboration using social network analysis. First Monday, 23(4).

Da Silva, C., & Albano, C. (2017). Open data intermediaries: coproduction in budget transparency. Transforming Government: People, Process and Policy, 11(1), 119–131.

European Commission. (2018). Co-production: Enhancing the role of citizens in governance and service delivery (Technical Dossier no. 4). Luxembourg.

Fledderus, J., & Honingh, M. (2016). Why people co-produce within activation services: the necessity of motivation and trust – an investigation of selection biases in a municipal activation programme in the Netherlands. International Review of Administrative Sciences, 82(1), 69–87.

Granier, B., & Kudo, H. (2016). How are citizens involved in smart cities? Analysing citizen participation in Japanese ``Smart Communities’’. Information Polity, 21(1), 61–76.

Ho, A. T.-K. (2002). Reinventing Local Governments and the E-Government Initiative. Public Administration Review, 62(4), 434–444.

Lecluijze, I., Penders, B., Feron, F. J. M., & Horstman, K. (2015). Co-production of ICT and children at risk: The introduction of the Child Index in Dutch child welfare. Children and Youth Services Review, 56, 161–168.




Linders, D. (2012). From e-government to we-government: Defining a typology for citizen coproduction in the age of social media. Government Information Quarterly, 29(4), 446–454.

Meijer, A. (2011). Networked coproduction of public services in virtual communities: From a government centric to a community approach to public service support. Public Administration Review, 71(4), 598–607.

Meijer, A. (2012). Co-production in an Information Age: Individual and Community Engagement Supported by New Media. Voluntas, 23(4), 1156–1172.

Meijer, A. (2015). E-governance innovation: Barriers and strategies. Government Information Quarterly, 32(2), 198–206.

OECD. (2018a). Transformative Technologies and Jobs of the Future: Background report for the Canadian G7 Innovation Ministers’ Meeting.

OECD. (2018b). Embracing Innovation in Government: Global Trends 2018

Osborne, S. P., Radnor, Z., & Strokosch, K. (2016). Co-Production and the Co-Creation of Value in Public Services: A suitable case for treatment? Public Management Review, 18(5), 639–653.

Porumbescu, G. A. (2016). Linking public sector social media and e-government website use to trust in government. Government Information Quarterly, 33(2), 291–304.

Szkuta, K., Pizzicannella, R., & Osimo, D. (2014). Collaborative approaches to public sector innovation: A scoping study. Telecommunications Policy, 38(5–6), 558–567.

Torvinen, H., & Ulkuniemi, P. (2016). End-user engagement within innovative public procurement practices: A case study on public–private partnership procurement. Industrial Marketing Management, 58, 58–68.

Thijssen, P., & Van Dooren, W. (2015). Who you are/where you live: do neighbourhood characteristics explain co-production? International Review of Administrative Sciences, 82(1), 88–109.

Tursunbayeva, A., Franco, M., & Pagliari, C. (2017). Use of social media for e-Government in the public health sector: A systematic review of published studies. Government Information Quarterly, 34(2), 270–282.

Tuurnas, S. (2015). Learning to co-produce? The perspective of public service professionals. International Journal of Public Sector Management, 28(7), 583–598.

Uppström, E., & Lönn, C.-M. (2017). Explaining value co-creation and co-destruction in e-government using boundary object theory. Government Information Quarterly, 34(3), 406–420.

West, D. M. (2004). E-government and the transformation of service delivery and citizen attitudes. Public Administration Review, 64(1), 15–27.

Williams, B. N., Kang, S.-C., & Johnson, J. (2016). (Co)-Contamination as the Dark Side of Co-Production: Public value failures in co-production processes. Public Management Review, 18(5), 692–717.




Annex 4: Explaining citizens’ co-production through ICTs in health: A multi-level model analysis.

Citizen co-production of public services has been studied and discussed for some decades (Bovaird, 2007; Meijer, 2011; Whitaker, 1980). However, increased use of information and communication technologies (ICTs) in the public sphere has given rise to new forms of co-production (European Commission, 2018; OECD, 2018). ‘ICT-based’ co-production consists of the implementation and deployment of ICTs in a form of collaborative structure between citizens and government to produce public services (Moon, 2018; Rodríguez Muller, 2019; Yeh, 2017). ICTs-based co-production enables a two-way, continuous and mass communication that allows citizens to express their varied point of view, encourage debate and exchange information (Granier & Kudo, 2016; Uppström & Lönn, 2017; West, 2004).

A large body of research has largely focused on motivational and demographic influences on citizens’ engagement with public services (Bovaird et al., 2015; Fledderus and Honingh, 2016; van Eijk and Steen, 2016). However, empirical evidence incorporating the impact of ICTs to explain citizen engagement in public services’ co-production is under researched (Meijer, 2015). Of the existing, emerging research, scholars have sought to identify which attitudinal characteristics help explain why citizens do not use ICTs for accessing public services (Van de Walle et al., 2018). Other authors have pointed out, just as ICTs created a “digital divide”, ICT-based co-production may do a similar thing, raising a “double digital divide” affecting specific groups of citizens (Criado and Villodre, 2018).

In the health sector, ICTs can support citizens with a range of health care needs by, for example, allowing electronic communication among health and care professionals, patients, caregivers and other providers within health and care systems (World Health Organization, 2018). ICTs also offer innovation solutions to foster citizens to actively engage in health actions. For example, citizens’ engagement in the collection of atmospheric information helped scientists understand the effects of pollution on health, climate and air traffic (World Health Organization, 2017). However, insufficient ICT knowledge and technical skills associated with a less citizen groups’ participation have the potential to create health inequalities (Ravn and Mejlgaard, 2015). In response to this, some health organizations such as the NHS England have made efforts to improve the use of electronic health apps, offering citizens’ training on basic online skills and information on using transactional service online (World Health Organization, 2017).

Using data from a large health database that contains a sample of 5850 respondents, this paper develops a multi-level model to study citizens ICT-based co-production behaviors in health sector. This research design is particularly pertinent for understanding individual-level factors nested in a higher level of analysis, where it is possible to measure geographical contextual effect (Hox et al., 2017). As regards the individual-level factors of our analysis, this paper explores the influence of citizens’ factors associated with the use or not of ICTs in health co-production in Northern Ireland (UK). In particular, it focuses on identifying if the impact of motivational factors such as locus of control, self-efficacy and self-esteem (Fledderus and Honingh, 2016), as well as demographic characteristics such as age, education and gender (Alonso et al., forthcoming; Parrado et al, 2013) have a significant effect on citizens’ ICT-based co-production in the health sector. This analysis involves a comparison between two specified co-production behaviors: The first includes health activities carried out using an online computer such as health information searches on the Internet, communication with a health provider by e-mail and usage of chat groups to learn about health topics. The second involves traditional health behaviors that do not require the use of an Internet-enabled computer, such as exercising regularly, following a healthy diet and caring for others’ practices. As regards our knowledge about the effect of geographical structures on the analysis, this paper examines if citizens’ ICT-based co-production behaviors vary across the five regional health trusts in the country.




Northern Ireland reflects an interesting geographic split as result of historical demographic shifts that may influence its citizens’ co-production behaviors (Northern Ireland Department of Health, 2018).

Findings indicate that individuals who exhibit higher level of locus of control, self-efficacy as well as lower levels of self-esteem and health perception are more likely to co-produce health through ICTs. Findings show the existence of an ‘inverse gender’ effect, through which females participate more than males in the two included co-production models. However, it is observed a lower participation of older people than young people only in the ICT model, suggesting evidence of a digital ‘grey’ divide effect. In terms of geographical effects, it is found that the use of ICTs is less prevalent in rural areas, suggesting the existence of a ‘rural/urban’ digital divide. Regional differences across the five regional health trusts in Northern Ireland are identified to influence citizen’s health co-production in the non-ICT model, but not in the ICT model.

The rest of the paper is organized as follows. We first review the literature on the citizen use of ICTs in public services’ co-production, with a particular emphasis on health sector. We then explain the key measures, hypothesis and analytics methods. This is followed by the results. Finally, we conclude with a discussion about implications.

References

Alonso, J. M., Andrews, R., Clifton, J., & Díaz-Fuentes, D. (Forthcoming). Factors influencing citizens’ co-production of environmental services: a multi-level analysis. Forthcoming.

Bovaird, T. (2007). Beyond engagement and participation: User and community coproduction of public services. Public Administration Review, 67(5), 846–860.

Bovaird, T., Van Ryzin, G. G., Loeffler, E., & Parrado, S. (2015). Activating citizens to participate in collective co-production of public services. Journal of Social Policy, 44(1), 1–23. https://doi.org/10.1017/S0047279414000567

Criado, I., & Villodre, J. (2018). Public employees in social media communities: Exploring factors for internal collaboration using social network analysis. First Monday, 23(4).

European Commission. (2018). Co-production: Enhancing the role of citizens in governance and service delivery. https://doi.org/10.2767/562

Fledderus, J., & Honingh, M. (2016). Why people co-produce within activation services: the necessity of motivation and trust – an investigation of selection biases in a municipal activation programme in the {Netherlands}. International Review of Administrative Sciences, 82(1), 69–87. https://doi.org/10.1177/0020852314566006

Fledderus, J., & Honingh, M. (2016). Why people co-produce within activation services: the necessity of motivation and trust – an investigation of selection biases in a municipal activation programme in the Netherlands. International Review of Administrative Sciences, 82(1), 69–87. https://doi.org/10.1177/0020852314566006

Granier, B., & Kudo, H. (2016). How are citizens involved in smart cities? Analysing citizen participation in Japanese “smart Communities.” Information Polity, 21(1), 61–76. https://doi.org/10.3233/IP-150367

Hox, J. J., Moerbeek, M., & Van de Schoot, R. (2017). Multilevel Analysis: Techniques and Applications. (Routledge, Ed.) (3rd Editio). New York, NY.

Meijer, A. (2015). E-governance innovation: Barriers and strategies. Government Information Quarterly, 32(2), 198–206. https://doi.org/10.1016/j.giq.2015.01.001

Meijer, A. J. (2011). Networked coproduction of public services in virtual communities: From a government centric to a community approach to public service support. Public Administration Review, 71(4), 598–607.




Moon, M. J. (2018). Evolution of co-production in the information age: crowdsourcing as a model of web-based co-production in Korea. Policy and Society, 37(3), 294–309.

Northern Ireland Department of Health Social Services and Public Safety. (2018). Co-production Guide: Connecting and Realising Value Through People.

OECD. (2018). Transformative Technologies and Jobs of the Future: Background report for the Canadian G7 Innovation Ministers’ Meeting. Montreal (Canada).

Parrado, S., van Ryzin, G. G., Bovaird, T., & Löffler, E. (2013). Correlates of Co-production: Evidence From a Five-Nation Survey of Citizens. International Public Management Journal, 16(1), 85–112. https://doi.org/10.1080/10967494.2013.796260

Ravn, T., & Mejlgaard, N. (2015). Public engagement innovations - catalogue of PE initiatives. D.1.2. Brussels: Public Engagement Innovations for Horizon 2020 (PE2020) Project EU.

Rodríguez Muller, P. (2019). ICT-Based Coproduction of Public Services: When and How Dies It Lead to Better Realizing Public Values. Public Governance Institute, KU Leuven.

Thijssen, P., & Van Dooren, W. (2015). Who you are/where you live: do neighbourhood characteristics explain co-production? International Review of Administrative Sciences, 82(1), 88–109.

Uppström, E., & Lönn, C. M. (2017). Explaining value co-creation and co-destruction in e-government using boundary object theory. Government Information Quarterly, 34(3), 406–420. https://doi.org/10.1016/j.giq.2017.08.001

Van de Walle, S., Zeibote, Z., Stacenko, S., & Migchelbrink, K. (2018). Explaining non-adoption of electronic government services by citizens. A study among non-users of public e-services in Latvia. Information Polity.

van Eijk, C., & Steen, T. (2016). Why engage in co-production of public services? Mixing theory and empirical evidence. International Review of Administrative Sciences, 82(1), 28–46. https://doi.org/10.1177/0020852314566007

West, D. M. (2004). E-government and the transformation of service delivery and citizen attitudes. Public Administration Review, 64(1), 15–27.

Whitaker, G. P. (1980). Coproduction: Citizen Participation in Service Delivery. Coproduction: Citizen Participation in Service Delivery. Retrieved from https://www.researchgate.net/publication/245800027_Coproduction_Citizen_Participation_in_Service_Delivery

World Health Organization. (2017). Engagement and participation for health equity. Cardiff (UK).

World Health Organization. (2018). Continuity and coordination of care: A practical brief to support implemntation of the WHO Framework on integrated people-centred health services. Geneva.

Yeh, H. (2017). The effects of successful ICT-based smart city services: From citizens’ perspectives. Government Information Quarterly.




Annex 5: Additional Resources

Many great guides and courses already exist in the field of co-creation, service design and participatory design. The Cc-c methodology includes some of these as additional resources, that can be useful for its users.

Table 3. CITADEL Co-Creation methodology – Additional Resources

Resource Description

Design Thinking Crash Course

Ideo and d.school are known in the field. They offer a “Design Thinking Crash Course” compressed in a short amount of time. This is an ideal starting point for all those who are interested in the co-creating services. The course can be found on the website. Here, you can also find different, downloadable material that is very helpful: https://dschool.stanford.edu/resources/gear-up-how-to-kick-off-a-crash-course

Have a look at the playbook, for example, an instruction manual for the facilitator:

Coursera Course on Agile + Design Thinking

“Despite everyone's good intentions, hard work and solid ideas, too many projects end up creating unneeded, unusable, and unsellable products. But it doesn't have to be this way. Agile and design thinking offer a different--and effective--approach to product development, one that results in valuable solutions to meaningful problems. In this course, you’ll learn how to determine what's valuable to a user early in the process--to frontload value--by focusing your team on testable narratives about the user and creating a strong shared perspective. https://www.coursera.org/learn/uva-darden-getting-started-agile

IDEO toolkit The IDEO toolkit is very accessible and user friendly. Several of the methods and tools listed in this methodology are also listed there. If you are looking for more instructions, browse Ideo’s toolkit, too: https://www.ideo.com/tools

IDEO Designkit

The IDEO Designkit is a great place to get more inspiration regarding human-centred design. http://www.designkit.org/

NESTA guide for designing public services

Also, the NESTA guide for designing public services is a great overview of methods and tools. Here, however, the focus is on public service design. Created by Nesta and IDEO as part of Design for Europe, it offers ways to do things differently by introducing the process of design thinking.

Importantly, it also provides guidance on how to introduce the new approach into day-to-day work in the public sector: https://www.nesta.org.uk/toolkit/designing-for-public-services-a-practical-guide/

Living Lab Methodology Handbook

Similarly to the Design Thinking Crash Course, this handbook is providing the framework for an entire innovation journey from start to finish, as well as key principles of Living Labs: Living Lab methodology handbook.

https://www.ltu.se/cms_fs/1.101555!/file/LivingLabsMethodologyBook_web.pdf

U4IoT End User Engagement Toolkit

The European project U4IoT put together a well-designed and accessible toolkit for end user engagement. http://u4iot.eu/end-user-engagement-toolkit

U4IoT Living Lab Methodology Handbook

The U4IoT Living Lab Methodology Handbook is a comprehensive introduction into the livinglab approach.

https://drive.google.com/file/d/1168aWXTib0-_oGtch1cIRG47I5FY8wSY/view




https://www.coursera.org/learn/uva-darden-getting-started-agile

https://www.coursera.org/learn/uva-darden-getting-started-agile

http://www.designkit.org/

https://www.nesta.org.uk/toolkit/designing-for-public-services-a-practical-guide/

https://www.nesta.org.uk/toolkit/designing-for-public-services-a-practical-guide/



http://u4iot.eu/end-user-engagement-toolkit

https://drive.google.com/file/d/1168aWXTib0-_oGtch1cIRG47I5FY8wSY/view



Resource Description

IDEO course on prototyping

This is an intensive, hands-on learning experience that will equip you to prototype products, services, interactions, and environments. You’ll learn methods that IDEO.org designers use to quickly build prototypes, learn best practices for testing those ideas in the field, and collecting real user feedback to iterate efficiently.

The course lasts four weeks and is focusses on prototypting. The course starts at set intervals, so check when the next course is starting.


SILK The SILK method, developed by the Social Innovation Lab Kent, is a set of participatory design methodologies. The method cards provide insights for all phases of the co-creation process, from the very early phases to the later ones such as communicating the outcomes of your project. They are divided into five themes: plan, communicate, insight, workshop and design.

http://socialinnovation.typepad.com/silk/about-silk-1.html



http://socialinnovation.typepad.com/silk/about-silk-1.html

empowering citizens to transform european public ......d3.8 - final citadel methodology for...

Documents