philippines evaluation results

“CAP on a Map”Dissemination Workshop

Nuwan Waidyanathanuwan {at} sahanafoundation {dot} org

Kunming, China

Philippines Warning and Situational-Awareness: Lesson Learned

www.sahanafoundation.org

What is Sahana?

2

● Sahana EDEN is open source.

● No licensing fees or restrictions.

● You have “ the rights to study, change, and distribute the software to anyone and for any purpose.”

● Sahana adopts MIT license

Sahana is an Open Source Community with a mission to save lives by providing free and open source information management systems that improves the effectiveness of organizations and communities throughout the disaster cycle.

Driving adoption to over 25 countries with 20 different disaster management modules


Sahana Alerting and Messaging Broker

3

function

scope

architecture keep it Simple

Alerting / Warning(EDXL-CAP)

SAMBRO

● Functions within the Preparedness and Response disaster management phases

● Messaging Broker allowing to interconnect with other brokers to integrate Response Organizations

● Alerting / Warning dissemination is the key function

● Map-based visualization and interaction for improved situational-awareness


SAMBRO Interconnection Capabilities

4

BROWSER APP

Google Public Alerts Red Cross Hazard APP

MOBILE APP

Federation of Internet Alerts

SAMBRO Server (Browser App) and Mobile APP talk to each other

SAMBRO Server can talk with Google, IFRC, FIA, and any other CAP Alert Hubs


What is Situational-Awareness?

“Cross-Agency Situational Awareness System is an information aggregation system that facilitates sharing situational awareness within the public safety community. Information shared relates to incidents and planned events. It includes public alerts, risks to responders, and community profiles.” - Canada’s Multi-Agency Situational-Awareness

5


Why do we need Situational-Awareness

6

Perception Comprehension Projection

What is happening?

Why do I care? What do I do about it?


Effects of Situational-Awareness

7

1. Improves information sharing among first-responders (e.g. Common Alerting Picture)

2. Immediate collaboration in response and mitigation

3. Creates connected agencies for public safety

4. Manages resource more efficiently and cost effectively

5. Saving lives and Livelihoods

Situational-AwarenessDecision Perform

ActionsPerception Comprehension Projection

Philippines Warning and Situational Awareness System


CAP on a Map Project

8

Improving Institutional Responsiveness to Coastal Hazards through Multi- Agency Situational Awareness


Report and Blog: https://sahanafoundation.org/philippines-kickoff-workshop/

Awareness and Requirements

9

Discussed the “CAP on a Map” project objectives and agreed on a plan

Understood the current warning and response practices in the Philippines

Gathered preliminary requirements to begin the SAMBRO customization

Explored opportunities for Formulate a CAP Working Group (NDRRMC)

https://sahanafoundation.org/philippines-kickoff-workshop/

https://sahanafoundation.org/philippines-kickoff-workshop/


Current Warning Practices in the Philippines

10

NDRRMCOP Center

Reg- 1

Reg -2

Reg- 3

Reg 4A

Reg 4B

ARMM

CAR

NCR

provinces

FAX

cities/municipalities








Barangays(communities)Barangays(communities)Barangays(communities)Barangays(communities)Barangays(communities)Barangays(communities)Barangays(communities)

Barangays(communities)Barangays(communities)

provinces

provinces

provinces

17 RDRRMO

provinces

provinces

81 PDRRMO

1490 C/MDRRMO

42,028 BDRRMO

provincesFAX

FAX FAX SMS

Warning


SAMBRO Simplifying Warnings

11

● Publisher and a Subscriber messaging broker

● CAP 1.2 compliant system. ● Efficiency gains through reduced traversal

times● increased cost-effectiveness by

complementing current practices ● low-cost technology always on and ready to

use (integrated into the daily lives)

EOC - Emergency Operation Center RC - Red Cross (Society) Community - community of practiceAuthority - Alerting Authority Hub - other relay and rendering agents Media - TV, Radio, SocMedline Agencies emergency services - police/fire/SAR/health…


SAMBRO ALERT HUB

Proposed CAP-PH / SAMBRO WorkflowComplementing the existing warning procedures

12

PAGASA

PUBLISHER

PHIVOLCS

PUBLISHER

BULLETIN

Email

SMS

WEB

MOBILE APP

OTHER

OTHER

Free Mobile Alert Act


Two PAGASA and One PHIVOLCS Staff Members were trained on as CAP/SAMBRO Stewards on

● GIS techniques,

● Sahana software,

● CAP standard, policies, procedures

● SAMBRO warning and situational-awareness workflows

Training of Trainers

Report and Blog: https://sahanafoundation.org/tot-final-day/ 13

https://sahanafoundation.org/tot-final-day/


Training of National Trainers

14

PAGASA CAP/SAMBRO Stewards trained National Stakeholders to

1. Introduce CAP standard, policies, and procedures to their departments

2. Train a set of users for using SAMBRO

3. Assist NDRRMC Members with the implementation

4. Carry out Silent-Tests to ready the system


Report and Blog: https://sahanafoundation.org/philippines-simux/

Controlled-Exercises

15

A series of controlled-exercises were carried out with:

● PAGASA & PHIVOLCS staff to test the system and user readiness to operationalize SAMBRO

● other Stakeholders at the OCD to understand their perception of SAMBRO

● First-Responders in Manila Bay and Subic Bay (Olangapo) Barangay (Councils)

https://sahanafoundation.org/philippines-simux/

https://sahanafoundation.org/philippines-simux/


Evaluation Methodology

16

1. No Surprises, prior the the exercisesa. Implementation should be complete (terminology, classifications, templates)b. Users should have been trained for for originating / relaying messagesc. Siltet-test should have been carried out

2. During the exercisesa. Users defined a scenario, KPI, goals, intent, and actionsb. Discuss the steps for issuing with SAMBROc. Issue the alert/warning with SAMBRO

3. Evaluationa. Observers record the user's’ behaviour applying a complexity indexb. Record the behaviour with screen capture software (CamStudio)c. Users indicate the gulf of execution; i.e. “achieved level of the goal, intent, and actions”)d. Users indicate their perception on the technology acceptance (usefulness, ease-of-use,

On the day of the exercise did the technology and the people work?

Carried out with both Publishers and Subscribers


Dept Irrig & Dam Safety “relays” the message to their First-Responders

Exercise Workflows

17

Heavy RainThreatening floods and landslides

Dept Meteo & Hydro “originates” (issues) a “heavy rain” warning

Dept Irrig & Dam Safety Township focal person relays the message to other First-Responders

Detect Hazard Event

Alerting Authority Issue Alert

Response Organization Relay Alert

First- Responders Receive Alerts


1. Technical issues delayed the alert during the exercise

2. Most users were trained and competent but few were not

3. Users need to make an effort to understand the CAP standard

4. Compared to Myanmar similar pattern but PAGASA more seasoned on Cyclone work

Time To Completion

18


Observed Complexities

19

Workflow attempts show Flood and Heavy Rains showing complexities

Issues vary across Alert, Info, and Area segments

mostly in Area because of complexities in geocodes lookup

No predefined area based on risk / hazard maps

Average = 1.29 Standard Deviation = 0.83


Observed Complexities

20

5.0 = Extremely Simple

4.0 = Simple

3.0 = Moderate

2.0 = Difficult

1.0 = Extremely Difficult

Inconsistencies are in the Flood and Heavy Rain warnings

Average = 3.4 Standard Deviation = 1.48


Message Coding Errors

21

Common mistakes:

● Understanding the difference between a “test” and an “exercise”

● Using acronyms and specific country context names (alien to others) - headlines, senderName, description, address,

● Incomplete descriptions and instructions

● Uncertainties in using Severity, Certainty, and Urgency


Technology Acceptance Model Evaluation● Publishers :

○ 09 PAGASA ○ 10 PHIVOLCS

● Subscribers: ○ 10 Manila Bay (Metro Manila)○ 11 Subic Bay (Olangapo)

● All users are inclined towards: ○ AGREEING that SAMBRO is

easy to use and useful○ QUITE a POSITIVE attitude

towards using SAMBRO

SAMBRO Acceptance

22

Phili

ppin

es


Project Outcomes

23

1) PAGASA CAP “public alerting system” and SAMBRO interoperability was tested and proven to work well

2) Uptake or further use of SAMBRO by other Stakeholder:a) PHIVOLCS implementing their own instance of SAMBRO

b) OCD EOC has realized value and discussing an implementation strategy

c) First-Responders were not trained but quick to adopted SAMBRO


Recommendations

1. Expanding CAP and SAMBRO in Philippines

2. Formulating a Philippines CAP Working Group

3. Training and Certification of National Trainers

24


Real Need:

Realize the potential of CAP and adopt CAP as the all-hazard all-media National warning information exchange standard

Action plan: t = T Conduct a Stakeholder meeting, including members from

NDRRMC, to:● realize the potential of SAMBRO● share the knowledge from the “CAP on a Map”

project

t = T + 1 Develop a plan, in consultation with the relevant NDRRMC members to:● build CAP and SAMBRO capacity within those

Organizations● extend SAMBRO to PHIVOLCS, OCD, and any

other Stakeholder for early warning dissemination


25


Action plan: t = T + 3 Collaborate with the Stakeholder Organizations, to:

● Train the users in the respective organizations● gather Stakeholders’ information needs● iteratively, design, implement, and test (silent-tests)

t = T + 6 Conduct a set of controlled-exercises, with the Stakeholder Organizations, to evaluate their readiness to integrate SAMBRO into their warning practices

t = T + 12 Use the evidence gathered from the implementation, testing, and controlled-exercises to support the policy advocacy and policy development


26

Real Need:

Set a policy for Philippines to adopt CAP as the all-hazard all-media National warning information exchange standard


2. NDRRMC CAP-PH Working GroupAreas for Collaboration:1. Common terminology

a. Event Types, Incident Types, Codes, Parameters

2. Warning Classificationsa. PAGASA already has for Tropical Cyclone but needs develop for floodb. PHIVOLCS already has for Tsunami and Volcanic Eruptionc. Other Organizations may need to develop classifications

3. Description and Instructionsa. Different stages of an event required different description and

instructions NDRRMC should harmonize across all agencies

b. Relaying organizations (e.g. OCD) needs to augment the description and instructions

4. Alert Area Polygonsa. Predefined alert area polygons for flood, volcanic eruption, tsunami 27

Real Need:

Establishing common policies and procedures towards a CAP-PH-enabled interoperable and coherent warning practices in the Philippines for originating and relaying messages


2. CAP-PH Working GroupHow might NDRRMC CAP-PH Working Group involve other Stakeholders?

● Rappler project AGOS● GSM I’m Ready● Batingaw Mobile APP● eBanyanihan● DREAM project● SMART National Text Blast● e-Health & Public Health Alerting

28

Real Need:

Establishing common policies and procedures towards a CAP-PH-enabled interoperable and coherent warning practices in the Philippines for originating and relaying messages


Real Need:

Build capacity in Philippines to enhance interoperability and the exchange of disaster information through CAP-enabled all-hazard all-media and cross-agency situational-awareness approach

3. Training and Certification

Syllabus:

1. CAP warning standard, implementation, and operating procedures

2. Operationalizing and maintaining a SAMBRO Situational-awareness platform

29


Discussion Points

1. How do the NDRRMC members envision expanding CAP in the Philippines?● Are their plans to make CAP “the” warning standard in the Philippines?

● Are their plans to develop a CAP-PH policy, procedure, and implementation plans?

2. Should the other Stakeholders be using SAMBRO:● to “relay” (or re-originate) issued warnings to disseminate to their members? If yes,

what are the next steps?

● to serve as the Philippines Alert Hub at the OCD EOC for the media to subscribe to alerts?

3. Any other issues?30


Dissemination Workshop

Report and Blog:

31

Have we met the objectives?

● Make the Stakeholders, especially the decision makers, aware of the project achievements to date.

● Create an opportunity for the Stakeholders to discuss the uptake and integration of CAP and/or SAMBRO in Philippines


Thank You

32


Messaging Stages (or cap:status)

33

Originate AlertEvent

Originate Update

Originate Update

Originate All-ClearPA

GA

SAIrr

igat

ion

& D

am

Safe

tySAMBRO

SAMBRO

Relay Alert

Relay Update

Relay Update

Relay All-Clear

Originate Update

Originate Update

Originate Update


PAGASA CAP/SAMBRO Stewards trained National Stakeholders to

1. Introduce CAP standard, policies, and procedures to their departments

2. Train a set of users for using SAMBRO

3. Assist NDRRMC Members with the implementation

4. Carry out Silent-Tests to ready the system

Training of National Trainers

Report and Blog: https://sahanafoundation.org/ph-sambro-national-training/

34

https://sahanafoundation.org/ph-sambro-national-training/

https://sahanafoundation.org/ph-sambro-national-training/