lng model evaluation protocol & validation database update/media/bb51173392704bcfa6e... · lng...

LNG Model Evaluation Protocol & Validation Database Update

Presented by:

James Stewart

U.K. Health & Safety Executive and

Daniel Gorham

Fire Protection Research Foundation

2017 NFPA Conference & Expo

© British Crown Copyright, HSE 2017

• Session Learning Objectives

• LNG Facilities and NFPA 59A

• The LNG Model Evaluation Protocol (MEP)

- What is the LNG MEP and what is its purpose?

• The LNG Model Validation Database

• Updating the LNG MEP and Database

- Why have the LNG MEP and Database been updated?

- What changes have been made? - How does this affect the process of gaining approval for alternate vapor dispersion models for use in LNG siting applications?

• Reviewing Updated Versions of Approved Models

• Key Messages

Overview


Session Learning Objectives

The aims of this NFPA Conference & Expo 2017

Education Session are to:

• Provide information on how U.S. energy needs have lead to

increased attention for LNG facilities and NFPA 59A

• Explain the purpose of the LNG Model Evaluation Protocol

(MEP) and the LNG Model Validation Database

• Describe the updates and changes that have recently been

made to the LNG MEP and Database


Existing LNG Import/Export Terminals

Source: http://www.beaumontenterprise.com/news/article/10B-LNG-export-terminal-planned-for-Sabine-Pass-4506985.php

Source: https://https://en.wikipedia.org/wiki/LNG_carrier

Source: http://www.lngworldnews.com/ferc-updates-u-s-lng-terminals-summaries/


Proposed LNG Export Terminals

Source: Federal Energy Regulation Commission (FERC) https://www.ferc.gov/industries/gas/indus-act/lng.asp


History of NFPA 59A

AGA begins

work on LNG

1960

1964

Draft submitted

to NFPA as basis for

new standard

Standard prepared by Sectional Committee

on Utility Gas tentatively adopted

1966

1967

First official

edition of NFPA 59A adopted

Committee on LNG established to

develop standard with scope broader than

utility gas plant applications

1969

1971

First edition of NFPA 59A developed

under broadened

scope


Purpose

To provide minimum fire protection, safety, and related requirements for the location, design, construction, security, operation, and maintenance of LNG plants.

Source: http://soldoza.com/participation-on-lng-tanks/tk-3001-lng-tank-barcelona-port/


Applies to:

(1) Facilities that liquefy natural gas

(2) Facilities that store, vaporize, transfer, and handle LNG

(3) The training of all personnel involved with LNG

(4) The design, location, construction, maintenance, and operation of all LNG facilities

Does not apply to:

(1) Frozen ground containers

(2) Portable storage containers stored or used in buildings

(3) All LNG vehicular applications, including fueling of LNG vehicles (NFPA 52 and 30A)

Scope and Application


LNG Facilities and NFPA 59A

LNG Facility Siting Requirements

• NFPA 59A requires that any LNG container or transfer system

must have a dispersion exclusion zone

• The size of such zones must be determined using an

approved vapor dispersion model, one of:

• 49 CFR 193 Subpart B, states that alternative models may be

used, subject to the approval of the “Administrator”

• In the case of LNG facilities the administrator is the Pipelines

and Hazardous Materials Safety Administration (PHMSA)

DEGADIS 2.1

FEM3A

PHAST v6.6/6.7

FLACS v9.1 r2


LNG Facilities and NFPA 59A

LNG Facility Siting Requirements

• PHMSA state that alternative models may be used in LNG

siting applications provided that the models “…take into

account the same physical factors as the approved

models, are validated by experimental test data, and

receive the Administrator’s approval…”

• The LNG Model Evaluation Protocol should be used as a

means of evaluating alternative models

• The LNG MEP, and the accompanying LNG Model Validation

Database, have recently been updated to take account of

additional requirements imposed by PHMSA


LNG Model Evaluation Protocol

What is the LNG Model Evaluation Protocol (MEP)?

• The LNG MEP is a document outlining the process that

should be used to assess the suitability of vapor dispersion

models for evaluating hazard distances in LNG siting

applications

What is the purpose of the LNG MEP?

• The purpose of the LNG MEP is to provide a comprehensive

evaluation methodology for determining the suitability of

models to accurately simulate the dispersion of vapors

emanating from accidental spills of LNG



What does the LNG MEP involve?

• The LNG MEP comprises three distinct phases:

Scientific Assessment

Model Verification

Model Validation

• Each stage must be completed during an evaluation of a model

against the LNG MEP

• The outputs are then recorded in a Model Evaluation Report

• This process must be followed when seeking approval to use

an alternate vapor dispersion model for LNG siting applications



LNG MEP: Scientific Assessment

• Involves critically reviewing the physical, mathematical and

numerical basis of a model by someone independent of the

model development

Key details of the model should be available for scientific assessment

Model produces output suitable for assessment against MEP SPM

Model should be based on accepted/published science

Model accepts a credible source term

Model accounts for the effects of wind speed

Model accounts for the effects of surface roughness on dispersion

Model accounts for the effects of atmospheric stability on dispersion

Model accounts for passive dispersion

Model accounts of gravity-driven spreading

Model accounts for the effects of buoyancy on dilution

Numerical methods are based on accepted/published practice practice

Model is suitable for

evaluation against MEP

Model contains necessary

physics with appropriate

mathematical

implementation

Model is based on suitable

numerical methods



LNG MEP: Model Verification

• Model verification is the process of checking that a computer

implementation of the model accurately represents its

mathematical basis

• In the LNG MEP, verification is treated as a passive element of

the scientific assessment, rather than an exercise in its own

right

• Evidence of model verification should be provided to

demonstrate that the model being evaluated has been suitably

verified

• PHMSA makes the ultimate decision about model suitability

and limitations



LNG MEP: Model Validation

• Model validation is the most significant stage of the LNG MEP

• Scientific assessment and model verification give an indication

of model suitability, whereas validation assesses how well a

model actually performs for scenarios of practical interest

• Model validation for the LNG MEP involves comparing model

predictions to experimental data for a range of dense gas

dispersion scenarios

• The experiments included in the model validation database

were selected to test the key physical processes involved in

the dispersion of LNG


LNG Model Validation Database


• The Database is an Excel spreadsheet used

during model validation for the LNG MEP

• It contains data from 33 experiments taken

from 8 different series of tests

• Model predictions of gas concentrations at

specified locations are entered into the

Database for each experiment

• Max. arc-wise concentrations and Statistical

Performance Measures (SPM) are then

automatically calculated by the spreadsheet

and then used for quantitative assessment

of the model




• Details of each experiment are

given in separate Excel worksheets

• These details comprise:

- Basic test information

- Details of the substance released

- Details of the conditions of release

- Atmospheric conditions

• The Database also lists sensor

locations and averaging

times/windows for each experiment




• The Database produces graphs of predicted vs. measured

maximum arc-wise gas concentration for each experiment

• Two of the worksheets give tabulated values of SPM, color-

coded to show whether or not the suggested quantitative

performance measures are met, e.g. for Maplin Sands trial 27




• The final worksheet of the Database contains scatter plots of

predicted vs. measured concentrations to give a visual

representation of model performance

• Five such plots are produced

to show model performance

for:

- All Trials

- Field Trials

- Wind Tunnel Trials

- Unobstructed Trials

- Obstructed Trials


LNG Validation Database Guide

Guide to the LNG Model Validation Database Version 12

• The Database is described in:

Stewart et al. (2016), Guide to the Model Validation Database Version 12


LNG Validation Database Guide

Guide to the LNG Model

Validation Database Version 12

• The Database guide describes the

Model Validation Database and

how it should be used as part of

the LNG MEP

• It gives details of each series of

experiments and describes the

experimental uncertainties

• The methods used in the Excel

spreadsheet to automatically

calculate the SPM are also

outlined


Updating the LNG MEP & Database

Aims of the update:

• To add new features to the Database and amend the MEP to

incorporate the additional requirements given in the PHMSA

Advisory Bulletin PHMSA-2010-0226

• To incorporate previously-omitted experimental data and to

correct errors within the Database

• To automate the calculation of SPMs and maximum arc-wise

gas concentrations in the Database to reduce user effort

during model validation

• To provide more extensive documentation and guidance on

using the Database and MEP



Deliverables from the LNG MEP & Database Update

• The 2nd Edition of the LNG Model Evaluation Protocol

M. J. Ivings et al. (2016), Evaluating Vapor Dispersion Models for Safety

Analysis of LNG Facilities, 2nd edition

• The LNG Model Validation Database version 12

• An updated guide to the LNG Model Validation Database

J. R. Stewart et al. (2016), Guide to the LNG Model Validation Database Version 12

• The two reports are available from the NFPA research reports

webpage1 and the Database is available as an Excel spreadsheet

on request

1 http://www.nfpa.org/news-and-research/fire-statistics-and-reports/research-reports/hazardous- materials/gases/lng-model-evaluation-protocol-and-validation-database-update



Requirements of the PHMSA Advisory Bulletin

PHMSA-2010-0226

• Maximum arc-wise gas concentrations

- Method for determining these is now prescribed

• Experimental and modeling uncertainty

- Discussion of experimental uncertainty incorporated into the Database Guide

• Additional SPM:

- Concentration Safety Factor (CSF)

- Concentration Safety Factor to LFL (CSFLFL)

- Distance Safety Factor to LFL (DSFLFL)



Maximum Arc-wise Gas Concentrations

• Comparison of measured and predicted maximum arc-wise

gas concentrations forms a key part of the LNG MEP model

validation exercise

• Ensuring all models are compared in a like-for-like manner is

fundamental for consistent model evaluation

• PHMSA has prescribed the method which should be used

when applying for approval to use an alternate vapor

dispersion model for use in LNG siting applications



Maximum Arc-wise Gas Concentrations

“The maximum arc wise concentration should be based on

the location of the experimental sensor data that produced

the maximum arc wise concentration relative to the cloud

centerline” (PHMSA-2010-0226)”



Maximum Arc-wise Gas Concentrations:

What’s the difference?

• Difference between the PHMSA-prescribed method and the

common1 alternative is subtle but potentially significant

1 The method often used is that of Hanna et al. (1993) where the max. arc-wise concentration at a given downwind distance is

taken as the max. at any position along the arc at the height of the lowest experimental sensor

Predicted max. arc-wise concentration Predicted max. arc-wise concentration Cloud centerline

Cloud centerline

Sensors

Gas source

‘Common’ Method PHMSA Method


Maximum Arc-wise Gas Concentrations:

What are the benefits of the PHMSA approach?


• The method is conservative, it

will typically result in larger

exclusion zones for LNG sites

• It accounts for cases where the

plume bypasses the gas

sensors

• It encourages the development

of plume meandering models


Experimental and Modeling Uncertainty

• Following guidance given in the PHMSA Advisory Bulletin

PHMSA-2010-0226, sensitivity analyses must be undertaken

during the model validation stage of the LNG MEP

• The updated Database Guide gives an overview of the

experimental and modeling uncertainties for each test series

included in the Database

• These uncertainties are based on the review of the DEGADIS

model by FERC and on experimental analysis reports

• This information should be used to guide sensitivity analysis

studies conducted as part of the LNG MEP process



Additional Statistical Performance Measures (SPM)

• The Advisory Bulletin PHMSA-2010-0226 requires calculation

of three additional SPM from the model validation data:

- Concentration Safety Factor (CSF): 𝐶𝑆𝐹 =𝑐𝑝

𝑐𝑚

- Concentration Safety Factor to LFL (CSFLFL): 𝐶𝑆𝐹𝐿𝐹𝐿 =𝑐𝑝

𝐿𝐹𝐿

- Distance Safety Factor to LFL (DSFLFL): 𝐷𝑆𝐹𝐿𝐹𝐿 =𝑋𝑝,𝐿𝐹𝐿

𝑋𝑚,𝐿𝐹𝐿

• These parameters are automatically calculated within the

Database to ensure that a consistent approach is used to

evaluate all models




Additional Experimental Data

• Time-varying concentration data has been processed to produce

point-wise concentrations for the Maplin Sands (1980) trials

© Shell




• Point-wise concentration data has been added for the Thorney

Island (1982) experiments




• Point-wise concentrations at

multiple sensor heights

• Highlighting of sensor locations

where negligible concentrations

were measured

• Negligible (below 0.1 % v/v)

concentrations excluded from

SPM calculations

e.g. Thorney Island 45



Corrections to the Database

• Adjustment of sensor height for

one of the downwind

measurement locations for

CHRC Trials B and C

• Height used in earlier version of

Database coincided with

location of the dike

CHRC A, Havens and Spicer (2006)



Corrections to the Database

• Corrected ambient and release conditions to be consistent with

original experimental reports

• Adjusted surface roughness height to coincide with inflow

profiles from experiment

• Corrected geometry sizes and positions for obstructed scenarios



Further Database Changes

• Visual representation of concentration sensor locations

• Transformed sensor locations for use with models in which the

wind is aligned with the grid



Further Database

Changes

• Cloud widths are calculated

in cases where the gas

cloud is not bifurcated in the

cross-wind direction

• Plots (see right) are created

for the Database user to

assess cloud shape

• The user then indicates

whether or not the cloud is

bifurcated

Long Time-averaged Predicted Arc-wise Concentration Profiles (at 1 m elevation)

Note that only concentration values at the lowest sensor height are shown here and are included in any cloud

width calculations

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

10.0

-50.0 -25.0 0.0 25.0 50.0

Co

nce

ntr

atio

n (%

)

Cross-wind distance (m)

57 m Arc Measured Predicted

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

-100 -50 0 50 100

Co

nce

ntr

atio

n (%

)



0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

-200 -100 0 100 200

Co

nce

ntr

atio

n (%

)



0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.0 50.0 100.0 150.0 200.0

Co

nce

ntr

atio

n (%

)




Updated Versions of Models

Reviewing Updated Versions of Approved Models

• A new process has been proposed for the expedited review

of updated versions of models already approved by PHMSA

• The approval of a model for use in LNG siting applications is

only valid for the exact version of the model evaluated

• However, models are frequently updated to fix errors or to

include additional features or model improvements

• PHMSA intend to provide the option of an expedited review

for updated versions of approved models so as to avoid

unduly impeding the use of new code developments



Reviewing Updated Versions of Approved Models

• The new process is based on the comparison of the updated

model to the previously-approved version

• The extent to which the updated model is assessed is based

on whether the changes are deemed to be ‘major’ or ‘minor’

• Here, a ‘minor’ change is meant to address four issues:

- Bug fixes in the software

- Changes to the Graphical User Interface (GUI)

- Addition of compatible hardware support

- Modification of sub-models not relevant to LNG dispersion

• A ‘major’ change makes advances to the scientific basis of

the model for modelling LNG dispersion



Reviewing Impact of Model Updates: Minor Changes

• Assessing an updated model that has undergone only minor

changes requires the model to be run against a subset of the

LNG validation database experiments

• This subset includes the following 6 tests:

Burro 8 Thorney Island 47

Coyote 5 CHRC B

Falcon 1 BA Hamburg DAT223

• Point-wise concentration predictions for these tests are then

compared to the results from the approved version of the model

• The two versions of the model should use identical setups for

each of the tests




• The next step is to determine if any changes in model predictions

are ‘significant’

• In this context a change in predicted concentration is considered

to be significant if there is:

A 1% relative difference in predictions from the two versions of the

model where the absolute predicted concentration is > 𝟏𝟎% v/v

Or

A 0.1% v/v absolute difference in predictions from the two versions of

the model where the absolute predicted concentration is ≤ 𝟏𝟎% v/v




• If there are no significant differences between the model

predictions for the validation subset then the following items

should be submitted to PHMSA as a record of the review:

- A short report to describe the changes to the model since

the previous version was approved

- Point-wise concentration predictions for the subset of

validation test cases

- Model input files for the subset of validation test cases

• If there are significant differences as a result of model changes,

the ‘minor’ changes must be treated as ‘major’ changes and the

model should be reviewed accordingly



Reviewing Impact of Model Updates: Major Changes

• For models that have been updated to include a major change,

e.g. to a LNG dispersion related sub-model, the process for

evaluating the updated version of the model is more rigorous

• In this scenario the model must be evaluated against the three

stage LNG MEP process, i.e.

Scientific Assessment – to assess the impact of the model updates

Model Verification – to ensure that the model is verified to an equal or

higher standard than the previous version of the model

Model Validation – against the entire LNG model validation database with

significant differences in model predictions highlighted

and explained



Reviewing Impact of Model Updates: Major Changes

• On completion of an evaluation of an updated model with major

changes from the approved version, the following should be

submitted to PHMSA:

- A change-log report to describe changes to the model since it was

previously approved by PHMSA. This report should include results

of the scientific assessment, verification and validation of the

updated model in accordance with the LNG MEP

- Model input files for the LNG MEP model validation exercise

- Access to the updated and currently-approved versions of the

model



Major or Minor update?

Run subset of validation

cases

Are any differences significant?

Evaluation against full LNG MEP

Submit following to PHMSA: 1. Change-log report 2. Model input files 3. Access to new and currently

approved versions of model

Submit following to PHMSA: 1. Short report 2. Point-wise concentration

predictions 3. Model input files

Major Minor

No

Yes


Key Messages

Summary

• Updated versions of the LNG Model Evaluation Protocol

(MEP), Model Validation Database and Database Guide are

available from the NFPA

• When seeking approval to use alternative vapor dispersion

models for LNG siting applications from PHMSA, the latest

version of the MEP and Database must be used

• A streamlined process to gain approval of an updated

version of a model currently approved for determining LNG

facility dispersion exclusion zones has been defined

• PHMSA are currently working towards using this new

approach


References

• PHMSA Advisory Bulletin, PHMSA-2010-0226 http://phmsa.dot.gov/pv_obj_cache/pv_obj_id_B1E12F1E74C27BEAB343DEB90D621DF5BB340

700/filename/ADB-10-07%20LNG%20Facilities.pdf

• DEGADIS 2.1 evaluation for LNG MEP https://www.ferc.gov/industries/gas/indus-act/lng/degadis-report.pdf

• PHAST v6.6 and v6.7 evaluation for LNG MEP http://www.regulations.gov/#!docketDetail;D=PHMSA-2011-0075

• FLACS 9.1r2 evaluation for LNG MEP https://www.regulations.gov/#!docketDetail;D=PHMSA-2011-0101

• Hanna et al. (1993) http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA270095

• Havens J. and Spicer T. (2006) Vapor dispersion and thermal hazard

modelling, Final topical report to Gas Technology Institute under sub-contract

K100029184, October 2006.


HSE's Health and Safety Laboratory (HSL) is a multi-disciplinary laboratory whose work delivers high quality science to meet the needs of industry and government in the UK and overseas. Our commercial customers can commission services and research using our state-of-the-art scientific laboratory in Buxton, U.K. as well as analytical expertise from other parts of HSE’s science base.

http://www.hse.gov.uk/

http://www.hsl.gov.uk/

This presentation and the work it describes were undertaken by the Health and Safety Laboratory under contract to Oak Ridge National Laboratory. Its contents, including any opinions and/or conclusion expressed or recommendations made, do not necessarily reflect policy or views of the Health and Safety Executive.

CEUs: To receive CEUs for this session, scan your badge at the back of the room before leaving Evaluation: Complete a session evaluation on the mobile app. (Search app store for ‘NFPA 2017 C&E.’) Handouts: Handouts will be available via the mobile app and at nfpa.org/conference Recordings: Audio recordings of all sessions will be available free of charge via NFPA Xchange.

2017 NFPA Conference & Expo

lng model evaluation protocol & validation database update/media/bb51173392704bcfa6e... · lng...

Documents