exor corporation limited · 2011-11-24 · 4 nov 2010 ah product options bypassctl, bypasshac have...

Exor Corporation Limited r

The Global Leader in

Infrastructure Asset Management

Street Gazetteer Manager System Admin V4.5

Exor Leading the way in Infrastructure Asset Management Solutions...

© Copyright Exor Corporation Ltd, England, 2008 All rights reserved

This page intentionally blank


© Copyright Exor Corporation Ltd., England, 2008. All rights reserved

i

Quality Assurance Statement

Document Details File:

Street Gazetteer Manager System Admin V4.5.doc

Prepared by: T.C. Stewart

Manual Name: Street Gazetteer Manager System Admin V4.5

Reviewed by: P.Rogers

Version: 1.0

Approved for issue by: T.C Stewart

Date of Issue: Nov 2010

Product Manager: P.Rogers

File Name: Document Centre/Exor/Product Manuals/4.5.0.0 Product Manuals/4.5.0.0 completed

documentation/Street Gazetteer Manager Admin v4.5.docx

Document Version Control Revision Date By Chapter

/ Page Description

1 30-Jan-2008 CS 6 The following new or previously undocumented Product Options have been added: ESULIMIT, FORCEVERS, LOADHIST, NSGEXPACC,NSGEXPHIST, NSGSPECIAL,STRLIMIT,XREFLIMIT

1 30-Jan-08 CS 6 The following product Option has been removed: NSGNUMERIC. It has been superseded by NSGSPECIAL

1 30-Jan-08 CS 6 Highway Authority Code removed from NSG0060

1 31-Jan-08 CS New Modules added

16 NSG0070 – Maintain Highway Authorities

18 NSG0080 – Maintain Naming Authorities

19 NSG0090 - Configure ASD Record Type

20 NSG0110 – Organisations and Districts

2 Sep-2008 CS Released with 4.0.5

3 Jan 2010 AH All Released with 4.2

4 Nov 2010 AH Product Options BYPASSCTL, BYPASSHAC have been added.

4 Nov 2010 AH NSG0040 – NSG Loader Batches

4 Nov 2010 AH NSG0045 – Monitor NSG Loader

5 May 2011 IS None No changes for v4.4

6 Nov 2011 IS P33 NSG metadata refresh note added



ii

Contents Quality Assurance Statement ......................................................................................................... i Product Options .............................................................................................................................. 7 This value is shipped by Exor and must not be changed. ........................................................... 9 Adding New Towns and Localities – NSG0060 ........................................................................... 12

Language ................................................................................................................... 13 Counties ..................................................................................................................... 13 Towns ......................................................................................................................... 14 Localities .................................................................................................................... 14

Maintaining Highway Authorities – NSG0070 ............................................................................. 16 Highways Authority ..................................................................................................... 16 County ........................................................................................................................ 17 Refresh ....................................................................................................................... 17

Maintaining Naming Authorities – NSG0080 ............................................................................... 18 Configure ASD Record Types – NSG0090 .................................................................................. 19

Classification .............................................................................................................. 19 Record Type ............................................................................................................... 19

Maintaining Organisations and District Information – NSG0110 ............................................... 20 EToN .......................................................................................................................... 21 FPN (Fixed Penalty Notice) Contact ........................................................................... 21 FPN Payment ............................................................................................................. 22 Validation Warnings .................................................................................................... 23

Reset Street Coordinates – NSG0015 .......................................................................................... 24 Generate ASD Placements – NSG0025........................................................................................ 25

Common Errors .......................................................................................................... 26 Importing NSG and ASD Data ...................................................................................................... 27 NSG Load Batches - NSG0040 ..................................................................................................... 28

Batches tab ................................................................................................................ 29 Files in Batch tab ........................................................................................................ 32 Batch Schedule tab .................................................................................................... 32 Records tab ................................................................................................................ 35 Batch Log tab ............................................................................................................. 36 Floating Toolbar .......................................................................................................... 38

Monitor NSG Loader - NSG0045 .................................................................................................. 41 Administer User Districts – NSG0120 ......................................................................................... 45

General Information .................................................................................................... 45 User Districts .............................................................................................................. 46

My Districts - NSG0130 ................................................................................................................. 47 General Information .................................................................................................... 47 User Districts .............................................................................................................. 47



iii



4

CHAPTER

1 Introduction

Exor Street Gazetteer Manager is a BS7666 compliant application for creating and maintaining a level 3 Street Gazetteer and Associated Street Data (ASD). Developed using Oracle Spatial technology, it provides an open and interoperable database solution that can be accessed by any other System in use across the Organization. Street Gazetteer Manager provides an easy to use, map driven interface that provides an Authority with industry standard, accurate methods of creating and managing Street and Associated Street Data which underpins Exor’s solution for the Traffic Management Act. Under the TMA, one of the key responsibilities of the Network Manager is to coordinate works and other activities occurring on the Network. Accurate, easily understood Associated Street Data is essential to this task. Street Gazetteer Manager assists in this process by automatically creating a ‘linear event’ representing the location of the ASD record. This can then be viewed on the map without the need to ‘hand digitise’ the ASD record manually. All changes to Street information and Associated Street Data are audited and ‘closed’ Streets and ASD are retained for historical purposes. An ‘effective date’ may be set which allows the User to view the Street Gazetteer and any Associated Street Data that were open at the specified effective date. This is essential for dispute resolution when an undertaker may be subject to a Fixed Penalty Notice for an invalid submission. Street Gazetteer Manager is suitable for all scales of implementation and can be deployed on a single Desktop with the data being available to the entire organization via Exor’s web mapping functionality. Street Gazetteer Manager has been developed as an extension to Spatial Manager which is based upon ESRI® ArcGIS® 9.x - ArcMap application, utilizing the advanced graphic editing functionality and user interface that ArcMap offers, including many tools to manage the way that information is displayed on maps in the on-screen map view and in printed map layouts. To find out more about this functionality see ArcMap documentation. This document describes the functions and options available to the Systems Administrator to assist in the maintenance and definition of the Local Street Gazetteer. The options described are normally carried out by the Systems Administrator and not the end user, however this may vary from site to site.



5

Refer to the Street Gazetteer User Guide for information on the how to manage and maintain your LSG.



6

CHAPTER

2 Using Street Gazetteer Manager

The following functions are included within this document:

SGM related Product Options

Adding new Towns and Localities – NSG0060

Maintaining Highway Authorities – NSG0070

Maintaining Naming Authorities – NSG0080

Configure ASD Record Types – NSG0090

Maintaining Organisation and District Information – NSG0110

Reset Street Coordinates – NSG0015

Generate ASD Placements – NSG0025

NSG Load Batches – NSG0040

Monitor NSG Loader – NSG0045

Administer User Districts – NSG01020

My User Districts – NSG0130

Note Before using Street Gazetteer Manager ensure that all necessary Product Options are set to an appropriate value for your Organisation. Product Option Details can be found on page 6.



7

Product Options

Figure 1 Product Options Menu

Product Options allow ‘switches’ or default values to be set, which defines how certain functionality operates within the Exor suite of Products. The following Street Gazetteer Manager related Product Options should be set to suit your Organisational requirements. Product Options are set using HIG9130 – Product Options. Product Options marked with a ‘*’ are also User Options (see the Street Gazetteer Manager User Guide for more information).

Product Code

Option Id Option Name Sample Value

Remarks

HIG NSGDATA System uses NSG data

Y This MUST be set to Y if the Organization is a Highway Authority who maintains their LSG within Exor.

NSG BYPASSCTL Bypass CTL refresh

N Bypass refresh of County/Town/Locality during the load of NSG data

NSG BYPASSHAC Bypass HA County refresh

N Bypass refresh of Highway Authority counties during a load of NSG data.

NSG CHECKODXML Check OD XML Content on Export

Y If set to Y when OD XML files are produced they will be checked for compliancy against EtoN Organisation/ District XSD



8

Product Code


Remarks

NSG CREATE_CTL Create counties, towns and loc

Y This option works in conjunction with the NSG Loader. If set to Y any New Counties, Towns or Localities detected during a data load will be automatically created. Counties, Towns and Localities can be viewed and maintained using NSG0060 (see page 6).

NSG DFLTDMODE Default Loader Mode

FULL Default mode to use when submitting NSG Loader batches

NSG DFLTSUBMIT Default Loader Submission Time

18:00 Default time to use when submitting NSG Loader batches HH24:MM

NSG* DFLT_AU Default Admin Unit Code

DCC This is the Admin Unit Code assigned to a Street. This is not used with BS7666 , but is used as part of the overall Exor security mechanism to restrict access to certain Streets to certain Users.

NSG* DFLT_CNTY Default County DORSET This is the default value for the ‘County Name’ field. The ‘Highway Authority’ code will be derived from this.

NSG* DFLT_NAUTH Default Naming Authority Code

1235 This is the default value for the ‘Naming Authority’.

NSG* DFLT_TOWN Default Town TURTON This is the default value for the Street Town name. The List of Towns will only contain valid Towns within the selected ‘County’. (See page 6 for details on maintaining a List of Towns).

NSG* DFLT_TRNCE Default Tolerance

10 This is the default value for the Street Tolerance (in metres).

NSG* DFLT_TYPE Default Record Type

1 This is the default NSG Record Type. The available options are 1,2,3 or 4.



9

Product Code


Remarks

NSG DFTVERSION DFT spec version number

5.3 This value is shipped by Exor and must not be changed.

NSG FORCEVER Force Version for Export

Y This option is used when exporting the LSG for submission to the NSG Hub. If set to ‘Y’, any Streets where new ESU’s have been added or ESU’s have been removed since the date of the last Street version will be automatically versioned.

NSG LOADSTATS Gather Stats as data is loaded

Y If set to Y the system will gather statistics on tables as data is loaded i.e. with the aim of improving performances of the load process

NSG NSGDISP NSG ESU Displacement value

1 This value is used when the ESU mid point is being calculated to ensure uniqueness. It should only be changed if instructed to do so by Exor support staff.

NSG* NSGDISTREF NSG Extract District Ref

001 This is the District ref used when creating the Organisation and District extract file for submission to the Hub. This value should not be changed.

NSG NSGEXPACC NSG Export Accuracy

3 This value determines the number of decimal places used for Street, ESU and ASD coordinates. When exporting the LSG to the NSG Hub this must be set to ‘0’.

NSG NSGEXPSIZE NSG Export Record Size

500 This value is used in conjunction with the NSG exporter. It is the number of records retrieved per fetch. This value should only be changed if instructed to do so by Exor support staff.



10

Product Code


Remarks

NSG NSGLANG1 Primary Language

ENG Primary language to attribute to descriptions LG submissions/to attribute to streets when loading gazetteer files.

NSG NSGORGREF NSG Extract Organisation Ref

1245 This is the Organisation ref used when creating the Organisation and District extract file for submission to the Hub.

NSG NSGSPECIAL Allow special characters

N If this is set to ‘Y’, characters defined within the ‘NSGSPECIAL’ character set may be used within the Street record. This should only include Characters other than those defined by the Custodian for Character Set 2. Special Characters are maintained using HIG9150 – Maintain Character Sets. Refer to the General System Admin Guide for further details.

NSG USEALIASES Aliases are in use?

N This flag should only be set to ‘Y’ for Welsh Authorities. It is used to enable the use of Street Aliases

NSG VERSIONING Versioning in use

Y This option must be set to Y if the Organisation maintains its LSG data within Exor, i.e. a Highway Authority. For Utilities Customers this should be set to N.

NSG NSGIMPMSG NSG Import message Level

S This option should be set to D or S. If set to 'D' - all messages that the NSG loader produces will be logged If set to 'S' - just the key messages that the NSG loader produces will be logged.



11

Product Code


Remarks

NSG ESULIMIT Limit for ESU loading

5000 This value is used in conjunction with the ‘NSG File Loader’. It determines the commit points during a load. The default value should not be amended unless recommended by Exor Support.

NSG LOADHIST Load historical data

Y If this value is set to Y, then all versions of Streets and ESU’s will be processed when loading data using the ‘NSG File Loader’. If set to ‘N’ only the most recent versions are processed. This should be set to ‘Y’ after initial data take on.

NSG NSGEXPHIST Export Full NSG History

N If set to ‘Y’ all versions of Streets and ESU’s are exported. When exporting the LSG to the NSG Hub for all TMA compliant submissions, this must be set to ‘N’

NSG STRLIMIT Limit for Street loading

5000 These values are used in conjunction with the ‘NSG File Loader’. It determines the commit points during a load. The default value should not be amended unless recommended by Exor Support.

NSG XREFLIMIT Limit for XREF loading

10000



12

Adding New Towns and Localities – NSG0060

Figure 2

Street Gazetteer Manager provides validation when creating new Streets to ensure that valid County, Town and Locality values are added. These ‘locations’ exist within a hierarchical order with the County (Highway Authority) being at the top of the hierarchy. Each County will have many Towns and each Town may have many Localities. When adding these details for a Street, only those Towns that exist within the selected County may be chosen. Similarly, only Localities that exist within the selected Town may be chosen. This ensures that data integrity and quality is maintained. The initial hierarchy of County/Town and Localities is established from existing Street Data when Street Gazetteer is implemented. If new Gazetteers are loaded using the Street Gazetteer Loaders the new hierarchies can be created and existing hierarchies maintained automatically if Product Option CREATE_CTL is set to ‘Y’.

Note: this option would normally be set to ‘N’ for non Highway Authority customers.

Locations are maintained within the main Exor Application using the Maintain Locations – NSG0060 module.

Figure 3



13

The Locations module is made up of 4 separate areas, one each for maintaining Language, Counties, Towns and Localities and. Any changes made in this form are immediately available with the Street Gazetteer form. The multi language functionality allows counties/towns and localities to be grouped by language code.

Figure 4

Language

The Language panel will display the available languages. These are set up using Domains – HIG9120 under the option NSG_STREET_LANG. Name Display Only

The code for the language will be displayed. Language Display Only

A description of the language will be displayed.

Figure 5

Counties

This panel is used to maintain the List of Counties managed within Street Gazetteer Manager. Name Required

This is the Name of the County that will appear in the Highway Authority Description field and the County field within the Street Gazetteer form.



14

Figure 6

Towns

This panel is used to add and maintain a valid list of Towns within the selected County (Highway Authority). To add a new Town ensure the cursor focus is set within the Town panel and press [Create Record] button on the menu toolbar. Town

This is the Town Name. A maximum of 30 characters is allowed

Figure 7

Localities

This panel is used to add and maintain a valid list of Localities within the selected Town. To add a new Locality ensure the correct Town is selected and that the cursor focus is set within the Locality panel. Press [Create



15

Record] button on the menu toolbar. Locality

This is the Town Name. A maximum of 35 characters is allowed

Figure 8

The [Regenerate Data] button when pressed looks at all of the streets that exist and the attributes of the street and builds up a definitive list of counties/towns/localities.



16

Maintaining Highway Authorities – NSG0070

Figure 9

Street Gazetteer Manager is shipped with a complete set of UK Highways Authorities and Codes and should not normally need to be amended. Each Authority may be associated with one or more ‘Counties’. Under normal circumstances this should be a one to one relationship, however, several different spellings of the same ‘COUNTY’ value may exist within some Local Street Gazetteers. During the loader stage new ‘Counties’ are automatically created (only if Product Option CREATE_CTL is set to Y) if they do not already exist within Street Gazetteer Manager. This facility prevents failure of the loading process if the data is of poor quality. Highways Authorities are maintained using NSG0070 – Maintain Highways Authorities. To ensure that any new ‘Counties’ create during a ‘Gazetteer Refresh’ are correctly associated with their appropriate Highway Authority the County List should be refreshed using the [Refresh] button at the bottom of the form.

Highways Authority

The top block displays the Highway Authority Code and Name. A Flag may also be set to determine if the Authority Name is displayed in the relevant List of values in the main Streets Form.



17

Code Required

This is the Code for the Highway Authority Name Required

This is the Highway Authority Name. Visible Checkbox

If the Visible flag is set, the Authority will be included in the relevant List of values within the Streets form both with Spatial Manager and the Forms module. Authorities not flagged as Visible are still valid and may be queried, but will not appear within the LOV. Figure 10 shows an example where 2 Authorities have been set as Visible and the ‘Highway Authority’ List of Values within the Streets Form.

Figure 10

Figure 11

County

This panel allows a ‘County’ to be associated with a Highway Authority. This should normally be the same as the Highway Authority but in some cases several spellings of the same ‘County’ may exist with a Local Street Gazetteer that has been loaded from a 3rd party application. Where this is the case a flag may be set to define the ‘Preferred’ County. Only the preferred value will be displayed within the List of Values for the County attribute within the main Streets Form.

Refresh

To ensure that any new ‘Counties’ create during a ‘Gazetteer Refresh’ are correctly associated with their appropriate Highway Authority the County List should be refreshed using the [Refresh] button at the bottom of the form.



18

Maintaining Naming Authorities – NSG0080

Figure 12

A full set of Naming Authorities along with their respective Codes and USRN Prefixes is shipped as standard within Street Gazetteer Manager and should not normally need to be maintained. If however changes are required, this is done using NSG0080 – Maintain Naming Authorities.

Figure 13

The List of Values for the Naming Authority field within the Street Form can be restricted to those Naming Authorities used within a Highways Authority by checking the ‘Visible’ flag for only those values required. If a change is made within the form press the [Save] button on the menu toolbar. Figure 14 shows the Naming Authority LOV for the ‘values flagged as Visible above.

Figure 14



19

Configure ASD Record Types – NSG0090

Figure 15

Street Gazetteer Manager is designed to enable the management of ASD data in England/Wales, Scotland or both simultaneously. The Description of the ASD record Types, displayed within all relevant screens, can be defined as required. ASD classifications and Record Type descriptions are defined using NSG0090 – Configure ASD.

Figure 16

Classification

This panel is used to indicate the ASD regime(s) required. Customers working in the proximity to the England/Scotland border may require both regimes to be in place.

Record Type

The Description for an ASD record type can be amended as required. The value entered is used within all relevant screens when displaying ASD records. Figure 17 shows an example where both the English/Welsh and Scottish ASD regimes are in use.

Figure 17



20

Maintaining Organisations and District Information – NSG0110

Figure 18

The Organisations and Districts module is used to identify all organisations that are involved within the street works arena. This includes Utilities, Highways Authorities and Promoters. This form is pre-populated by the loading of an organisations OD XML file (Operational District File). Organisations can be split into as many districts as required to simplify the day to day running of their business.

Figure 19

Note that the District Works Ref prefix column is only displayed if the TMA product is licenced.



21

On the right of the form there are a number of tabs with details of Eton requirements, FPN Contact details, FPN Delivery Address and FPN Payment Details.

Figure 20

EToN

EtoN (Electronic transfer of notices) details are used to send and receive files electronically. These details get transmitted within the OD files to other Organisations to allow this electronic communication.

Figure 21

FPN (Fixed Penalty Notice) Contact

These are the contact details of the person issuing the FPN’s. An undertaker may need to contact this person in the event of Fines being issues. These details also get sent in the OD file.



22

Figure 22

This is the Delivery address that the fines will go to. This can be either a postal address or a URL address.

Figure 23

FPN Payment

The FPN Payment panel contains the postal address information and or the bank details that the fines will be paid into. The Organisation and District details can be filled in manually to enable the production of your own OD batch file in either XML or the old standard text file. These may be produced using either the [Produce OD Batch File] or [Produce OD XML File] depending on the post or pre TMA regime in use. These files are also produced when the LSG is extracted for submission to the National Hub.



23

Figure 24

Validation Warnings

If for any reason any Mandatory attributes values are missing the [Validation Warnings] button will be made available. The number of warnings will also be displayed. To view the list of issues press the button.



24

Reset Street Coordinates – NSG0015

Figure 25

The Reset Street Coordinates – NSG0015 module is an Administrators module and allows the Start and End Coordinates for every Level 3 street to be checked and reset where appropriate, using the Street’s constituent ESU’s. Any Streets which have their Start and/or End Coordinates updated will be versioned. The Start and/or end Coordinates can be checked and reset for individual Streets using the [Calc Coords] button (Spatial Manager) or [Reset Nodes] button (Web Forms) on the Maintain Streets form. During the process an indicator will be displayed showing the progress of the job.

Figure 26

Note that running this process may take several hours

Tip

If this is being done for error correction purposes and Street Versioning is not required, set product option ‘VERSIONING’ (page 10) to N. On completion of the process this Product Option MUST be reset to Y.



25

Generate ASD Placements – NSG0025

Figure 27

The Generate ASD Placements – NSG0025 module is an administrator’s module that generates ASD membership records for all ASD items flagged as not ‘Whole Road. Within the form each column heading can be used as a sort button. Select the heading once to sort ascending (^) and again to sort descending (v). For Exor customers who create and manage their ASD within Street Gazetteer Manager, running this module is a one-off task that should be run before ASD Spatial Layers are initially created. The creation of subsequent ‘partial’ ASD member records is automated when new ASD records are created or existing records are amended. For Exor Customers who load their ASD data using the NSG Loader – NSG0040, this process must be repeated after each successful data load.

Figure 28



26

To create the ‘Partial’ ASD members press the [Perform Operation] button. If ASD data from more than one Highway Authority has been loaded, the process can be restricted to a specific Authority by selected the required Authority from the available List. The default is to generate ASD placements for All Authorities. When the operation is performed, all Level 3 Streets (i.e. Streets with constituent ESU’s) with no ASD member records are displayed. The process cycles through each Street in turn and creates the ‘Partial’ ASD members for each ASD record Type. To view all errors at the end of the process press the [Enter Query] button, tick the ‘Errors’ field then press the [Execute Query] button.

Common Errors

There are several common errors that can occur during the generation of ASD placement records.

Error Message Meaning

ORA-20001: NO PATH ORA-06512: AT "DCCNSG.NM_CNCTS", LINE 625 ORA-06512: AT "DCCNSG.NM_CNCTS", LINE 1

This error means that the ESU’s between the Start and End Coordinates of the ‘Partial ASD’ record are not connected, i.e. there is no connected path between the start and end of the ASD record.

ORA-20001: NO STREET ELEMENTS CLOSE ENOUGH TO THE XY CO-ORDINATES ORA-06512: AT "DCCNSG.NM_CNCTS"

This error means that the Start and/or End Coordinates of the ‘Partial ASD’ record are not within the tolerance distance of the Street. This is generally due to poor quality ASD data

Refer to the Spatial Server Admin Guide for details on the actual Layer Creation process.



27

Importing NSG and ASD Data

Figure 29

Street Gazetteer Manager allows the Import of a Level 3 Gazetteer complete with Associated Street Data (types 21, 22 and 23) and an Organisation and District file. Importing data is done using the NSG Load Batches – NSG0040 module. The progress of the load can be viewed using the Monitor NSG Loader – NSG0045.



28

NSG Load Batches - NSG0040

Figure 30

Figure 31

The NSG Loader Batches – NSG0040 module is used to load the gazetteer files into the database. Multiple files can be added to any given batch. Multiple batches can be set up and then loaded sequentially or at a defined time. For example, gazetteer loads can be set up during the working day but then be submitted to run overnight when no users are on the system. A batch can be composed of a mixture of files of any data transfer format and from a mixture of suppliers.



29

As a load progresses, full details of the activities undertaken are written to a log. There is one log per batch. This is similar to the functionality of the previous loader but is less verbose. It is designed to complement the higher level information shown in the batch schedule. The file column headings can be used as sort buttons. Press once to sort in ascending order (^). Press again to sort in descending order (v).

Figure 32

Batches tab

The Batches tab is used to identify the batch name when creating a new batch. Once the batch has been selected for processing further details regarding the batch will be shown automatically. Batch Default

The batch number will be added by the system when the record is saved. Name Required

Enter the name of the batch. Available Date Display Only

The available date will display the date and time that the batch is to be processed once it has been selected. Load Started Display Only

The date and time the load process started will be displayed. Load Ended Display Only

The date and time the load process finished will be displayed. Successful Display Only

The number of files that have been loaded successfully will be displayed. Unsuccessful Display Only

The number of files that have not been loaded successfully will be displayed. Completion Message Display Only

Once the batch loading has finished this will display the message ‘Processing Complete’.



30

[Files Selection] The [Files Selection] button is used to select the files to be placed in the batch. Pressing this button will call the File Selection window.

Figure 33

Directory Default

This will display the filepath set in the Directories – HIG1895 module under the option NSG_DATA. This can be amended by pressing the [Set] button to call form Directories – HIG1895. .csv Radio Button

Select this radio button to display only csv files in the directory. .xml Radio Button

Select this radio button to display all xml files in the directory. It may be that the user wishes to load all the organisation and district files first. This can be done easily be selecting the .xml button and then the [Select All] button. [Refresh Directory] button refreshes the directory listing to pick up files deposited in or removed from the NSG_DATA directory since the last listing was produced. [Copy file to Directory] button allows a file held locally to be copied down to the NSG_DATA directory. Details of the files available will include:

File name

Organisation code

Organisation name

Date

Format



31

DTF version number Files are selected by checking the box against the required record. [Auto Pick] will apply a filter on filenames in the directory that have previously been attributed to a batch and tick the file checkbox for inclusion in the batch. [Create a Batch per Supplier] will list all files in the directory (not just those queried back in the form) and create a batch per distinct supplier and read the files in for each batch. It does not automatically submit the resultant batch. [Add Selected File(s) to Batch] will add the selected file to the batch. Status Display Only

Initially the status will refer to the file highlighted and give a status message of, for example, ‘File appears valid’. Once the files have been selected and the [Add Selected File(s) to Batch] pressed the status message will change.

Figure 34

Once the files have been read and pre-processed with no errors the File Selection window will close. The files will have been moved from the NSG_DATA directory to the NSG_PROCESSED directory.



32

Figure 35

Files in Batch tab

This tab will display the files selected during the File Selection process and display:

File name

Organisation code

Organisation name

Date

Format

DTF version number

Read error If there is an error in reading the file it can be removed from the batch using the [Delete Record] button. Status Message Display Only

Initially the status message will read ‘File ready for processing’.

Figure 36

Batch Schedule tab

This tab will track the progress of the batch load process. A number of operations must be performed, depending on the records read from files associated to the batch. At the time of submission these operations are identified and built into the batch schedule. This information is presented to the end user to gauge progress, success and performance of the load.



33

Each operation attributed to a batch schedule is referred to as a stage. Once a stage has been processed it will be attributed with a status depending on the outcome of the processing. Dropping of Spatial Indexes If a batch contains street or ESU or cross references or ASD records then the very first stage of the batch schedule will be to drop the spatial indexes. This aids performance when processing data. When indexes are dropped a warning message is shown against the stage within the batch schedule tab. This is to re-iterate to the user the fact that the indexes have been dropped and that they will need to be recreated before the application can be used.

Figure 37 Warning

Street Spatial Consolidation If a batch contains street or ESU or cross references records then the last but one stage of the batch schedule will be a ‘Street Spatial Consolidation’. The processing carried out at this stage ensures that the spatial data for streets within the batch is kept up to date.

Where a street has been closed, existing street shapes will be end dated.

Where an esu member has been removed from a street the effected shape will be end dated.

Where an esu member has been added to a street a new shape will be created.

Figure 38 Spatial Consolidation

Note: The NSG metadata will need refreshing after loading the data because the area covered by the gazetteer could have increased so it will need re-assessing. This is carried out using the GIS Layer Tool – GIS0020 (Spatial Server Admin Guide – Creating Street Gazetteer manager Layers, Page 53).

Dropping Spatial Indexes

Street Spatial Consolidation



34

ASD Spatial Consolidation If a batch contains street or ESU or cross references or ASD records then the last stage of the batch schedule will be a ‘ASD Spatial Consolidation’. The processing carried out at this stage ensures that the spatial data for ASD against streets within the batch is kept up to date.

Firstly, the partial ASD memberships (also referred to as ASD placements) need to be kept in line with the existing streets and esus for streets in the batch. Therefore existing partial ASD members are dropped.

Partial ASD memberships are then re-created on a street by street basis.

Note: if a single ASD record fails, perhaps because of xy coordinates of the ASD record are nowhere near the street that the ASD is attributed to, then ALL partial ASD memberships upon that street will not be created.

Next existing ASD shapes for whole road and partial ASD on the effected streets in the batch are removed.

Finally ASD shapes are re-generated.

Figure 39 ASD Spatial Consolidation

The Schedule will display:

Sequence number

Processing stage The Record Counts will display the number of records:

Unprocessed

With error

With no error

A count for each The Timings will display the time taken for each record type processed:

Days

Hours

Minutes

Seconds

A count for each

ASD Spatial Consolidation



35

The Outcome will show whether the record type has been successful, has errors or has warning. Errors will be highlighted in red. Warnings will be highlighted in yellow.

Figure 40

Records tab

This tab will list records that either have errors or have been unprocessed. The Filter can be used to display only Unprocessed records, only Failure records or both by selecting the relevant radio button. The file column headings can be used as sort buttons. Press once to sort in ascending order (^). Press again to sort in descending order (v). Details shown for each record are:

Type of record

File name

Organisation name

Record identifier

Status The Notes field will display an explanation of for each record with a status or Error. Common errors can include:

ESU does not have a valid geometry

Interested Authority <number> does not exist as an Organisation

Maintainer district <district number> does not exist as a district for <district number>

Street <number> does not exist (this note may be shown when loading asd data)



36

Figure 41

Batch Log tab

This tab shows a breakdown of the batch process. During a load batch it will be continually updated. As a load progresses, full details of the activities undertaken are written to a log. There is one log per batch. It is designed to complement the higher level information shown in the batch schedule. Now, after a batch has been processed, four log files are automatically created in the NSG_PROCESSED directory. These can be opened in a text editor or Excel.

batch<batch_id>_schedule.csv lists the processing stages and outcome of each stage.

Figure 42

batch<batch_id>_file_summary.csv lists the files contained within the batch and a count of records which were not successfully processed.

Figure 43

batch<batch_id>_problem_recs.csv lists records within a batch which were not successfully processed.

Figure 44

batch<batch_id>_log.csv lists the full batch log.



37

Figure 45

There are a number of buttons on the bottom of the form. [Submit Now] Select the [Submit Now] button to submit the selected batch. The batch will be submitted using the system date and time. [Submit Later]

Figure 46

Date Default

The date will default to the system date. Time Default

The time will default to the time defined in module Product Options – HIG9130 under option Id DFLTSUBMIT. [Withdraw] Selecting the [Withdraw] button will withdraw the submission of the batch. This will remove the ‘Available Date’. The batch can be resubmitted when convenient. If a batch is withdrawn which is already being processed the load process will finish at the earliest convenient opportunity. The processing of a batch that has failed and is re-submitted OR has been withdrawn and then re-submitted will pick up where it left off.



38

Floating Toolbar

The Floating Toolbar contains functions that can be performed in the NSG Loader Batches form. To call the toolbar press the Toolbar icon on the main toolbar. The functionality behind each button is as follows: [Spatial Indexes] The [Spatial Indexes] button can be used to re-create spatial indexes on NSG tables that the loader works with. Certain tables that the NSG Loader interacts will have a spatial index which allows that data in that table to be retrieved efficiently. A consequence of having such an index in place is that as data is added to/removed from these tables it takes time to keep the index in line. Therefore, these indexes are now dropped as part and parcel of NSG loader processing to aid performance. As and when all batches have been processed this button can be used to reinstate them.

Note: Without spatial indexes in place the relevant spatial layer(s) will not be visible in Locator.

[Gather Statistics] The [Gather Statistics] button allows statistics on necessary tables that the NSG Loader works with to be refreshed. With up-to-date statistics the loader can make the correct choices when deciding how to process data. Pressing the [Gather Statistics] button after preparing all of batches (but before submitting them for processing) should make sure the loader is primed to process data efficiently. Pressing the button again after all batches have been process should make sure that the Street Gazetteer Manager application is primed to work with data efficiently as well. If necessary you can still opt to gather statistics on tables whilst the loader is running. However, the result of taking this approach to keeping statistics up-



39

to-date is that it may take longer to gather the statistics than the performance benefit gained by having then in place. [Refresh CTL] The [Refresh CTL] button can be pressed once all batches have been processed to allow the list of Counties, Towns and Localities to be refreshed. The NSG loader will always attempt to refresh Counties, Towns and Localities whenever streets have been included in the batch that is being processed. However, if Product Option – BYPASSCTL is set to Y then this part of the load will be bypassed. [Refresh HA] The [Refresh HA] button can be pressed once all batches have been processed to allow the list of Highways Authorities Counties to be refreshed The NSG loader will always attempt to refresh HA Counties whenever streets have been included in the batch that is being processed. However, if Product Option – BYPASSHA is set to Y then this part of the load will be bypassed. [Purge Batch] The [Purge Batch] button will clear down records from database that are related to the current batch but which are no longer required. This will aid performance when processing other batches and it is advised that you purge on a regular basis. [Purge Successful Batches] The [Purge Successful Batches] will perform the same functionality as the [Purge Batch] button, but for all batches that have been processed without error. [Shape Validation] The functionality behind the [Shape Validation] button will validate the shapes of all ESUS and allow removal of those shapes that are invalid and cannot be repaired. One common example of an invalid ESU shape is one that is actually a point, rather than a line. It’s expected that this will be a one-off exercise and should be done as soon as upgrading to release 4.3.0.0.



40

[Shapeless ESU’s] Street Gazetteer Manager relies on all ESUs having a shape. The [Shapeless ESU’s] button gives access to a screen which allows ESU’s without a shape to be removed from the database. Such ESUs could have been created by the loader prior to release 4.2.0.0, or be shapeless as a result of using the aforementioned ‘Shape Validation’ functionality. It’s expected that this will be a one-off exercise and should be done as soon as upgrading to release 4.3.0.0 but after the [Shape Validation] has been carried out.



41

Monitor NSG Loader - NSG0045

Figure 47

Figure 48

This module is used to monitor the progress of files that have been submitted to the queue and processing activity. A database job runs constantly which looks for available batches to process.



42

[Withdraw Batch] Selecting the [Withdraw Batch] button will withdraw the submission of the batch. This will remove the ‘Available Date’. The batch can be resubmitted when convenient. If a batch is withdrawn which is already being processed the load process will finish at the earliest convenient opportunity. The processing of a batch that has failed and is re-submitted OR has been withdrawn and then re-submitted will pick up where it left off. The screen is split into three areas:

the batch queue

batch schedule of the current batch in the queue

progress information relating to the batch currently being processed

The Batch Queue will display:

Batch Id

Batch name

Available date – the date and time the batch is to be processed

Load started date The progress information relating to the batch currently being processed will display:

The batch schedule stage being processed highlighted in green

Any batch schedule stage with a status of Error highlighted in red

Any batch schedule stage with a status of Warning highlighted in amber

Any batch schedule stage with a status of Success highlighted in grey

A stage can be broken down into several operations. Where possible, when an operation is expected to take some time to complete a progress bar indicating a percentage complete and estimated end time will be shown. Errors and warnings can be checked by using module NSG Loader Batches – NSG0040.



43

[Options] Latest Activity Display Only

The date and time of the last job run through the NSG Loader Batch will be displayed. Session Display Only

The session number of the last job run will be displayed. Job Display Only

The job number of the latest activity will be displayed. Watch Checkbox

This checkbox allows the user to switch on/off the constant monitoring of progress. When exiting the form the checkbox is automatically unchecked. If the form is then re-opened and the box was previously checked it will be re-checked automatically and monitoring of progress will resume. Gather Statistics Whilst Loading Checkbox

In order for Oracle to determine the most efficient approach to manipulate data, it requires up-to-date statistics against tables in the database. Due to the nature of any loader, data volumes can increase/decrease during the load process and therefore statistics can become out of date (stale) and potentially have an adverse effect on load performance. Therefore the end user has the option to use this checkbox to re-gather statistics as and when they are needed by the load process. It is advised that this option is always taken. However it is recognised that there is always an offset of the time taken to gather statistics VS the performance benefit the statistics offer. Statistics will now only be gathered if the existing statistics on the database table in question are out of date i.e. there is potential benefit in collating them.



44

The default value will have been defined in module Product Options – HIG9130 under option Id LOADSTATS. Bypass Count/Town/Locality Refresh

If this box is ticked the County/Town/Locality refresh will be skipped when processing data.

Note: This can be done manually in form NSG Loader Batches - NSG0040 using the [Refresh CTL] button on the toolbar.

Bypass HA Refresh

If this box is ticked the Highways Authority refresh will be skipped when processing data.

Note: This can be done manually in form NSG Loader Batches - NSG0040 using the [Refresh HA] button on the toolbar.

The [Recreate Job] button can be used to restart the job if possible. The [Drop Job] button can be used to drop the job if, for example, it doesn’t exist. The [Close Window] button closes the Options window and returns to the front screen.



45

Administer User Districts – NSG0120

Figure 49

Figure 50

General Information

The Administer User Districts form enables the System Administrator to define which operational districts are available to a specific user. For example: a user may perform the function of a street authority and monitor notices coming in from a works promoter. The same user may also perform the function of a works promoter and enter notices on behalf of the highway authority. The user then has the option of swapping from one district function to another by using the My Districts – NSG0130 form.



46

The Administer User Districts form is available from either the Street Gazetteer Manager menu or the TMA Manager menu.

Figure 51

User Districts

Organisation Optional List

Enter the organisation to be available to the selected user. District Optional List

Enter the organisational district to be available to the selected user. District Function Display Only

The function of the selected district will be displayed. Current Checkbox

The current district assigned to the user will be checked.

Use the [Set as Current] button to change the current district.

Note: the new district must be selected before the [Set as Current] button is available.

Note: all forms should be closed before changing the district of the user.

The following alert prevents the district being changed for a user.

Figure 52



47

My Districts - NSG0130

Figure 53

Figure 54

General Information

The My Districts enables the user to define which operational district they are currently using. The user can swap between the district functions assigned to the user by the System Administrator. For example: a user may perform the function of a street authority and monitor notices coming in from a works promoter. The same user may also perform the function of a works promoter and enter notices on behalf of the highway authority.

User Districts

Organisation Display Only

The organisation set as available to the selected user will be displayed. District Display Only

The organisational district set as available to the selected user will be displayed.



48

District Function Display Only

The function of the selected district will be displayed. Current Checkbox

The current district assigned to the user will be checked.

Use the [Set as Current] button to change the current district.

Note: the new district must be selected before the [Set as Current] button is available.

Note: all forms should be closed before changing the district.

The following alert prevents the district being changed for a user.

Figure 55

exor corporation limited · 2011-11-24 · 4 nov 2010 ah product options bypassctl, bypasshac have...

Documents