interface control document mobilize maintenance m20 with

INTERFACE CONTROL DOCUMENT MOBILIZE MAINTENANCE M20 WITH ESRI

PORTAL

October 30, 2017

of 18

Interface Control Document Mobilize Maintenance M20 with Esri Portal

Table of Contents Section 1 Document Information ............................................................................................................................................ 3

1.1 Information .................................................................................................................................................................... 3

1.1.1 Functional Requirements – Data Management and Reporting on M-20 ............................................................... 4

1.1.2 Functional Requirements – Data Exporting ............................................................................................................ 5

1.1.3 Functional Requirements – User Interface ............................................................................................................. 5

1.1.4 Functional Requirements – System Interfaces ....................................................................................................... 5

1.1.5 Functional Requirements – System Access and Security ....................................................................................... 6

1.2 Version History ............................................................................................................................................................... 6

1.3 References ..................................................................................................................................................................... 6

1.4 Relevant Glossary........................................................................................................................................................... 7

Section 2 Integration Conceptual View ................................................................................................................................... 7

2.1 Diagram and Description ............................................................................................................................................... 7

2.2 Products, Platforms and Software Matrix ..................................................................................................................... 7

Section 3 Logical View.............................................................................................................................................................. 8

3.1 Diagram and Description ............................................................................................................................................... 8

3.2 Portal Content and Web Services .................................................................................................................................. 8

3.2.1 M20 Feature Services ............................................................................................................................................. 8

3.2.2 Portal Content ......................................................................................................................................................... 9

3.3 ETL ................................................................................................................................................................................ 10

3.3.1 Data Integration: Overview .................................................................................................................................. 10

3.3.2 M20 Data: Import from PMSS to GIS & Update (Tools A, B & C) ......................................................................... 11

3.3.3 M20 Data: Import from ArcGIS Enterprise Geodatabase to PMS Staging Table (Tools D & E) ............................ 14

3.3.4 M20 Data: Import from PMS Staging Table into PMS M20 Tables (Tool F) ......................................................... 16

Section 4 Physical View .......................................................................................................................................................... 17

4.1 Diagram and Description ............................................................................................................................................. 17

Section 5 Assumptions and Risks ........................................................................................................................................... 18

http://www.virginiadot.org/default.asp


PORTAL

October 30, 2017

of 18



PORTAL

October 30, 2017

of 18

Section 1 Document Information This document includes the systems integration information and implementation in order to facilitate the flow of

data between PMSS (Pavement Maintenance Scheduling System), field collected data managed by Esri Portal

internally hosted by VDOT, and PMS (Pavement Management System). Generally, Systems integration is

defined as facilitating the communication of information between 2 or more different applications, systems,

platforms and/or products. This document includes the details for the following types of components in an effort

to provide sufficient to identify the method of information flow among systems:

• Web service integrations

• Data integrations

• Mobile field data integrations

However, this document does not include information for the following components as they are considered

Universal across most applications:

• User-specific logins

• Database connections made by an application to perform CRUD transactional operations as these

connections do not qualify as integration efforts among different systems

1.1 Information

The Pavement Management program requires a technology based solution for agency-wide data collection and

management system whose primary function is to expedite and streamline the process of collection, storage and

reporting on M-20 data.

To accomplish the above articulated need, VDOT requires an agency-wide data collection and management

system that includes the following functionality:

• M-20 Data Management Services

o tracking pavement maintenance work completion and associated business data (paving location,

material layers and thicknesses, contract data, and field notes)

o pre-populate pavement maintenance and associated pavement material layer data based on

pavement maintenance schedule contract data available in PMSS

o allow update to planned work location, material layers and thickness, contract data, and field

notes

o allow capture of unplanned or non-PMSS pavement maintenance work through addition of new

records during field collection or office data entry

o house incomplete or planned maintenance work records with a process for adding records to

primary database (PMS) when record is indicated complete

o load GIS data into PMS staging tables

o utilize mobile technologies for data collection

o capacity to export business data to support District information needs (e.g. planned work

completion vs. remaining)

• Exporting Capacity

o capacity to select from map and export the associated data

o access to image files and reports

o capacity to generate a map output of current extent



PORTAL

October 30, 2017

of 18

• User Interface

o provide a visual indication of required data elements in online screens

o provide a save option for data editing

• System Interfaces

o Pavement Management System

o Pavement Maintenance Scheduling System

o Portal for ArcGIS

o Collector for ArcGIS

o WebApp Builder

• System Access and Security

o Integrated technical safeguards to ensure a level of privacy and security, including

Administrator controlled user name and password access

Administrator controlled user level read, write, edit and delete capabilities

o Capable of supporting 25 concurrent users (collection at multiple paving locations across

multiple Districts, with simultaneous data entry/review in office )

1.1.1 Functional Requirements – Data Management and Reporting on M-20

Req. ID Requirement Definition

FR2.0 The system shall provide the ability to manage maintenance treatment

material layer and pavement schedule data as captured on VDOT Form

M-20

FR2.0.1 The system shall provide functionality to support communication of

segment specific decision making and details during field collection

within comments fields (e.g. location of misc. patching locations,

comments regarding field decisions)

FR2.1 The system shall provide functionality to pre-populate Form M-20

information based on PMSS contract data for field validation and update

FR2.1.1 The system shall provide opportunity for update of pre-populated

information (including location and business information) as well as

identification of planned work completion

FR2.1.2 The system shall provide opportunity to capture unplanned or non-

PMSS pavement maintenance work through addition of new records

FR2.2 The system shall provide functionality for data selection, identification,

query and exporting from a mapping interface.

FR2.3 The system shall provide functionality for data selection, identification,

query and exporting from a graphic interface.

FR2.4 The system shall provide method for data transfer of pre-populated

and/or field validated Form M-20 information into PMS pavement work

history



PORTAL

October 30, 2017

of 18


FR2.4.1 The system shall store incomplete or planned work records with a

process for adding records to primary database (PMS) when record is

indicated as complete. This process will developed with consideration

of possibility for future PMS system functionality to allow pre-

population of “shell” M-20 directly from PMSS

FR2.5 The system shall provide the capability to convert data stored against

VDOT LRS into GIS and vice versa

FR2.5.1 The system shall represent VDOT pavement maintenance segments as

line features based on associated route geometry

FR2.6 The system shall provide the functionality for data entry, update, and

review through mobile technology

1.1.2 Functional Requirements – Data Exporting


FR3.0 The system shall provide capability to select from map and export data

FR3.1 The system shall provide user access to image files and reports

FR3.2 The system shall provide ability to generate a map output from current

extent

1.1.3 Functional Requirements – User Interface


FR4.0 The system shall provide a visual indication of required data elements

on screen in online and mobile screens during user data entry

FR4.1 The system shall provide a save option for data editing

FR4.2 The system shall provide a tool to collect data in the field

1.1.4 Functional Requirements – System Interfaces


FR5.0 The system shall interface with the Pavement Management System to

provide completed M-20 data for import to PMS pavement work history

FR5.1 The system shall interface with the Pavement Maintenance Scheduling

System to capture planned pavement maintenance work for pre-

population of M-20 data for field validation, update, and approval.

FR5.2 The system shall interface ArcGIS Portal to exchange GIS data across

VDOT firewall

FR5.3 The system shall interface ArcGIS Server for extraction or load into

PMS and PMSS systems



PORTAL

October 30, 2017

of 18

1.1.5 Functional Requirements – System Access and Security


FR6.0 The system shall provide integrated technical safeguards to ensure of

level of data privacy and security

FR6.0.1 The system shall provide Administrator controlled user name and

password access

FR6.0.2 The system shall provide Administrator controlled user level read, write,

edit, and delete capabilities

FR6.1 The system shall support 25 concurrent users (through both office and

mobile field data review, entry, and update)

1.2 Version History Version Date Author Description

1.0 3/24/17 Courtney Moore Beginning draft document

1.1 3/29/17 Rachel O’Neil Technical, logical and physical architecture

1.2 3/29/17 Matt McCracken Review

1.3 3/29/17 Rachel O’Neil Updated an error 1.4 3/29/17 Will Duke Updated functional requirements

1.5 5/19/17 Courtney Moore Splitting M20 from Pavement Core

1.6 6/12/17 Courtney Moore Updated ICD 1.7 6/22/17 Courtney Moore Updated ICD

1.8 9/21/17 Jarrett Green Updated ICD diagrams

1.9 9/25/17 Jarrett Green Updated ICD diagrams

1.10 9/27/17 Will Duke Review

1.11 10/1/17 Jarrett Green Updated ICD

1.12 10/30/17 Jarrett Green Updated Section 3.2

1.3 References Reference Name Link Description

<<Project Solution Architecture Document>>

Link to the SA Document Library [both document link and project View]

Will contain overall Project Architecture information if the integration is being created as a result of a project.



PORTAL

October 30, 2017

of 18

1.4 Relevant Glossary

Glossary Item Link

Section 2 Integration Conceptual View

2.1 Diagram and Description

The M20 data on ArcGIS Server, which will reside in locally hosted Portal for ArcGIS, will provide feature services for field

collection of data and will be used to update PMS. An ETL process initiated by an administrator will import data from

PMSS into the ArcGIS Enterprise Geodatabase in order to provide data for field crews to update through ArcGIS Server,

although they will also have the ability to create new records (records that did not originate in PMSS).

2.2 Products, Platforms and Software Matrix

No. Name Version Vendor Name Purpose Lifecycle Status

License Status

1 VDOT PMSS 5.172 VDOT Supports Pavement Maintenance Scheduling and the creation of contracts

Static Licensed

2 VDOT PMS 7.0.4.0 VDOT Provides Pavement management by collecting and retaining information on pavement management activities

Static Licensed

3 Portal for ArcGIS 10.5 Esri Will host WebMaps and WebApps to aid in field collection and to allow the viewing and editing of data via in office applications

Static with scheduled updates

Licensed

4 ArcGIS Server 10.3.1 Esri To provide feature services to facilitate and store geospatial and business data


Licensed

5 ArcGIS Enterprise Geodatabase

10.3 Esri All data will be stored in this GIS database


Licensed



PORTAL

October 30, 2017

of 18

Section 3 Logical View


The below sequencing diagram depicts a logical starting point for PMSS, Portal and PMS interactions. This diagram shows

the initiating process of exporting data from PMSS into ArcGIS Portal to be exposed as a published feature service. This

published feature service is then consumed by the Field Collector App where data collected is sent through the service

back to Portal. Upon completion of field collection at a given interval, data collected in the field collector and now stored

in Portal is then exported to PMS.

3.2 Portal Content and Web Services The following Portal content and web services are used in the creation and editing of field collected data. Please note

that all item URLs and service URLs are subject to change if/when moved up the environment stack. The feature services

are secure services published to VDOTGISAGSDEV. The M20_editor user is assigned and when the services were added

to Portal, the credentials were stored so users are not prompt when they access the item. VDOT’s system architecture

also allows users to view this service and data outside the VDOT network.

3.2.1 M20 Feature Services

The following feature services are consumed by Collector and the M20 Web AppBuilder.



PORTAL

October 30, 2017

of 18

Name: M20_MapService_DEV

Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=caf933debc8a4e3f8737cac79c620d9d

Service URL:

http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/caf933debc8a4e3f8737cac79c620d9d/rest/services/M20

/M20_MapService_DEV/FeatureServer

Purpose: This feature service contains the paving start and end points that users create in the Web AppBuilder and

Collector. A scheduled python script will generate the reference line between the start and end points and perform

validation against the data.

Name: M20_LRS_DEV

Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=7b8b7d7a8df742c9a3d3e0adbc81f417

Service URL:

http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/7b8b7d7a8df742c9a3d3e0adbc81f417/rest/services/M20

/M20_LRS_DEV/MapServer

Purpose: This map service contains the LRS layers for users to visualize their location and access route information.

Name: M20_Boundaries_DEV

Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=f4edf0f5a20b4f21a629f95fb6ba6b62

Service URL:

http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/f4edf0f5a20b4f21a629f95fb6ba6b62/rest/services/M20/

M20_Boundaries_DEV/MapServer

Purpose: This map service contains the District and County layers for users to visualize their location.

3.2.2 Portal Content

The following content is used to display GIS data to end users.

Name: M20 Collector DEV

Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=1fb82aaf2f724c52bcf591c2d9b0a84e

Purpose: This web map is used by Collector. It is editable and references the M20_MapService_DEV feature service.

Name: M20 WebMap DEV


http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=caf933debc8a4e3f8737cac79c620d9d

http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/caf933debc8a4e3f8737cac79c620d9d/rest/services/M20/M20_MapService_DEV/FeatureServer

http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/caf933debc8a4e3f8737cac79c620d9d/rest/services/M20/M20_MapService_DEV/FeatureServer

http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=7b8b7d7a8df742c9a3d3e0adbc81f417

http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/7b8b7d7a8df742c9a3d3e0adbc81f417/rest/services/M20/M20_LRS_DEV/MapServer

http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/7b8b7d7a8df742c9a3d3e0adbc81f417/rest/services/M20/M20_LRS_DEV/MapServer

http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=f4edf0f5a20b4f21a629f95fb6ba6b62

http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/f4edf0f5a20b4f21a629f95fb6ba6b62/rest/services/M20/M20_Boundaries_DEV/MapServer

http://vdotgisuportal.vdot.virginia.gov/portal/sharing/servers/f4edf0f5a20b4f21a629f95fb6ba6b62/rest/services/M20/M20_Boundaries_DEV/MapServer

http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=1fb82aaf2f724c52bcf591c2d9b0a84e


PORTAL

October 30, 2017

of 18

Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=188e2797d9734946af365bcdc51a6e26

Purpose: This web map is used as the input for the web application below. End users will generally not need to access

this map but must be shared so they can access the data in the web app. Layer configurations and pop-ups will be set

here by the item owner.

Name: M20 WebApp DEV

Item URL: http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=2b02c00c481c414fac1eec7322d65a8e

Purpose: This web app is the primary desktop application used by end users. Users can query, edit and export table data

from this application. Users will also verify edits in the field and update any M20 Description records that have errors.

3.3 ETL

3.3.1 Data Integration: Overview

The process to prepopulate, field update, finalize and import M20s from VDOT’s PMSS to the GIS field data collection

tools and into the PMS is described below. This process consists of the following general steps:

1. M20 Pre-population from PMSS to GIS – executed on demand 1-2 times / year

2. Field Data Collection, Review and Approval – through Collector mobile app or Desktop web application

3. M20 Import from GIS to PMS – combination of GIS and PMS scripts and business processes executed on demand


http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=188e2797d9734946af365bcdc51a6e26

http://vdotgisuportal.vdot.virginia.gov/portal/home/item.html?id=2b02c00c481c414fac1eec7322d65a8e


PORTAL

October 30, 2017

of 18

3.3.2 M20 Data: Import from PMSS to GIS & Update (Tools A, B & C)

Import from PMSS to ArcGIS Enterprise Geodatabase: on-demand tool allows an administrator/user to import data

from PMSS to ArcGIS Enterprise Geodatabase. The tool will require the user to specify through a configuration file the

Dashboard Year, Schedule Start Year, and LRS data to use to generate the M20 data in GIS. The tool will then query from

two views in the PMSS database (a Contract-Level view like the existing vPerformance, and a Materials Layers-Level

view) using the specified Dashboard Year and Schedule Start Year.

The import process will be initiated by Central Office Pavement Management through submission of a Change

Management Request to VDOT Information Technology Division. This process will then be executed by ITD staff per

their routine procedures for such a request.

Update Tool: Scheduled process updates M20 Reference Lines to reflect changes to M20 Paving Start and M20 Paving End

points made by field users every one minute. Field users will not create reference lines (as creating M20 lines that match

the underlying route is tedious and/or impossible in the field). This process will also create M20 Reference Lines for

records that were created in the field. A secondary function of the update tool is to perform business validation of all

M20 business data. These checks fall into three (3) general categories:



PORTAL

October 30, 2017

of 18

1. Location Validation: application of user entered route information to select desired route from nearby routes,

flagging where there are no nearby routes matching the user provided route information

2. Contract Information Validation: verification that all required descriptive information is provided during field

update/entry of M20 records. Will flag incomplete or improperly entered data identified on a record

3. Material Information Validation: screening to ensure accuracy of Layer, Material, and Thickness information,

flagging records with improper Layer IDs, Thickness or lacking valid Material selection.



PORTAL

October 30, 2017

of 18

M20 GIS records are created when the following conditions are met in the PMSS database:

Field Description Use

Schedule_Start_Year Year when contract begins. User-selected parameter in Import Tool.

Only create M20s for contracts with the selected Schedule Start Year Value

Dashboard_Year Year mainline pavement will be paved at this contract location. User-selected parameter in Import Tool.

Only create M20s for contracts with the selected Dashboard Year

Include_Exclude

Include = location part of contract advertised Exclude = location not part of contract advertised Not a user-selected parameter.

Only create M20s for "Included" records

Route_in_RNS Indicates if Route exists in VDOT Location Referencing System. Not a user-selected parameter.

Only create M20s for routes that are in VDOT Location Referencing System

Validate_in_RNS

Indicates if Route (and MP) are validated against VDOT Location Referencing System. Not a user-selected parameter.

Only create M20s for routes that have validated locations

Route_Name

LRS validated route on which paving is occurring and along which the M20 line should be drawn. Not a user-selected parameter.

Only create M20s for routes that begin with “R-VA”

MP_From MP_To

Used, in combination with the selected LRS, to create M20 Reference Lines showing the location along the route that the M20 record pertains to. From the line, M20 Paving Start and M20 Paving End points are also created. Not a user-selected parameter.

Only create M20 for records with valid From and To information.

Project_Status_Cd Shows status of UPC (project) in the construction process. Not a user-selected parameter.

Only create M20s for contracts which are moving forward through advertisement/award/construction process. Do not create M20s for contracts with status “No Dates Yet Set”.

Contract Width Shows overall width of paving activities at the contract location

Only create M20s for contracts which involve a minimum 9’ width. Less than 9’ would be considered partial lane width paving/patching and will not be pre-populated



PORTAL

October 30, 2017

of 18

3.3.3 M20 Data: Import from ArcGIS Enterprise Geodatabase to PMS Staging Table (Tools D & E)

M20 Data: Import from GIS to PMS Staging Table tool runs on a scheduled basis to identify GIS data that is prepared and

ready for Migration from the GIS to a PMS staging table. This process is triggered by the GIS Administrator by updating

an administrative parameter which indicates the GIS to PMS import process will proceed. Once this parameter is updated,

the Python script updates all “Accepted” GIS data (records which have been approved by Pavement field staff, and pass

all business validation) to a “Ready to Migrate” status. Once data has been marked as “Ready to Migrate” by the tool, it

will be available to be imported to the PMS Staging tables by the PMS Administrator (note: “Ready to Migrate” will not be

editable in GIS).

The administrative parameter used to trigger the process must be updated by the GIS Administrator to flag the import

process as completed (or otherwise ended). This should only be updated after PMS Import scripts to import the GIS data

to staging have been executed. Until that point, the Python script will run in the background, comparing the “Ready to

Migrate” records to the PMS M20 table; when a match is found, the script will update the GIS data to “Migrated” status.

After the GIS Administrator has confirmed that PMS Import to Staging is complete, the GIS Administrator updates the

administrative parameter to indicate that the import process will no longer be running. At that time, the Python script

performance will revert any GIS records with “Ready to Migrate” status back to “Accepted” to allow continued editing.

Records with status “Migrated” will no longer be managed in the GIS, as PMS now will retain those records as the source

system of record.



PORTAL

October 30, 2017

of 18



PORTAL

October 30, 2017

of 18

3.3.4 M20 Data: Import from PMS Staging Table into PMS M20 Tables (Tool F)

M20 Data: Import from PMS Staging Tables into PMS M20 Tables is an import process that users interact with from the

PMS front end, like other existing import processes. This process pulls information from the PMS Staging Tables and into

the PMS M20 Tables. This utilizes existing PMS functionality. PMS Administrative staff may modify data in the Staging

tables as necessary to ensure proper import to the As-Built Paving Information tables.



PORTAL

October 30, 2017

of 18

Section 4 Physical View




PORTAL

October 30, 2017

of 18

Section 5 Assumptions and Risks

Assumptions:

• VDOT Maintenance Division staff will be granted write access to M20 related GIS tables

• Timmons Group/Spy Pond Partners Team will be granted access to necessary data

• Timmons Group/Spy Pond Partners Team will be granted access to necessary systems

• VDOT ITD will support in an advisory and consulting role

Risks:

• Portal deployment will be complete at VDOT before required end date of contract

• ArcGIS Portal will be functional and available in VDOT Production environment


interface control document mobilize maintenance m20 with

Documents