Data & Information Integration Framework for Highway Projects

Mid-Continent Transportation Symposium

Asregedew WoldesenbetDavid H. Jeong (Ph.D.)

Michael P. Lewis (Ph.D., P.E.)

August 15, 2013

Outline

Research QuestionLessons LearnedMethodology EvolutionIntegration FrameworkCase StudyGap AnalysisConclusion/Future Work

Research QuestionIs data currently being collected provides the

information needed for decision-making?

◦ Minimal recognition or interest in using these data

◦ Lack of in-house resources and capabilities to analyze data

◦ Insufficient data for any meaningful analysis

◦ Nonstandard /non-digital data format

◦ Poorly defined procedures/mechanism

Lessons Learned◦ Strategic decisions supported by statistically reliable

information Credit card industry Retail industry Healthcare industry

◦ Big Data

◦ System/Tools KM tools and KDD approaches

◦DM, AI, DSS, ML, BI Management philosophies

◦BPR, TQM, SCM, CE, LC Database System

◦Ontology frameworks, cloud computing

Generations of Data & Information Management: Transportation Industry

Expert

Judgment

File Cabinet, PC

Database/Datawarehouse

Excel/

Statistics

KDD/DM

1st Generation

2nd Generation

3rd GenerationKnowledge

Portal

System

Data

Col

lectio

n

Approach

Sem

i-Aut

omat

edAu

tom

ated

Man

ual/

Pape

r-bas

ed

Evolution of Data and Information Integration for Highway Agencies

Active Information & Knowledge Extract

Integrated Data & Information Framework ion

to Support Decision MakingVarious DATABASES

- Data Collection Efforts

3rd Generation

2nd Generation1st Generation

Data Collection - Manual/Paper-BasedApproach - Expert JudgmentSystem - File Cabinet e.g. Contract Documents - PC e.g. Cost Data - Database e.g. Road Inventory - Other Databases

Data Collection - Semi-Automated/Automated Approach - Statistical Tools - Artificial IntelligenceSystem - Project Management System - Database e.g. SiteManager - Data Warehouse (DW)

Data Collection - Automated - Standard Data Collection ProcedureApproach - Pattern Recognition - Knowledge Discovery in Database (KDD) - Data Mining (DM)System - Ontology Based Knowledge Management System - Big Data Analytics Algorithm - Knowledge Portal e.g. cloud-based system

Data & Information Integration

X X

X X

X

X

DM1

DM2

DM3

DM4

I1 I2 I3

X X

X X

X

X X

D1

D2

D3

D4

Row Form

Column Form

Element Form

D1

D2

D3

Dn

I1

I2

In

DM1

DM2

DM3

DMn

Input Processor Output

..…

..… ..…

Context GraphInput/Output Matrix

Data & Information Integration Framework

Three-Tiered Hierarchical Framework

Planning Phase

Design Phase

Bidding Phase

Construction Phase

Operation Phase

DMA DMB DMN

I1N I21 I22 I2n Im1 Im2 Im3 ImnI12

D11 D12 D13 D14

I11

D1n D21 D22 D23 D2n Dm1 Dm2 Dm3 Dmn

Decision

Information

Data

Active PathInactive PathNon-Existing Path

Legend :

…..

DATABASE I DATABASE II DATABASE N…………….

…..

…..…..

…..…..

….....

Case StudyDaily Work Reports (DWR)

Preconstruction Cost Data

Pavement Condition Data

Case StudyDivision/Source Database Type of Data Sub-Elements Collection Method

System Planning/ Research

Grip lite/ Highway Inventory

Roadway InventoryFunctional Class, Right of Way, Route Classification, Terrain Area Type, right-of-way, railroad crossing, etc.

Manual / Semi-AutomatedTraffic

Average Annual Daily Traffic (AADT), signals, lightings, traffic control, crash statistic, etc.

Bridge Inventory Bridge span, width, length, load limit, inspection reports, etc.

Preconstruction In-house Spreadsheets

Preliminary Engineering Data Engineering hours, number of sheets, etc. Manual

Construction Division SiteManager Construction Data

Daily work report, reported quantity, material, change order contractor payment etc.

Manual

Pavement Management

Pavement management System (PMS)

Pavement History Pavement surface type, thickness, composition, etc. In-house - Automated

Distress Data Longitudinal Cracking, Transverse Cracking, Patching, Spalling, Fatigue, etc. Consultant - Roadware

Friction Data Average Roughness, Ride, Average Rut etc. In-house

Other (structural) Deflectometer (FWD), ESAL In-house Roadrater

Current Data Utilization

Type of Data Data Attributes Data Description Data Type No Use

Current UseContrac

tor Paymen

t

Dispute

Resolution

Reporting

Percentage Completio

n

I1 I2 I3 I4

DWR Info Contractor ID D1 ID 000001-100000 Numeric : Ordinal       X    Inspector Name D2 Last and first name Character : Nominal       X    Date D3 xx/xx/xxxx Numeric : Ordinal       X    Low Temperature D4 Temp. oF Numeric : Interval     X X    High Temperature D5 Temp. oF Numeric : Interval     X X    AM Condition D6 Sunny, windy, cloudy, etc Character : Nominal     X X    PM Condition D7 Sunny, windy, cloudy, etc. Character : Nominal     X X    Work Suspended Time D8 Time AM/PM Numeric : Ordinal     X X    Work Resumed Time D9 Time AM/PM Numeric : Ordinal     X X    Humidity D10 - - X          Precipitation D11 - - X        Contractor Contractor D12 Name Character : Nominal     X X    Subcontractor D13 Name Character : Nominal     X X    Supervisor D14 Foreman, superintendent, etc. Character : Nominal       X    Personnel D15 Laborer, concrete finisher, etc. Character : Nominal     X X  

  Supervisor Hourly work D16 Number of Hours Numeric : Interval     X X  

  Personnel Hourly work D17 Number of Hours Numeric : Interval     X X    Supervisor Number D18 Count Numeric : Interval     X X  

Ideal Data, Information & Decision-Making Framework

Three-Tiered Framework

Planning Phase Construction Phase

Decision

Information

Data

Sitemanager

Project Management

Performance Measure

Construction Data

Databases

Design Phase

Contractor Type

Production Rate Accident Analysis

Inspector Date Precipitation Project Type

Determine Contract Time Maintenance Roadway Design Traffic & Safety

Design

Distance Accidents Supervisor Remark

Bridge Design

Prime Contr. Work

Cost TrackingResource Allocation

……….D1 D2 D3 D11 D12 D29

……….

D30 D36

……….

D37

Type of Day……….

D39

I5 I6

DM1 DM2 DM3 DM4 DM5 DM6 DM7

Contractor Payment I1

Gap Analysis

Current Data

Current Information

Current Decisions

Ideal Data

Ideal Information

Ideal Decisions

Missing data (D1 - D3)- Humidity, precipitation, etc.Unstructured Data (D1 - D3)- Remarks ((D33 –D41) Not used data (D1 - D3)- Accidents (D30), delays (D31), etc.

Missing information (I5 - I9)- Production rate - Accident analysis- As-built information, etc.

Missing decisions (DM1 - DM9)- Resource Allocation- Contract time determination- Maintenance, etc.

Gap AnalysisCriteria Gap

StaffNeed for data analyst or data scientist Need for responsible party in data collection, information generation and decision-making

Function Need for decision-maker requirement, identifying characteristics and use

Time Need for data and information to reach the user or decision-maker in a timely manner

Availability Missing data and information

Format/Structure Need for change of textual or linguistic data types, lack of standard

Individuality Division having standalone units to match only particular needs

Technology Need for appropriate tools and technology to extract information

ConclusionSummary

◦ DWR are often utilized in reporting and preparation of legal disputes.

◦ Reported quantity and work item are the primary data that are utilized in contractor payments and tracking project progress.

◦ More than 35% of the DWR data are linguistic in nature.◦

Conclusion◦ Lack of skilled data analysts and experts to analyze data◦ Lack of well-developed requirement analysis and

performance measures.◦ Focus of specific divisions or business processes to promote

own division’s need rather than develop integrated system

ConclusionData, Information & Decision-making Guideline

Identify Key Decisions

Identify Data, Information & Knowledge (DIK)

Identify Key Performance Indicators

Identify Database & Decision Tools

Check Availability of Data, Information & Knowledge

Assess Current level of Use & Quality

Develop Data, Information, Decision-Making Path

Cleanup Data

Define Data, Information & Knowledge

Develop New Module/Database

Perform Cost/Benefit Analysis

Convert Data into Information

Apply Appropriate Tool or Decision Support System

Requirement Analysis

Evaluation/Assessment

Data Process/Manipulation

Data Generation Scheme

Strategic & Network Level Decisions

Program & Project Selection Level Decisions

Project Level Decisions

Quality Function Deployment

ConclusionContribution

◦ Ability to show types of data that should be collected and potential information & knowledge generation

◦ A general guide to highway agencies in the development of active utilization of currently existing databases.

◦ Help develop new data collection, information & knowledge generation plan to support key decisions

Future Study◦ Emphasize in developing an enterprise wide ontology-based

framework ◦ Application of big data analytics to justify the return on

investment for the data collection efforts and effectively utilize the increasing amount of data.