plenary session 3: it best practices for - home -...
TRANSCRIPT
![Page 1: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/1.jpg)
Plenary Session 3: IT Best Practices for
DOT’s Turning Big Data into Meaningful Information within the
Transportation Enterprise
July 11th, 2017
Bruce Aquila, Executive Consultant
Hexagon Safety and Infrastructure
![Page 2: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/2.jpg)
1. What is Big Data?
2. Roadblock to the Usage of Big Data
3. Exploiting Big Data
4. Use Case
5. Conclusion
Agenda
2
![Page 3: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/3.jpg)
What is Big Data?
![Page 4: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/4.jpg)
• Massive Volume of Structured/Unstructured
Data
• Difficult to Process with Traditional Database
and Software Techniques
• Identified by:
• Volume Large volumes
• Variety Varied types of data
• Velocity Speed at which data is processed
• Veracity Reliability of the data
• Value Meaningful information
Definition
4
Courtesy Margaret Rouse
![Page 5: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/5.jpg)
• Imagery
• LiDAR
• Sensor
• Live Traffic
• Crash
• 3D CAD
• Public Relations
DOT Big Data (Examples)
5
![Page 6: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/6.jpg)
DOT Sources
6
![Page 7: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/7.jpg)
Big Data - Roadblocks
![Page 8: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/8.jpg)
• Inadequate Staffing
• Lack of Business Sponsorship
• No Governance or Stewardship
• Ambiguous Business Case
• User Engagement
• Investment Shortfall
Institutional
8
![Page 9: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/9.jpg)
• “We’ve Never Done Business like this
Before !!”
• Territoriality
• Political Alliances
• Organizational Realignment
• Business Process Re-design
• Lack of Awareness
• Privacy Concerns
Cultural
9
![Page 10: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/10.jpg)
• Archaic IT Architectures
• Insufficient Physical Infrastructure
• Existing Data Quality
• Lack of BD Technology Knowledge
• Adequate Security
Technology
10
![Page 11: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/11.jpg)
Big Data Exploitation
![Page 12: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/12.jpg)
• The Process of Examining Large and
Varied Data Sets:
• To Uncover Hidden Patterns
• Unknown Correlations
• Trends
• Customer Preferences
• Other Useful Information
Big Data Analytics
12
Allows DOT’s to make more-informed Business Decisions!!!
![Page 13: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/13.jpg)
Hadoop – Open source framework for distributed
storage of LARGE datasets on computer clusters
• Allows Data Scaling Up/Down without Hardware
Failures
• Written Predominately in Java
• Modules consist of:
• Common – Libraries/Utilities used by other Hadoop
Modules
• Distributed File System (HDFS)
• Stores data on commodity machines
• High aggregate Bandwidth across the Cluster
• YARN – Manages computing resources for clusters and
uses them for scheduling users' applications
• MapReduce – Implementation of Google MapReduce
Programming Model for Large-scale Data Processing
Framework – Apache Hadoop
13
![Page 14: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/14.jpg)
Use Case – Hawaii DOT
![Page 15: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/15.jpg)
HDOT RIMS Traffic Station Analyzer
15
• Live Stream Traffic Data
• Raw Data into Formatted Records
• Edit Station Data
• Identify Deviations (False Readings)
• Several Reports:
• 15 Minute
• Peak Hours
• Peak Volume
• Upstream-Downstream
• Oracle Apex used to Design
Interface and Analytics
![Page 16: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/16.jpg)
![Page 17: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/17.jpg)
Conclusion
![Page 18: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/18.jpg)
• All DOT’s Face the BD Issue
• IT Groups Understand Challenges
• Can Assist in Removing Silos
• BD Success is ACHIEVABLE !!
• Need Business Visionary/Champion
• New Thought Paradigm Required
• Technology Platforms can sustain BD
• Pick the Right “Jump Off” Project
Chief Takeaways
18
![Page 19: Plenary Session 3: IT Best Practices for - Home - NASTOnasto.org/wp-content/uploads/2017/07/IT-Best-Practices... · 2017-07-31 · Plenary Session 3: IT Best Practices for DOT’s](https://reader033.vdocuments.site/reader033/viewer/2022042307/5ed35f22e17a6756ef7671b8/html5/thumbnails/19.jpg)
Thank You
19