data science and nato’s needs: better decisions, faster · •nato can provide the needed...
TRANSCRIPT
Data Science and NATO’s needs:
Better decisions, faster
Peter Lenk, Chief Strategy and Innovation, Service Strategy [email protected]
Michael Street, Head, Innovation and Data Science, Service Strategy Directorate,[email protected]
Data Science is key to ongoing efforts
• Volume & Velocity:
• NATO Data in Afghanistan: rate ~ 1TB per week
• NATO Alliance Ground Surveillance: rate ~ 1PB per week
• Velocity:
• Cyber attacks happen in very short periods of time; human decision making may be too slow in many situations
• Veracity:
• Variety of sources: Open Sources (Real and Fake news), coalition partners, IoT sensors, smart city sensors, etc.
• All these have different provenance - How do we deal with this known (or unknown) uncertainty?
• Variety:
• Structured / Unstructured: binary, databases, imagery, chat, email, portals, PowerPoint, Word documents, voice, …
• How do we correlate this information, across the various sources?
An Essential Enabler
• These technologies have potential to reduce workload, and analyse the information across sources and types
• They have the potential to help make better, more timely decisions
• Classifying records from theatres
• Sentiment analysis of open source intel / detecting adversarial StratCom
• Intel product analysis
• Analysis of exercise data
• Clustering ISAF civilian survey responses
• Natural language processing of content of NATO documents
• “Learning” translation between protocols
• Clustering IT incidents
• Architecture and Capability visualization & dependencies
• Polaris – application migration assessment
NCIA supports NHQC3S, ACT, JALLC, STO, ACO and internal Agency activities
CMRE also active in big data analytics
Application of Data Science at NCI Agency
Shared Data Services
Core Data Services
Exploitation Tool Services
Data Science Services
End-Users
A Layered Model for Data Science Services
• The central layer will consist a data lake, one that is continually growing
• Not limited to a ‘centralised’ model. Data will of necessity be distributed; however, it should still be considered as part of the Shared Data
• Commercial or open source data subscriptions should be brokered to ensure consistency, avoid duplication and provide cost efficiency
• Shared Data Services ≠ an Archive
• Relies on infrastructure that can store and retrieve data at rates that are in line with the needs of the source or of the subsequent layers
• Business model:• Because data will be NS, any infrastructure will be owned by NATO• The actual running of the services could be outsourced to an on-premises provider, perhaps
bundled with services at other layers• NATO would charge for the services, on a per-use basis; i.e., a Data as a Service model (DaaS)
Layer 0 – Shared Data Services
• Trusted services that support data collection, data access management, and curation.
• The intent is to allow access to the data, under controlled conditions, but not to release copies of the data.
• Services needed:• Data collection services• Curatorial services, data indexing, quality indicators, provenance information, etc. • A single point of entry into the entire data set, wherever the data actually resides.• Access management services to ensure that trusted users are authorised to access specific items of
data or to introduce new data.
• Business model:• NATO can provide the needed services organically, • The services delivered might be outsourced via a NATO-Owned Contractor-Operated (NOCO) model,
perhaps in conjunction with the Layer 0 services, or • A hybrid where NATO organically provides some services and eco-system partners can provide others.
Layer 1 – Core Data Services
• A library of tools and infrastructure, exposed to analysts as services, allowing exploitation of the available data.
• Data preparation tools, • Normalisation tools, • Machine learning tools, • Big data tools, • Artificial intelligence tools, • Infrastructure suited to the conduct of data science work, etc.
• NATO will need to own any infrastructure.
• Tools can be open-source, NATO-owned, or commercial
• Business Model:• Software and Infrastructure as a Service (SaaS, IaaS),• NATO may host some tools in this environment and offer these to the eco-system as SaaS, • Other eco-system partners will want to introduce services at this level
• Using IaaS, for their own purposes, or • Providing additional SaaS, to sell their tools to the broader community.
Layer 2 – Exploitation Tool Services
• Provide a variety of value added services that allow exploitation of data. • Using tools provided in the Exploitation Tool Services, accessing data in the Shared Data Services layer
through the Core Data Services layer.
• Specialised data scientists will provide the services
• Besides data scientists, important will be access to individuals with domain knowledge that can bring understanding of the technology and data used in the domain.
• The services can be:• ‘Standard’ conducting repetitive analysis on data• Custom to answer specific questions.
• Business Model:• Hybrid• Organic NATO data scientists will be needed, • Domain level experts will be needed, and • Eco-system partners can provide services.
Layer 3 – Data Science Services
• End-users, form a vital role in such an environment. • They have questions to answer, processes to support and decisions to make.
• Solutions may be iterative and will mature• ‘citizen data scientists’ can access Layer 2 and below, maybe supported by
Layer 3• Digital literacy of end-users will increase• The value of this layer was well articulated by Dr Theodore von Kármán:
• Scientific results cannot be used efficiently by soldiers who have no understanding of them, and scientists cannot produce results useful for warfare without an understanding of the operations.
• Business Model:• Use the Data Science stack to generate new services and new end-user tools
Layer 4 – End-Users
Shared Data Services
Core Data Services
Exploitation Tool Services
Data Science Services
End-Users
A Layered Model, a Collective Win
Agency Eco-system
Data Science and NATO’s needs:
Better decisions, faster
Peter Lenk, Chief Strategy and Innovation, Service Strategy [email protected]
Michael Street, Head, Innovation and Data Science, Service Strategy Directorate,[email protected]