assured autonomy & building trust into ai systems · • alignment and agility of operational,...
TRANSCRIPT
Assured autonomy & building trust into AI systems
John J. Forte
A discussion on how T&E will play a role in driving assurance into tomorrow systems
2March 28, 2019
Some context on autonomy & building AI capabilities
3March 28, 2019https://www.nsf.gov/attachments/137968/public/nsf_v05.pdf
Importance of trust in enabling AI capabilities
4
Pattern Recognition
Knowledge Representation & Reasoning
Self-Evaluation & Self-Guided Learning
Creativity & Innovation
Perceive • Decide • Act • Team
TRUST
Stair Steps to Capability Realization
March 28, 2019
Importance of trust in enabling AI capabilities
5
Countering Disruptions to Capability Realization
TRUSTTRUST
Pattern Recognition
Knowledge Representation & Reasoning
Self-Evaluation & Self-Guided Learning
Creativity & Innovation
Perceive • Decide • Act • Team
Counter –Pattern Recognition
Counter –Knowledge Representation & Reasoning
Counter –Self-Evaluation & Self-Guided Learning
Counter –Creativity & Innovation
March 28, 2019
Trust through vulnerability assessments and adversarial AI
6
• Leverage APL vehicle laboratory• Assess Tesla auto pilot system• Create and test adversarial techniques for
ML
March 28, 2019
GROUND
Trust through test & evaluation and formal methods
7
ACAS XA
ACAS XU
TCAS II Non-cooperative
• Leverage APL research in national airspace collision avoidance
• Drive autonomy into legacy architectures• Validate future sensing and collision
avoidance approaches• Determine the scalability of formal methods
within an increasingly autonomous airspace
Airbus Blueprint for the Sky [2018]
March 28, 2019
AIR
Trust through test & evaluation and simulation
8
Transmission Area
No-Go Area
Mission Area1
Mission Area2
Mission Area3
Keyport
Barriers
Waypoint 2
Recovery Point
Launch PointWaypoint 1
Obstacle 1
Obstacle 2
Obstacle 3Obstacle 4
• Leverage APL research in autonomous unmanned underwater vehicles
• Developed the Range Adversarial Planning Tool• Set pass/fail boundaries for autonomous
performance• Investigate exploits and vulnerabilities in
autonomous operations
March 28, 2019
SEA
With more reliance on AI-enabled capabilities and autonomous agents, what will trust look like in the cyber domain?
9March 28, 2019
Design Open, Resilient & Zero Trust Architectures
Embrace Automation & AI-enabled capabilities
Increase Response Speed
Balance the Silicon-Carbon Ratio
Become more data driven
Driving trust and assurance in tomorrow’s highly autonomous world – T&E challenges and insights
10
Future ecosystems will be complex systems of systems environments with physical- and cyber-based autonomous agents… just a few factors for T&E to consider may include:
March 28, 2019
• Evaluating continuous internal and external updates to network infrastructure for assurance
• Alignment and agility of operational, business and safety requirements (e.g. ops suitability versus safety tradeoff analysis)
• Cyber-physical vulnerability and performance weakness analyses will need to expand
• Validating ‘machine reasoning’ congruent with human intent… learning and leveraging the field of ‘explainable AI’
• Getting simulations/models to run faster to support real-time and even predictive analysis (in simulation and the field)
Challenge:Enable the safety, trust and security of the convergence of the Internet of Things (IoT), Artificial Intelligence (AI), Machine Learning (ML) and robotics within real-world environments
Approach:- Leverage the world class cybersecurity, AI and autonomous systems research and analysis
capabilities at JHU to rapidly surge against this challenge- Establish an ecosystem across greater academia, working with government and industry
partnerships to make arguments in assurance
JHU Institute for Assured Autonomy
11
Assure the future autonomous world
Fundamental & Applied Research
Partnerships & Collaboration
Education & Workforce Development
Translation & Entrepreneurship
IAAIAA Functions
March 28, 2019
JHU IAA Research Areas
12
Assured Transportation Research Area: Resiliency for highly unsupervised autonomous transportation through all potential conditions of faults, attacks and failures Autonomous Courier Testbed
Assured Public Safety & Security Research Area: Assured intelligent and connected sensing and monitoring systems for the safety and security of citizens as well as critical campus (city) infrastructure
Autonomous Safety & Emergency Response Systems Testbed
Assured Health Systems Research Area: Trusted autonomous medical systems and human-machine teaming methodologies for efficient, reliable and private health services
• Adversarial AI & Cyber Red Teaming• Assurance Assessments & T&E• Software Assurance & Formal Methods• Social and Economic Impact Analysis
• Fall-Back & Fail-Safe Modes• Information Integrity & Privacy• Human Machine Teaming & Goal
AlignmentIntelligent Autonomous Health
Technology Testbed
Cross-Cutting Fundamental Research:
March 28, 2019
Many aspects can undercut the advantages of a future AI-enabled world
Trust, which includes elements of ethics, safety, privacy and performance, is a major factor for the adoption and scalability of autonomous and AI-enabled capabilities
Final thoughts
13
As leadership across government, academia and industry collide in the space of intelligent and autonomous systems, how can we work together in building and codifying trust and assurance as broadly accepted principles of AI system design?
March 28, 2019
What new and/or expanded thinking will we need in our T&E and V&V approaches in order to standardize best practices and drive trust and assurance?
Questions
14March 28, 2019
Detailed view of IAA functions
15
• Adversarial AI & Red Teaming• Assurance Assessments &
T&E• Software Assurance & Formal
Methods
• Fall-Back & Fail-Safe Modes
• Information Integrity & Privacy
• Human Machine Teaming & Goal Alignment
Transportation Systems
Public Safety & Security Systems Health Systems
Fundamental & Applied Research
APL WSE
Partnerships & Collaboration
KSAS Medicine SAIS Carey Bloomberg
Academia Government Industry
• Cybersecurity• Artificial Intelligence &
Machine Learning• Computer Science• Robotics
• Formal Verification• Data Science• Cognitive Science• Communications &
Networks
Workshops & Forums
Certificate Programs
Degree & Research Support
Education & Workforce Development Translation & Entrepreneurship
IAA
Inventions, IP Protection &
Commercialization
Policy & Economic Impact Translation
Innovation Funding Opportunities
JHTV
Entrepreneurship Tracks in Degree
Programs
Industry Cooperative
Programs
Societal Impact
March 28, 2019