when you can't beat 'em, join 'em · •leveraged fractals and chaotic systems for...

$: When You Can't Beat 'em, Join 'em · •Leveraged fractals and chaotic systems for advanced NN prototypes •Deep dive on chaos & complexity science. •Modeling & Simulation (DFS$
WhenYouCan'tBeat'em,Join'em:LeveragingComplexityScienceforInnovative

Solutions

Presentedatthe2017NAVAIRAdvancesinResearch&Engineering(ARE)TechnicalInterchangeMeeting

by:Dr.JosefSchaff,NAVAIR4.5

DISTRIBUTIONSTATEMENTA

• Commander’sintent:NetworkedNavy&theintentofCYBERSAFE• Cyberthreats=lackofresilienceforSoS,networks.• Weaklinksonautonomousvehicles• Challengeswithlargescalead-hocbattlespacenetworks

• Needs:• Dynamicallyadaptablecyberresilience• Threatsmayuseautonomous(e.g.machinelearning)adaptation.• Collectivebehaviors,e,g,swarms.• Novelapproachmayneednovelmathematicsasfoundation.• Fundamentally,acomplexadaptivesystem.


CurrentProblemDomain

• BooksbyMoffat,Alberts,published2000-2003 describeaspectsoftheNet-CentricBattlespaceneededforNCW(Net-CentricWarfare):• Hasattributesofself-similarity(fractalnature)• Involvesthousandsofentities(networknodes)• Answersmayliesomewherewithincomplexityscience/chaostheory

• Asolutionwouldneed:• Adaptivedynamicbehaviorsforresiliency• Scaleupwardsatleastseveralordersofmagnitude• Becomputationallytractable• Convergetosolutioninshorttimeframe(millisecondstoafewseconds)


HistoricalProblemDomain:Net-CentricityanditsProblems

Autonomy

Architecture&Topology

Cyber

• A.I./M.L.• Emergentattributes

• Hierarchical• Self-similar(Fractal)

• Resilience• Adaptability

ComplexityScience:deterministic/non-deterministicchaos


Fieldsofstudyandtheiroverlap

• Physicsundergrad,softwareengineeringjobsincomms,videogames,robotics• StartedNAWCAD(NADC)asacomputerscientist/engineerresearchingNeuralNetworks(NNs)andmathematicalmodelingofphysical&biologicalphenomena• A.I.Branch– broadenedmyfocusonmachinelearning,alsohadopportunitiestoapplyNNstoreal-worldNavyproblems• Noticedneedfordistributedarchitectures&emergentphenomena• LeveragedfractalsandchaoticsystemsforadvancedNNprototypes• Deepdiveonchaos&complexityscience.

• Modeling&Simulation(DFSCentrifuge)developedexpertiseindistributednetworksandgraphicalsoftware• Privatestart-up“bigdata”focus,wasdirectorofresearchfocusedonsemantics,fractaltopologiesandgeneticalgorithms• M&S–ACETEF,software,specificfocusonalgorithms• 2010-now:cyberengineering,autonomy&MachineLearning,advancedarchitectures


Whatshapedmyperspectiveontacklingtheproblem

• Complexityscienceisinformallyknownasordercreationscience.Novelcoherentpropertiescanresultfromself-organizingSystemofSystems(SoS).Collectiveactionsofmanyentitiesinasystemproducesemergence.• TherearevariousmethodstocreatecomplexSoS andemergence,forexample:

• Newapproachesincomputational(experimental)mathematicsformulti-agentsystems.• Deterministicchaos(fractals).• Pecora &Carroll’sresearchoninformationembeddedbelowchaoticnoisethreshold,similarchaoticcircuitcan“decrypt”signalfromnoise.

• ApplicationFocus:Cognitiveroboticsincorporatesthebehaviorsofintelligentagentswithinthesharedworldmodel.• Multi-agentsystemscreatechallengesfordesiredbehaviorswithinaplannedenvironmentdueinparttotheproblemoftranslatingandusingsymbolicreasoningforworldabstractions.

• EventhelowestleveldistributedC2(Command&Control)comms canproducecomplexity.


Whatiscomplexityscience?

• Emergentbehaviorsresultnot fromstochastic(e.g.thermodynamics)models,butinsteadfrommulti-agentinteractions(e.g.RoboCup).• Emergencecanproduce‘creative’systembehaviors.• ArtificialLife- usesemergencegeneratingalgorithms:

• geneticalgorithms,neuralnets,cellularautomata.• E.g.“TheSims”usesgeneticalgorithmsforautomata.

• EmergentSoS cannot bedesignedbyfunctionaldecomposition.• Nonlinearsystems:Cantheyhavepredictablebehavior?

• Predictability‘collapses’assequenceprogresses(complexityincreases).• Chaoscanresultfromevensmallchanges.• Knowninitialandintermediateconditions canhaveunpredictableresults=Emergentbehavior.


EmergentBehavior:whatisit?

• Why?• SystemsengineeringislimitedbyitscurrentSystemofSystems(SoS)approachtoconsistentlypredictnovel/emergent behaviorsthatwouldgivetheU.S.anedgeonouradversaries.

• Large-scalemulti-agentSoS,whicharecomplexsystems,typicallyshowemergentbehaviors.

• Collectiveactionsofmanyentitiesinasystemproducesemergence.• Complexitycanprovideasolutiontotranslatingtheworldintoactions,byboundingthebehaviorsofdistributedagentstoproducenew(emergent)anddesiredcollectivebehaviors.

• How?• Systemelementsneedtobemoreadaptable,looselycoupled,andcreateadynamicallyinteroperableenvironment.

• Complexityscienceisbettermodeledbyusingalocalized,connectionistontologyofheterogeneousagentsthanbyusingequilibriummodelsfromthermodynamics.

• Novelcoherentpropertiescanresultfromtheseself-organizingsystems


Whyshouldweusecomplexityscience&how?

• Consistsofmanycomponentsassociatedbystructureorjustabstractrelationship.• Maybescalableandself-similaratmorethanonelevel.• Notdescribedbysimpleruleorfromthefundamentallevel.Predictablepartscanformunpredictablesystembehavior.• E.g.Mandelbrot (fractal’sinventor):“transmissionlinenoise”appearedrandom,waspredictable“CantorDust”.• Bifurcation- “Feigenbaum diagram”atphasetransitions(solid/liquid/gas),etc.representsnonlineardropoff.• Devil’sstaircase– atphasetransition=chaos.


WhatisaComplexSystem?


Diagrams: Feigenbaum and Devil’s Staircase

• Mostbodyfunctionsexhibitcomplexbehavior-fractalpatternofheartbeat,ionicchannels,etc.• whenECGpatternbecomesless complex,thenindicatespotentialheartproblem!!

• Chaotic(complex)chemicalreactions:• Belousov-Zhabotinskii reaction(colorchange)

• Canevenbuildanelectroniccircuitwithcomplexbehavior- canbedriventochaotic• Canwecontrolchaos?


ComplexityinOtherRealms

©2016,SchaffConcepts,LLC DISTRIBUTIONSTATEMENTA

Chaosrules!

Generalizedconjectureonchaos:• Simpledeterministicorevenrandomstochasticmodelsmaynotbetheanswerinourquestforhuman-likebehaviors,oreventheself-organizingpatternsthatoccurinnature• Perhapsweshouldlooktocontrollingchaoticphenomena,asnaturedoes,forthediscoveryofemergentpatterns.Thismayleadtosolutionsforself-organizinglargescalenetworks,orevenhuman-likebehaviorinrobots

©2016,SchaffConcepts,LLC DISTRIBUTIONSTATEMENTA

Wait…what?Chaosisgood???

Self-OrganizingComplexSystems:ChaosUnderControl

•Artificialbiologicalsystems:• Neuralnetworks,Geneticalgorithms,Booleannets(Kauffman),CellularAutomata(Wolfram).

•Realbiologicalsystems:• Civilizations,economies,evolution(Kauffman),biologicalorganisms,cognitivethoughtprocess.

• Experimentalmathematics:• A“new”typeofmathematics,previouslyunexploredduetocomputationallimitationsofthepast.• Not FormalMethods,andnoavailableproofs.• Maydependupondeterministicchaos.


Controlofchaos– anexample

Problem:Spatiallydistributedlargedynamicnetworks:• Loseedgenodecommunications.• CongressionalResearchReport(2007):

• Scalinglimitationsforlargenumbersofbattlespacenetworkednodes.• Combinatorialexplosionfrommassivenumbersofroutecalculations.

• Toincreaseavailabilityandresiliency innetwork-centriccloudsandswarms,ad-hocnodesmustrapidlyself-organizeusingsharedtopologydata.

• Topologycanaffectnetworkfailures andsuccess ofcyberoffenseanddefense.Perhapswecanleveragecomplexityscienceforasolution:

• Moffat's2003papertitled"ComplexityTheoryandNetworkCentricWarfare"referencedcomplexsystemsandtheirrelationshiptofractalsanddecentralizedNCW.

• Highvolumenetworktrafficpacketsself-organizetofractal(Leylandetal.,1994),thereforefractalmayincreaseavailabilityforlargenetworks.

• Useafractalthatcanadapttoneededtopology.


Adaptivefractalexperimentalmathdiscovery:anoutgrowthofthelinearchaosgame

Likethesimplepoint-slopeequationforline:• DeterministicchaosequationisX(n)=M*X(n-1)+Z.X(n-1)=currentpoint, X(n)=nextpoint.Z:“vertices”=asetofinitialpointsthatconstrainallnodepoints,canrepresentnetworkhubs.Z israndomly selectedoutofthisset.M: scaleparameter=controlswherethenextpoint isgeneratedfromthecurrentpoint.0<|M|<1.

BothvariablesM andZshareinterdependenciesthataffecttheoverallnetworktopologies,includingthresholdsforclusteringandthemappingstocertainclusterelements.


NamingthealgorithmandusingtheresultsAlgorithmName: Non-predeterminedParametricRandom(NPPR)IteratedFunctionSystem(IFS)Runningit:• Nodeandhubconsiderations:

• Pointsplottedshowdistributionofnetworknodes;vertices=hubs.• Hubsmaybevirtual,i.e.locationforcalculationpurposesonly,andcanadd,move,delete.• Nodesknowrelativelayoutofclusters,coalescearoundhubsforcommunicationsclusters.

Results:• CombinatorialexplosionandcyberimpactavoidedbyuseofNPPR.

• Usuallyisanissueinlargead-hocnetworks(Adams&Heard,2014).• NPPRtopologyisinformation-dense: alittleinfocanreconfigurenetwork.

• Hubchangesbroadcastedaslat/lon position.• Scaleparameterchangesfromchaostoorder.

• Producesrepeatablemacroscopicresults,evenwithuniquenodepositions• Canapplytolarge-scaleswarmcontrol,adaptivecyberwarfare.• Sharedstigmergic knowledgebyallnodes– i.e.eachknowspositionof“neighborhoods”


• Solutionis:• Self-similar– eachnodecan“know”thetopologyrelativetoothernodes• Facilitatessituationalawarenessfortensofthousandsofdistributednodes• UsesDeterministicChaos

• Solutionhas:• Adaptivefractaltopologywithdynamicbehaviorsforresiliency• Fractalself-similarity canscaleupwardsmanyordersofmagnitude• Linearequation=likepoint-slopeequationoflineiscomputationallytractable• Convergestosolutioninshorttimeframein10-100millisecondtimeframe• Exhibitsstigmergic behaviors

• Thisisbutonepossiblesolutionoutofmany,thatcanbediscoveredbyusingcomputational(experimental)mathematics


Attributesofthissolution

• Usedasmysuccessfullydefendeddissertationtopic• Discoveredinterestingemergentbehaviorsinasimpleequation• Received2015OutstandingWorkforceDevelopmentAwardasadirectresultofthisacademicresearchproject• WroteachapterforengineeringbookonEngineeringEmergence


PersonalConsequencesofthisResearch

ScreenlayoutofNPPR“tool”:


FromRandomtoOrder


MorePatterns


Changingthesign(+/-)


Somediffering4-vertexpatterns


Someofthereferences

• Stigmergy:• Lemmens andTuyls (2010)suggestedstigmergy forroutingprotocolsissues.Masoumi andMeybodi (2011)showedrelationshipofsharedinformationtostigmergy.

• NetworkTopology:• Kleinberg,etal.(2004)showedtopologyaffectsnetworkfailuresaswellasattacksuccesses.

• FractalTrafficSelf-organizing:• Paxson andFloyd(1995).


when you can't beat 'em, join 'em · •leveraged fractals and chaotic systems for...

Documents