IOT - Testing Trends

Welcome note:

about me

Nalinikanth
Quality Analyst @ThoughtWorksTwittter - @nalinikanth99about.me/nalinikanth

What do youExpect

Expectations...?

Take aways

Basic Knowledge on IOT.

Performance and Security of IOT.

How to test IOT.

Challenges in IOT Testing.

Challenges @ every slide

Smart Home

Explanation of how IOT works at home.One pattern from wake up to car

Patterns

Things to things communicationPeople process and things
Data and tesla example.--A sensor is not a machine. It doesntdoanything in the same sense that a machine does. It measures, it evaluates; in short, it gathers data. The Internet of Things really comes together with the connection of sensors and machines. That is to say, the real value that the Internet of Things creates is at the intersection of gathering data and leveraging it. All the information gathered by all the sensors in the world isnt worth very much if there isnt an infrastructure in place to analyze it in real time

IOT - Ecosystem

Ecosystem Users - the layer of usersNetwork things Where everything is connectedApplications The applications that user use to monitor the IOT systemsThings devices that are connected (Refrigerator, smart watch)

Scary Stories

Car stopped in a fast lane.

Cars drive themselves to thieves Disappearing cars.

Criminals steal your Amazon deliveries.

Self driving car met with accident and people died.

Thanks to my ISP, it had a fiber cut. so, no water today.

AC is not working as there an issue with router.

Testing Scope

Functional Excepted behavior vs actual behavior.Network Where they are connected(Wifi, bluetooth, Intra network or Internet)
How the network fluctuations might effect there(A self driving car lost network connection )Performance Communication happens b/w devices in RTOSSecurity Getting access to home (Data, Privacy is involved)Compatibility Applications either mobile or web. You cannot ask your user to upgrade to android 7 as app doesn't support.Exploratory Unlike classical system use cases are very more Testing the system as a user

Testing IOT

What a classical system is?Testing doesn't break things. when it is at your place it breaks when it is in customers hands Even if you do all the traditional types of testings.

One Scenario

Agriculture based system explain how it works,

One Scenario

Component wise testingvirtualization of sensors Tesla car example.In computer science, test stubs are programs that simulate the behaviors of software components (or modules) that a module undergoing tests depends on.

Performance Testing of IOT

Device to Device Communication.

Network bandwidth, latency & packet loss.

Device to server communication.

Interruptions in network.

Thing to Thing RTOS (Sensor says an obstacle at 50M distance car should stop or slow down there should be no delay)

Multiple request handling.

Synchronization.

How the system behaves at low band width, what happens if there is any packet loss

Hardware & power.

How should I power my nodewall adapterinstall wiringuse batteries(any idea how much life is expected from a water meter battery) Can I transfer all data back to cloudDo we have bandwidth for all sensorsRouting equipmentInternet connectivityWill my node work if internet connection is lost?will street lights work if internet is down?Thanks my ISP had a fiber cut so no water todaymy ac is not working is there any problem with router

Security breaches

Disrupt services at home, refrigerator sends spam.Apply breaks suddenlyShow wrong stats or remove stats.

Steal network credentials and take personal data.Steal the information of the car hardware and other stuff.Steal health data or daily data

Take control of your home control electrical devicesTake control of the car. Take it to thieves
Inject high insulin to body.

Device memoryEcosystemPhysical InterfacesNetwork trafficHardwareAuthentication and Authorization

Attack surface areas

Device memory -Cleartext usernames,Cleartext passwords, Third-party credentials,Encryption keys

Ecosystem -Interoperability standardsData governanceSystem wide failureIndividual stakeholder risks
Physical Interfaces - Firmware extraction, User CLIAdmin CLI,Privilege escalationReset to insecure stateRemoval of storage mediaTamper resistanceDebug portDevice ID/Serial number exposure
Network traffic LAN, LAN to Internet, Short rangeNon-standard, Wireless (WiFi, Z-wave, Zigbee, Bluetooth),Protocol fuzzing

Hardware - Sensing Environment ManipulationTampering (Physically), Damaging (Physically)

Authentication and authorization - Authentication/Authorization related values (session key, token, cookie, etc.) disclosureReusing of session key, token, etc.Device to device authenticationDevice to mobile Application authenticationDevice to cloud system authenticationMobile application to cloud system authenticationWeb application to cloud system authenticationLack of dynamic authentication

Should we secure the system ??

Secure by design.Secured technology, process, and people.Test for security.Emphasize security from day one.Lifecycle, future-proofing, updates.Access control and device authentication.Know your enemy.Prepare for security breaches.

Testing Table

A table like this where in having testing types and Components across will to understand what testing has to be applied at what parts of the system.

Testing process

Rethinking the
Testing Process

Sensor Monitoring
&Hardware Testing

Security & Performance

Real time scenarios

Getting the UX
right

Finally.....