osmius 9 - ucmantares.sip.ucm.es/tarot09/index_files/osmius_tarot09.pdf · integrated data...

78
Osmius 9.07 The OpenSource Monitoring Tool José Luis Marina :: 2009 Osmius 9.07 You can't manage what you can't see.

Upload: ngohanh

Post on 19-Sep-2018

215 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Osmius 9.07The OpenSource Monitoring Tool

José Luis Marina :: 2009 Osmius 9.07

You can't manage what you can't see.

Page 2: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

2Osmius 9.07: Professional Monitoring July 2009

What is Osmius?

Osmius allows you to monitor and keep track 

of everything connected to a network

Page 3: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

3July 2009

What is Osmius?

 We've designed Osmius to monitor:Computers and Network devicesApplications and DatabasesUser transactions...

 But also:Industrial sensors.Energy consumptions.Pollution levels in the neighbour.Stock shares.Any idea?

Osmius 9.07: Professional Monitoring

Page 4: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

4Osmius 9.07: Professional Monitoring July 2009

Why choose Osmius?

Briefly, with Osmius you can:

 Monitor thousands of devices in Real Time. Align IT with the Business (SLA Management). Data Mining and Business Intelligence. Use the power of GIS to see/analyze data. Monitor new “things” in minutes. Send automatic notifications to the staff. Predict Capacity shortages (ITIL). Access ALL the code and documentation.

Page 5: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

5Osmius 9.07: Professional Monitoring July 2009

Who is behind Osmius?

Peopleware: Created in 2000

~ 30 people~ 2,000,000 € Last year billing

 Services: Administration, Tuning and IT OperationsSoftware Development. Performance Demanding.Free Open Source Software.Information Security.R&D in collaboration with several Universities.

 More:UNE­EN­ISO/9001:2000 in IT Consultancy & Operations

PeoplewareFunds to Help R&D

Investments

Technological Partners

Geographic Systems

Business Intelligence

Osmius Partners

InstallationAdministration

SupportTraining

DeploymentDevelopment

MultiplatformInstallers

Page 6: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

6Osmius 9.07: Professional Monitoring July 2009

Who is behind Osmius?

Community

 In Osmius everything is Open... not just the “core” !!

First Stable version released in July 2008More than 4.000 downloadsSourceForgeRanking : ~ Top 50­100

Page 7: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

7

MA

AG1

AGn

MA

Osmius 9.07: Professional Monitoring July 2009

Architecture & Tecnology

With Osmius you can:

 Monitor thousands of devices in Real Time. Align IT with the Business (SLA Management). Data Mining and Business Intelligence. Use the power of GIS to see/analyze data. Monitor new “things” in minutes. Send automatic notifications to the staff. Predict Capacity shortages (ITIL). Access ALL the code and documentation.

AG1

AGn

Master AgentPoint from which you want to monitor and deploy Agents.You can use 1 to N

MA

AG1

AGn

CS

Central ServerProcess to receive andcorrelate events andto manage all the agents

MySql

SSL

DataBaseOpen and documentedDataModel.Events, SLA, etc

TomCat

J2EE

WebConsoleJ2EE connecting tothe DBase.

InstancesSomething you want tomonitor (Web, DB, Temp)

AgentSpecialized on monitoringspecific instances likeMySQL, Web sites...

Page 8: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

8

MA

AG1

AGn

MA

Osmius 9.07: Professional Monitoring July 2009

AG1

AGn

Master AgentC++ & ACEMultiplatformPersistent QueuesFault Tolerant

MA

AG1

AGn

CS

Central ServerC++ & ACE & MySql APIFast insert and CorrelationSecure Comms

MySql

SSL

DataBaseNormalizedExplain everythingStored Procs

TomCat

J2EE

WebConsoleJava J2EESprings – HibernateOpenLayers

InstancesUse APIReuse Connections

AgentC++ & ACEAPI (MySql, Curl,etc)Only new code

Architecture & Tecnology

Page 9: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

9Osmius 9.07: Professional Monitoring July 2009

Concepts: Easy to understand

Instance: Everything you want to monitor.

Instance Type or Class: Defines the kind of questions you can make to the instanceMySQL Database | Windows Server | Exchange

Instance Events: Depending on its type an Instance can receive certain eventsMySQL Database: # Users, Free Space, Kbytes Out,...Parameters: Every 10 seconds – Customized Warning & Critical Threshold

Event 1 :: Database Connections is: 250Event 2 :: Uptime is                         : 8h 30'

Event 1 :: Time to load Web is        : 15 s

Event 1 :: CPU Load is                    : 80%

Page 10: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

10Osmius 9.07: Professional Monitoring July 2009

Concepts: Easy to understand

Correlation Engine I:The Active Events view should be as clean as possible:

Show only 1 row for repeated events.Automatic clean “bad” events when arrives a “good” one.If Temperature is OK clean previous CRITICAL events.

Active andHistorical

ViewsAccumulateOnly 1 row

Correlation Engine II (*)Identify dependencies between Instances:

If this servers fails   DataBase and WebServer also fail.→Manual and automatic identification.

Page 11: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

11July 2009

Concepts: Easy to understand

Service: A group of instances joined to offer a service to some usersNormally represents some functional group into your business

The Intranet...Will be down this afternoon from 15:00 to 17:00due to the upgrade of the Oracle Database from 9i to 10gDon't take this unavailability into account in the SLAs

SLA – Service Level Agreement: Defines objectives to your services.

      The Intranet cannot be down more than 99.999 of its timeService Working Time 

Are we using the Intranet 24x7?     Services must accomplish SLAs within their Working Time     Osmius also implements: Planned DownTimes

Osmius 9.07: Professional Monitoring

Page 12: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

12

Firewall

July 2009

Concepts: Easy to understand

IntranetDB

SLA – Service Level Agreement: Defines objetives to your services.

      The Intranet cannot be down more than 99.999 of its time

Service Working Time Are we using the Intranet 24x7?

     Services must acomplish SLAs within their Working Time     Osmius also implements: Planned DownTimes

IntranetIntranet

GoldGold SilverSilver

e­Come­Com eMaileMailCRMCRM

99.999 Up99.900 Ok

99.999 Up99.900 Ok

Osmius 9.07: Professional Monitoring

24x7 24x7 24x7

HostMoon

ExchangeServer

Exch.FrontE

WinHost

...

From M to FFrom 9 to 17

SLAManagement

ServiceLevel

Instance &TechnicalLevel

Events sent by the agents

RulesPropagation

Page 13: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

13July 2009

Concepts: Easy to Manage

Osmius 9.07: Professional Monitoring

Manage the Configuration: Distribute changes from the console to the distributed agentsDistribute or upgrade agents with one click.

Create your own...Events using your customized scripts and distribute them.Agents: Calling Peopleware or building them yourself using the 

Osmius Development Framework.

Page 14: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

14July 2009

Integration

How to deal with millions of events?  Round Robin Database for the individual events.

You don't need to know wich was the exact CPU load on 01­01­2008 at 8:00.The older the data the lesser the detail you need.

Osmius 9.07: Professional Monitoring

 Integrated Data WarehouseSpecific Osmius processes to aggregate daily info (seconds in each state, etc)

 Silent ModeEvents in “silent mode” are only sent when there are changes in the Severity.“Send me the Temperature only when it raises 30 C”Prevents network and storage resources starvation.

User Notificacions & SubscriptionsLet Osmius inform you about changes.

e­mail

SMS....

Service Availability Changes

Instance State Change

Subscribe to Reports

Working time

Not Working time

Out of officeJabber

Global Mark

Sending Osmius Event to othersStandalone Agents: Integrate the agents directly with e­mail, Nagios, …Open Tickets automatically using notifications.Send Exchange Critical Events to another Monitoring system.

IntranetIntranet

eCommeComm

Receiving Events from othersWeb ServicesCommand Line Interface

whenusing

Page 15: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Introducción::ÍndiceReinforcement Learning Bots

15July 2009

Osmius Advantages

Osmius 9.07: Professional Monitoring

Deal with thousands of Instances and millions of events.Integrates SLA management and ITIL best practices.Manage everything from a Central Point.Business Intelligence mining tool.Leverage the power of GIS.Real Open: There is no community downgraded version.You can influence the RoadMap.Well documented. We care about documentation.

Page 16: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

16

Introducción

Osmius is supported by:Ministry of Science and Education

Ministry of Industry and TradeCDTI

European Funds

Next Step: Osmius and Passive Testing

Page 17: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

What is PasTe?

PasTe: Passive Testing tool.

A library to perform passive testing approach for timedsystems.

Integrated into Osmius as an agent.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 17

Page 18: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

What is PasTe?

PasTe: Passive Testing tool.

A library to perform passive testing approach for timedsystems.

Integrated into Osmius as an agent.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 17

Page 19: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

What is PasTe?

PasTe: Passive Testing tool.

A library to perform passive testing approach for timedsystems.

Integrated into Osmius as an agent.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 17

Page 20: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Passive testing

In testing, there is usually a distinction between two approaches:

Active testing.

We can interact with the system (apply tests).We have an assesment of correctness before the system isrunning.

Passive testing.

The system is already running.We cannot interact directly with the system.We can observe its traces.

Example

Large systems working 24x7 since this interaction might produce awrong behavior of the system.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 18

Page 21: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Passive testing

In testing, there is usually a distinction between two approaches:

Active testing.

We can interact with the system (apply tests).We have an assesment of correctness before the system isrunning.

Passive testing.

The system is already running.We cannot interact directly with the system.We can observe its traces.

Example

Large systems working 24x7 since this interaction might produce awrong behavior of the system.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 18

Page 22: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Timed Framework I

Specifications0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

Invariantsφ1 = login/option screen/[20, 40], ?/[0,∞],

disconnection 7→ {welcome screen}/[10, 20] B [35,∞]

φ2 = login 7→ {option screen, error user}/[10, 40] B [10, 40]

φ3 = data/profile screen/[10, 20], save 7→ {option screen}/[20, 40] B [35, 50]

φ4 = ?/option screen/[5, 35], marks 7→ {marks screen}/[20, 40] B [30, 70]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 19

Page 23: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Timed Framework II

Implementation

Traces

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 20

Page 24: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Time Finite State Machines I

Extension of the classical finite state machine model.

The main difference with FSMs is the addition of time.

M = (S , I,O,Tr , sin)

S is a finite set of states

I is the set of input actions

O is the set of output actions

Tr is the set of transitions

sin is the initial state

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 21

Page 25: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Time Finite State Machines II

Transition: tr = (s, s ′, i , o, t)

s, s ′ ∈ S are the initial and final states

i ∈ I is the input action

o ∈ O is the output action

t ∈ IR+ is the time that the transition needs to be completed

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 22

Page 26: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Specification of SSadmin

s0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

i1= connecti2= logini3= disconnectioni4= profilei5= datai6= canceli7= savei8= marksi9= register

i10= data subjecti11= save registrationi12= return optiono1= welcome screeno2= option screeno3= error usero4= profile screeno5= marks screeno6= register screeno7= confirm screeno8= no confirmation screeno9= not spected

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 23

Page 27: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Usual interaction between a student andSSadmin I

Welcome page

The student wants to connect to the system.

She connects to the web page and the system shows thewelcome screen.

s0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

i1= connecto1= welcome screen

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 24

Page 28: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Usual interaction between a student andSSadmin II

Logging into the system

When the student sees the welcome screen she can login into thesystem. If an erroneous login is introduced then the system showsthe error user.

s0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

i2= logino2= option screeno3= error user

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 25

Page 29: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Usual interaction between a student andSSadmin III

Interaction

If the student introduces a correct login, then the system willshow the option screen (in 30 seconds).

If the student wants to disconnect, then she only has to pressdisconnection and SSadmin returns to thewelcome screen (in 15 seconds).

s0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

i2= logini3= disconnectiono1= welcome screeno2= option screen

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 26

Page 30: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Specifying properties: Invariants

Each time that a user applies a and observes z and the time required by thetransition verifies the restriction p.

Some actions are performed and the time required verifies the time restriction p.

If after performing some operations the user inputs i then he must observe andoutput in O, the time required by this transiction satifies the restriction p andthe time for all the sequence satisfies the restriction t.

Formal Definition of Invariants

φ is defined according to the following EBNF:

φ ::= a/z/p, φ |

? /p, φ′ | i 7→ O/p B tφ′ ::= i/z/p, φ | i 7→ O/p B t

p, t ∈ IR, i ∈ I, a ∈ I ∪ {?}, z ∈ O ∪ {?}, and O ⊆ O.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 27

Page 31: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Specifying properties: Invariants

Each time that a user applies a and observes z and the time required by thetransition verifies the restriction p.

Some actions are performed and the time required verifies the time restriction p.

If after performing some operations the user inputs i then he must observe andoutput in O, the time required by this transiction satifies the restriction p andthe time for all the sequence satisfies the restriction t.

Formal Definition of Invariants

φ is defined according to the following EBNF:

φ ::= a/z/p, φ | ? /p, φ′ |

i 7→ O/p B t

φ′ ::= i/z/p, φ | i 7→ O/p B t

p, t ∈ IR, i ∈ I, a ∈ I ∪ {?}, z ∈ O ∪ {?}, and O ⊆ O.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 27

Page 32: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Specifying properties: Invariants

Each time that a user applies a and observes z and the time required by thetransition verifies the restriction p.

Some actions are performed and the time required verifies the time restriction p.

If after performing some operations the user inputs i then he must observe andoutput in O, the time required by this transiction satifies the restriction p andthe time for all the sequence satisfies the restriction t.

Formal Definition of Invariants

φ is defined according to the following EBNF:

φ ::= a/z/p, φ | ? /p, φ′ | i 7→ O/p B tφ′ ::= i/z/p, φ | i 7→ O/p B t

p, t ∈ IR, i ∈ I, a ∈ I ∪ {?}, z ∈ O ∪ {?}, and O ⊆ O.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 27

Page 33: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ1

If we see in the log the input login followed byoption screen and an amount of time included in [20,40],...

and we observe any sequence of input/output/time and thenthe input disconnection ...

... then if we observe disconnection we have to see theoutput welcome screen in an amount of time belonging to[10, 20].

... In addition, the sum of all time values observed from thelogin until the welcome screen output belongs to [35,∞].

Invariant

φ1 = login/option screen/[20, 40],

? /[0,∞], disconnection7→ {welcome screen}/[10, 20]B [35,∞]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 28

Page 34: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ1

If we see in the log the input login followed byoption screen and an amount of time included in [20,40],...

and we observe any sequence of input/output/time and thenthe input disconnection ...

... then if we observe disconnection we have to see theoutput welcome screen in an amount of time belonging to[10, 20].

... In addition, the sum of all time values observed from thelogin until the welcome screen output belongs to [35,∞].

Invariant

φ1 = login/option screen/[20, 40],? /[0,∞], disconnection

7→ {welcome screen}/[10, 20]B [35,∞]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 28

Page 35: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ1

If we see in the log the input login followed byoption screen and an amount of time included in [20,40],...

and we observe any sequence of input/output/time and thenthe input disconnection ...

... then if we observe disconnection we have to see theoutput welcome screen in an amount of time belonging to[10, 20].

... In addition, the sum of all time values observed from thelogin until the welcome screen output belongs to [35,∞].

Invariant

φ1 = login/option screen/[20, 40],? /[0,∞], disconnection7→ {welcome screen}/[10, 20]

B [35,∞]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 28

Page 36: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ1

If we see in the log the input login followed byoption screen and an amount of time included in [20,40],...

and we observe any sequence of input/output/time and thenthe input disconnection ...

... then if we observe disconnection we have to see theoutput welcome screen in an amount of time belonging to[10, 20].

... In addition, the sum of all time values observed from thelogin until the welcome screen output belongs to [35,∞].

Invariant

φ1 = login/option screen/[20, 40],? /[0,∞], disconnection7→ {welcome screen}/[10, 20]B [35,∞]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 28

Page 37: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ2

After observing any occurrence of login in the log,

then we have to observe option screen or error user.

The amount of time associated with this input/output mustbelong to the time interval [10, 40].

Invariant

φ2 = login

7→ {option screen, error user}/[10, 40] B [10, 40]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 29

Page 38: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ2

After observing any occurrence of login in the log,

then we have to observe option screen or error user.

The amount of time associated with this input/output mustbelong to the time interval [10, 40].

Invariant

φ2 = login 7→ {option screen, error user}

/[10, 40] B [10, 40]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 29

Page 39: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ2

After observing any occurrence of login in the log,

then we have to observe option screen or error user.

The amount of time associated with this input/output mustbelong to the time interval [10, 40].

Invariant

φ2 = login 7→ {option screen, error user}/[10, 40] B [10, 40]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 29

Page 40: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ3

After inserting the last change into the profile screen,

and after saving the current state of the system, theoption screen

and the time associated with these operations are included inthe intervals [10, 20] and [20, 40] respectively.

In addition, the total amount of time to perform this activitymust belong to the interval [35, 50].

Invariant

φ3 = data/profile screen

/[10, 20], save 7→ {option screen}/[20, 40] B [35, 50]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 30

Page 41: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ3

After inserting the last change into the profile screen,

and after saving the current state of the system, theoption screen

and the time associated with these operations are included inthe intervals [10, 20] and [20, 40] respectively.

In addition, the total amount of time to perform this activitymust belong to the interval [35, 50].

Invariant

φ3 = data/profile screen

/[10, 20]

, save 7→ {option screen}

/[20, 40] B [35, 50]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 30

Page 42: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ3

After inserting the last change into the profile screen,

and after saving the current state of the system, theoption screen

and the time associated with these operations are included inthe intervals [10, 20] and [20, 40] respectively.

In addition, the total amount of time to perform this activitymust belong to the interval [35, 50].

Invariant

φ3 = data/profile screen/[10, 20], save 7→ {option screen}/[20, 40]

B [35, 50]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 30

Page 43: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ3

After inserting the last change into the profile screen,

and after saving the current state of the system, theoption screen

and the time associated with these operations are included inthe intervals [10, 20] and [20, 40] respectively.

In addition, the total amount of time to perform this activitymust belong to the interval [35, 50].

Invariant

φ3 = data/profile screen/[10, 20], save 7→ {option screen}/[20, 40] B [35, 50]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 30

Page 44: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ4

When the student is in the option screen (after any input),

if she inserts the input interaction marks, then themarks screen will appear within 20 and 40 time units.

All the interaction must take between 30 and 70 time units.

Invariant

φ4 = ?/option screen/[5, 35],

marks 7→ {marks screen}/[20, 40] B [30, 70]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 31

Page 45: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ4

When the student is in the option screen (after any input),

if she inserts the input interaction marks, then themarks screen will appear within 20 and 40 time units.

All the interaction must take between 30 and 70 time units.

Invariant

φ4 = ?/option screen/[5, 35],marks 7→ {marks screen}/[20, 40]

B [30, 70]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 31

Page 46: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Invariant φ4

When the student is in the option screen (after any input),

if she inserts the input interaction marks, then themarks screen will appear within 20 and 40 time units.

All the interaction must take between 30 and 70 time units.

Invariant

φ4 = ?/option screen/[5, 35],marks 7→ {marks screen}/[20, 40] B [30, 70]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 31

Page 47: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Algorithm: Correctness of invariants

Input: M = (S, I,O,Tr, sin), φ = {a1, . . . , an−1, in 7→ M} // either ak = ik/ok/Fk or ak = ?Output: Bool.I ′ ← I ; S′ ← S ; j ← 1; S′′ ← ∅; error ← false;

While (j < n)If (Head(I ′) = ?)

While (S′ 6= ∅)Choose s ∈ S′ S′ ← S′ \ {s}S′′ ← S′′ ∪ afterInp{s, ij+1}

elseWhile (S′ 6= ∅)

Choose sa ∈ S′ S′ ← S′ \ {sa}Tr′ ← afterCond{sa, ij , oj}

While(Tr′ 6= ∅)Choose (sa, sb, ij , oj , F ′) ∈ Tr′

Tr′ ← Tr′ \ {(sa, sb, ij , oj , F ′)}If(F ′ = Fj ) then S′′ ← S′′ ∪ {sb}

I ′ = Tail(I ′) j ← j + 1; S′ ← S′′; S′′ ← ∅;

If (S′ = ∅) error ← true

While (S′ 6= ∅)Choose sa ∈ S′ S′ ← S′ \ {sa} Tr′ ← afterCond{sa, in, ?}While(Tr′ 6= ∅)

Choose (sa, sb, in, o, F ′) ∈ Tr′ Tr′ ← Tr′ \ {(sa, sb, in, o, F ′)}If (〈o, F ′〉 ∈ M)

S′′ ← S′′ ∪ {sb}else error ← true

If (S′′ = ∅) then error ← trueReturn (¬error)

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 32

Page 48: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Global view, specification, invariant,implementation, and logs

Specifications0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

Invariantsφ1 = login/option screen/[20, 40], ?/[0,∞],

disconnection 7→ {welcome screen}/[10, 20] B [35,∞]

φ2 = login 7→ {option screen, error user}/[10, 40] B [10, 40]

φ3 = data/profile screen/[10, 20], save 7→ {option screen}/[20, 40] B [35, 50]

φ4 = ?/option screen/[5, 35], marks 7→ {marks screen}/[20, 40] B [30, 70]

m

Implementation

Traces

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 33

Page 49: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Global view, specification, invariant,implementation, and logs

Specifications0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

Invariantsφ1 = login/option screen/[20, 40], ?/[0,∞],

disconnection 7→ {welcome screen}/[10, 20] B [35,∞]

φ2 = login 7→ {option screen, error user}/[10, 40] B [10, 40]

φ3 = data/profile screen/[10, 20], save 7→ {option screen}/[20, 40] B [35, 50]

φ4 = ?/option screen/[5, 35], marks 7→ {marks screen}/[20, 40] B [30, 70]

m

Implementation

Traces

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 33

Page 50: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Global view, specification, invariant,implementation, and logs

Do not have a fullspecification

Partialspecification.

Only interested insome properties.

Invariantsφ1 = login/option screen/[20, 40], ?/[0,∞],

disconnection 7→ {welcome screen}/[10, 20] B [35,∞]

φ2 = login 7→ {option screen, error user}/[10, 40] B [10, 40]

φ3 = data/profile screen/[10, 20], save 7→ {option screen}/[20, 40] B [35, 50]

φ4 = ?/option screen/[5, 35], marks 7→ {marks screen}/[20, 40] B [30, 70]

m

Implementation

Traces

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 33

Page 51: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Trace files

Since we assume that we have TFSM, logs will follow this pattern:

Input1/ Output1/ Time1,Input2/ Output2/ Time2,. . .

Inputn/ Outputn/ Timen

Trace

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 34

Page 52: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Algorithm: Correctness of traces with res-pect to invariants

Theorem

Let S (a specification) and I (animplementation) be two Timed FiniteState Machines,

Let φ be a correct timed invariant withrespect to S .

Let e be a log recorded from I .

If the invariant φ does not match e then I doesnot conform to S .

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 35

Page 53: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Core of PasTe

Invariants

Correctspec?

Spec

File

Correcttrace?

Traces

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 36

Page 54: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Core of PasTe

InvariantsCorrectspec?

Spec

File

Correcttrace?

Traces

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 36

Page 55: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Classification of different Invariant Suites

Specifications0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

Invariant Suite 2φ2 = login 7→ {option screen, error user}/[10, 40] B [10, 40]

φ3 = data/profile screen/[10, 20], save 7→ {option screen}/[20, 40] B [35, 50]

Invariant Suite 1φ3 = data/profile screen/[10, 20], save 7→ {option screen}

/[20, 40] B [35, 50]

φ4 = ?/option screen/[5, 35], marks 7→ {marks screen}/[20, 40] B [30, 70]

Question

Which one do we prefer? Invariant Suite 1 or Invariant Suite 2?

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 37

Page 56: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Classification of different Invariant Suites

Specifications0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

Invariant Suite 2φ2 = login 7→ {option screen, error user}/[10, 40] B [10, 40]

φ3 = data/profile screen/[10, 20], save 7→ {option screen}/[20, 40] B [35, 50]

Invariant Suite 1φ3 = data/profile screen/[10, 20], save 7→ {option screen}

/[20, 40] B [35, 50]

φ4 = ?/option screen/[5, 35], marks 7→ {marks screen}/[20, 40] B [30, 70]

Question

Which one do we prefer? Invariant Suite 1 or Invariant Suite 2?

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 37

Page 57: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Mutants: Evaluating invariants I

Mutants are produced from the specification.

We generate traces from mutants.

An invariant suite is better if it kills more mutants.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 38

Page 58: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Mutants: Evaluating invariants II

Specifications0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

ApplyingMutation ⇒Operators

Mutant 1s0s1

s2s3

s4

s5

s6s7

i1, o3, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

...

Mutant ns0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 25

i10, o6, 15

i11, o7, 80i11, o8, 80

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 39

Page 59: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Mutants: Evaluating invariants III

Changing an output

qa

qb

qc

a,x,1b,y,2

a,x,3

b,x,2

b,z,1a,y,3

qa

qb

qc

a,x,1b,y,2

a,y,3

b,x,2

b,z,1a,y,3

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 40

Page 60: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Mutants: Evaluating invariants IV

Changing the target state in a transition

qa

qb

qc

a,x,1b,y,2

a,x,3

b,x,2

b,z,1a,y,3

qa

qb

qc

a,x,1

b,y,2

a,y,3

b,x,2

b,z,1a,y,3

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 41

Page 61: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Mutants: Evaluating invariants V

Altering the time value

qa

qb

qc

a,x,1b,y,2

a,x,3

b,x,2

b,z,1a,y,3

qa

qb

qc

a,x,1b,y,2

a,x,3

b,x,17

b,z,1a,y,3

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 42

Page 62: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Core of PasTe

InvariantsCorrectspec?

Spec

Mutants

Traces

FileCorrecttrace?

generate

extract

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 43

Page 63: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Core of PasTe

InvariantsCorrectspec?

Spec

Mutants

Traces

FileCorrecttrace?

generate

extract

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 43

Page 64: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Integration into Osmius

Masteragent

OsmiusLog Agent

PasTedaemon

Log fileImpl. traces

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 44

Page 65: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Running Example I

Mutant of SSadmin specification executing in real time.

Paste daemon detecting periodically errors in the trace of themutant.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 45

Page 66: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Running Example II

Specification

TFSM

q0

q0 CONNECT WELCOMESCREEN 10 q1

q1 LOGIN ERRORUSER 30 q1

q1 LOGIN OPTIONSCREEN 30 q2

q2 DISCONNECTION WELCOMESCREEN 10 q1

q2 PROFILE PROFILESCREEN 20 q3

q2 MARKS MARKSCREEN 30 q4

q2 REGISTER REGISTERSCREEN 200 q5

q3 DATA PROFILESCREEN 10 q3

q3 SAVE OPTIONSCREEN 30 q2

q3 RETURNOPTION OPTIONSCREEN 10 q2

q3 RETURNOPTION NOTSPECTED 10 q2

q4 RETURNOPTION OPTIONSCREEN 10 q2

q5 DATASUBJECT REGISTERSCREEN 10 q6

q5 RETURNOPTION OPTIONSCREEN 10 q2

q6 DATASUBJECT REGISTERSCREEN 10 q6

q6 SAVEREGISTRATION CONFIRMSCREEN 80 q7

q6 SAVEREGISTRATION NOCONFIRMSCREEN 80 q7

q6 RETURNOPTION OPTIONSCREEN 10 q2

q7 RETURNOPTION OPTIONSCREEN 10 q2

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 46

Page 67: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Running Example III

Invariants

INVARIANTSI

LOGIN OPTIONSCREEN [ 20 40 ] , * [ 0 -1 ] ,

DISCONNECTION - < WELCOMESCREEN > [ 10 20 ] - [ 35 -1 ]

LOGIN - < OPTIONSCREEN , ERRORUSER > [ 10 40 ] - [ 10 40 ]

DATA PROFILESCREEN [ 10 20 ] , SAVE - < OPTIONSCREEN > [ 20 40 ] - [ 35 50 ]

? OPTIONSCREEN [ 05 35 ] , MARKS - < MARKSCREEN > [ 20 40 ] - [ 30 70 ]

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 47

Page 68: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Running Example IV

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 48

Page 69: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Current work

Planned applications

Analyze implementations of the Optimized Link State RoutingProtocol (OLSR). No formal specification(http://www.ietf.org/rfc/rfc3626.txt) currentresearch with Professor Ana Cavalli research group(Department of Network Software, TELECOM &Management SudParis).

Intrusion detection system (IDS). Invariants to expressbehavioral patterns of crackers, malware and/or disgruntledemployees.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 49

Page 70: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Current work I

Planned applications

Analyze implementations of the Optimized Link State RoutingProtocol (OLSR). No formal specification(http://www.ietf.org/rfc/rfc3626.txt) currentresearch with Professor Ana Cavalli research group(Department of Network Software, TELECOM &Management SudParis).

Intrusion detection system (IDS). Invariants to expressbehavioral patterns of crackers, malware and/or disgruntledemployees.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 49

Page 71: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Current Work II

Methods to extract a set of invariants

Extract⇒

Invariant Suite

φ1

. . .

φn

If we have a complete formal specification we have to checkthe correctness of the proposed invariant suite.

If we do not have a complete formal specification we assumethat the invariant suite is correct.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 50

Page 72: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Current Work III

Specifications0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

Extract⇒

Invariant Suite

φ1

. . .

φn

We automatically derive invariants from the specification...but there are a lot!!

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 51

Page 73: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Current Work IV

Specifications0s1

s2s3

s4

s5

s6s7

i1, o1, 15

i2, o2, 30

i2, o3, 30

i3, o1, 15

i4, o4, 25i8, o5, 30

i9, o6, 200

i7, o2, 30

i5, o4, 15

i10, o6, 15

i10, o6, 15

i11, o7, 80i11, o8, 80

+

User Model

InvariantSuite

φ1

. . .

φn

We automatically derive invariants from the specification.

We combine this technique with other formal approaches likeusers models in order to obtain a relevant set of behaviours.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 52

Page 74: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Current work V

Develop a native Osmius agent

Masteragent

OsmiusLog Agent

PasTedaemon

Log fileImpl. traces

OsmiusPasteAgent

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 53

Page 75: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Current work VI

Develop a native Osmius agent

Masteragent

OsmiusLog Agent

PasTedaemon

Log file

Impl. traces

OsmiusPasteAgent

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 53

Page 76: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

References

Osmius http://www.osmius.net

Paste http://kimba.mat.ucm.es/paste/

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 54

Page 77: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Publications

amn08b C. Andres, M.G. Merayo, and M. Nunez. Passive testing of timed systems. In 6th Int.Symposium on Automated Technology for Verification and Analysis, ATVA’08, LNCS 5311,pages 418–427. Springer, 2008.

amn09b C. Andres, M.G. Merayo, and M. Nunez. Formal correctness of a passive testing approachfor timed systems. In 5th Workshop on Advances in Model Based Testing, A-MOST’09 .IEEE Computer Society Press, 2009.

amn09c C. Andres, M.G. Merayo, and M. Nunez. Using a mining frequency patterns model toautomate passive testing of real-time systems. In 21st Int. Conf. on Software Engineeringand Knowledge Engineering, SEKE’09. Knowledge Systems Institute, 2009.

amn09d C. Andres, M.G. Merayo, and M. Nunez. Supporting the extraction of timed properties forpassive testing by using probabilistic user models. In Submitted to QSIC’09, 2009.

alr09 C. Andres, L. Llana, and I. Rodrıguez. Formally transforming user-model testing problemsinto implementer-model testing problems and viceversa. Journal of Logic and AlgebraicProgramming, pages 425-453, 2009.

amm09 C. Andres, M.G. Merayo, and C. Molinero. Advantages of mutation in passive testing: Anempirical study. In 4th Workshop on Mutation Analysis, Mutation’09. IEEE ComputerSociety Press, 2009.

amn09 C. Andres, M.G. Merayo, and M. Nunez. Passive testing of stochastic timed systems. In 2ndInt. Conf. on Software Testing, Verification, and Validation, ICST’09, pages 71–80. IEEEComputer Society Press, 2009.

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 55

Page 78: Osmius 9 - UCMantares.sip.ucm.es/tarot09/index_files/osmius_TAROT09.pdf · Integrated Data Warehouse Specific Osmius processes to aggregate daily info (seconds in each state, etc)

Thank you for your attention!

The PASsive TEsting tool and Osmius Las Navas del Marques, July 2009 56