![Page 1: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/1.jpg)
Automated Testing of Network Service Interactions
Contributions to Automated Testing of Network Service
Interactions
Rudy Deca1 ,Omar Mahrez1, Omar Cherkaoui1, Yvon Savaria2 and
Doug Slone3
Notere 2005, Ottawa/Gatineau, Canada
1University of Québec at Montréal, 2Ecole Polytechnique de Montréal, 3Cisco Systems, Inc.
![Page 2: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/2.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
2
Contents
The problem context The Meta-CLI Model The ScriptMaker tool Conclusion
![Page 3: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/3.jpg)
3
The Problem context
Connectionless statelessnetworks
Connection-oriented
circuit-switching stateful network services
Hybrid, contradictory
network service characteristics
Network convergence
NewSolutionsrequired
More services more features
more parameters
More combinationsmore interactions
more configurationsmore testsrequired
More environment
states
Increase in the number of states
Causes: Effects:
![Page 4: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/4.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
4
Contents
The problem context The Meta-CLI Model
General presentation Structure• Operations
Abstracting the environment The ScriptMaker tool Conclusion
![Page 5: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/5.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
5
The Meta-CLI Model solution
Abstracts: the network service and device configuration
states the environment states the dynamic changes of the connectionless,
stateless IP protocols Benefits:
Automated adaptation to environment changes Efficient generation and modification of
configurations, test scripts and tests for service interactions.
![Page 6: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/6.jpg)
6
Modelling a service configuration as a conceptual tree structure
Arcs: Hierarchical
relationships Inclusion relationships
Nodes: Configuration command modes Configuration commands Configuration parameters
Meta-CLI modelling
Created
Modified
device_1
service_1
service_1
Command mode
M
Command P
Command Q
Parameter R
Parameter S
Command mode
NConfiguration
commands and parameters
![Page 7: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/7.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
7
Contents
The problem context The Meta-CLI Model
General presentation • Structure Operations
Configuration Validation
Abstracting the environment The ScriptMaker tool Conclusion
![Page 8: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/8.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
8
Meta-CLI Model Operations Operation types:
Configuration Validation
Distribution: Single device Multiple devices of a network
Operation attributes: Atomicity Order (causal, sequential) Granularity (varying from parameters to
services)
![Page 9: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/9.jpg)
Automated Testing of Network Service Interactions
9
Configuring services using the Meta-CLI Model The main idea of the Meta-CLI configuration: compose thedevice and network service configuration models.
Deviceconfiguration
record
Networkservice
Conversion
Co
nve
rsio
n
Meta-CLI Model
Capsule
Construct
Co
mp
ositio
n
Configuration Construct + RetroversionDevice
configurationrecord +
![Page 10: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/10.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
10
Generic service configuration operations
Instantiate service configurations on devices. Instantiate service configuration parameter values. Instantiate service configuration constraints. Adapt service configuration to device and interface
environment. Adapt service configuration to device and interface
role. Determine environment support for various feature
combinations Determine compatibility of services and features. Determine environment support for service and
feature combinations.
![Page 11: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/11.jpg)
11
Examples of validation dependencies
Validation dependencies between: configuration commands: T depends on Q configuration parameters: S and U depend on R
Dep.U-R
device_1
Command P
Value1
Parameter R
Value3
Parameter U
Value4
Command Q
Value2
device_2
Command P
Value5
Parameter S
Value7
Parameter V
Value8
Command T
Value6
Dependency S-R
Dependency T-Q
![Page 12: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/12.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
12
Network service and feature interactions
Mutual exclusion, due to: Competition for resources Accessing the same parameters/variables Incompatible goals
Influence, due to : Scale factor Performance factor
Service combination constraints: Common supporting environment Existence of mandatory parameters
![Page 13: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/13.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
13
Contents
The problem context The Meta-CLI Model
• General presentation Abstracting the environment
The ScriptMaker tool Conclusion
![Page 14: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/14.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
14
Environment heterogeneity solution
The configuration environment: network-level component
network topology, technology, Protocol, Role.
equipment-level component. hardware components
device type, interface type, medium type, etc.
software components operating system, software version, etc.
![Page 15: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/15.jpg)
15
Adapt CLI commands to different environments
Command P is instantiated on device_1with one parameter (R) and on device_2 with two parameters (R,S).
Command P
Parameter R
Parameter S
service_1
env1, env2
Command P
env1, env2
Paramet. R
env1, env2
Paramet. S
env2
device_1 + service_1device_2 + service_1
device_1
env1
Command P
Parameter R
device_2
env2
![Page 16: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/16.jpg)
16
Adapt CLI commands to different device and interface roles
Command P is instantiated: on device_1 with parameter R and on device_2 with parameter S, based on the different roles assumed
by the two devices.
Command P
Parameter S
service_1
role1, role2
Command P
role1, role2
Paramet. R
role1
Paramet. S
role2
device_1 + service_1
device_1
role1
Command P
Parameter R
device_2
role2
device_2 + service_1
![Page 17: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/17.jpg)
17
Addition of a network service to a device configuration
The generic network service model is accessed from the repository.
A service instance model is generated and adapted to the given environment
and then composed with the device configuration model.
![Page 18: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/18.jpg)
Modification of a network service in a device configuration
The network service is already configured on an equipment when an environment change occurs.
The network service model instance in the old environment is extracted from the device configuration model.
The generic network service model is accessed from the repository.
A new instance model is generated for the new environment and composed with the device configuration model.
![Page 19: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/19.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
19
Determining environment support for various feature combinations
The information is obtained by querying the tree model.
Service features
service_1 P P P P
Feature R NP P NP P
Feature S NP NP P P
Environment instances
env1 S S NS NS
env2 S S S S
service_1
env1, env2
Command P
env1, env2
Feature R
env1, env2
Feature S
env2 P = is present
NP = is not present
S = supports
NS = does not support
![Page 20: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/20.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
20
Contents
The problem context The Meta-CLI Model The ScriptMaker tool
Functionality• Architecture• Generation scenario• Conversion scenario
Conclusion
![Page 21: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/21.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
21
Functionality of the ScriptMaker tool
Aimed to regression testing. Provides an effective, automated and easy way to
generate/modify testing scripts with test cases featuring various combinations of network services and features.
Solves the problem of the existence of a large number of : combinations among existing network services; features and parameters of the services; environment states in which the service parameters are
configured differently. Uses a modular approach for components:
Service configurations Traffic Routing, topology and connection among router interfaces Configurations validation
![Page 22: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/22.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
22
Contents
The problem context The Meta-CLI Model The ScriptMaker tool
• Functionality Architecture• Generation scenario• Conversion scenario
Conclusion
![Page 23: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/23.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
23
Architecture of the ScriptMaker tool
![Page 24: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/24.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
24
Contents
The problem context The Meta-CLI Model The ScriptMaker tool
• Functionality• Architecture Generation scenario• Conversion scenario
Conclusion
![Page 25: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/25.jpg)
Automated Testing of Network Service Interactions
25
The script generation scenario
Specific CLI servicecode
Genericservice
capsule stubs
Envi-Ron-ment
Selected Specific TC, TS,
Templates
Selected Generic TC, TS,
Templates
Signaling
OK
Environment (descriptor)
Feature name(s)
Script
Environment (descr.)
Feature name(s)
OK
Script
Script Manager Block
Test SelectionBlock
Test Filling Block Test Binding Block
Meta-CLI Model Block
![Page 26: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/26.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
26
The test selection block
Selected TC, TS,
Template names
Featurename(s)
Env.desc.
Envi-ron-
ment
Feature name(s) Constr
Constraints
Selected TC, TS, Templates
TC, TS, Template
names
Feature name(s)
Test Selector
Test Organizer
Spec Archive
Test Bench
Environment Constraint Database
![Page 27: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/27.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
27
The test filling block
Selected Spec CLI auxiliary, validation commands and TCL Traffic, Topology
commands
Test Filler
Converter
InstantiatorCode Generator
Selected Gen CLI auxiliary, validation commands and TCL
Traffic, Topology commands
Selected generic stubs Kx, Kv, Trf, Top
Selected Gen stubs Kx, Kv, Trf, Top
Selected Gen CLI auxiliary, validation commands and TCL
Traffic, Topology commands
Environment
Selected Spec CLI auxiliary, validation commands and TCL Traffic, Topology commands
Ks = Service capsuleKv = Validation capsuleKx = Auxiliary capsule
![Page 28: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/28.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
28
The Meta-CLI Model block
Meta-CLI Model manager
Service Instantiator
Service Retriever
Selected genericservice capsule
Selected generic service stubs Ks
Selected generic service capsule Environment descriptor
Selected specific
CLI service
Capsules (models)
Selected generic service capsule
Selected generic
service stubs
![Page 29: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/29.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
29
Contents
The problem context The Meta-CLI Model The ScriptMaker tool
• Functionality• Architecture• Generation scenario Conversion scenario
Conclusion
![Page 30: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/30.jpg)
Automated Testing of Network Service Interactions
30
Envi-Ron-ment
The script conversion scenario
selected servicesauxiliariesvalidationstopologies
traffics
Generic TC, TS, Template
servicesauxiliariesvalidationstopologies
traffics
service
Filled Selected Specific TC, TS,
Template
Selected Generic TC, TS,
Template
Signaling
Old script
Environment
Feature name(s)
Script
Environment
Feature name(s)
Script
New script
Script Manager Block
Test Selection
Block
Test Filling Block
Test Binding Block
Meta-CLI Model Block
ParsingBlock
Specific CLI servicecode
![Page 31: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/31.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
31
Conclusion Problem:
New Internet services, hybrid, with numerous: parameters features interactions heterogeneous environments.
FSMs not always useful. Solutions:
The Meta-CLI Model: Generic service configuration Environment-aware
The ScriptMaker tool: Automated generation/modification of testing scripts; Modular design; Uses the Meta-CLI for abstracting and reusing service
configurations.
![Page 32: Contributions to Automated Testing of Network Service Interactions](https://reader030.vdocuments.site/reader030/viewer/2022032805/568131a8550346895d9817dd/html5/thumbnails/32.jpg)
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
32
Questions