test strategy utilising mc useful tools

High Level Test Strategy overview

Test Strategy

• “Layer” the project

• Identify the components in each “layer”

• Identify the test basis

• Draw up a Dependency matrix

• Formulate a testing “Big Picture”

• Test Packs

• Summarising and going forward ….


“Layer” the project

• Integration services runs on (currently) 3 Biztalk platforms: IS2000 (PoS), IS2002 (ISGM) and IS2004

• Each platform contains various abstract layers:– Service layer– Application layer– Database layer– Biztalk layer

Queue

ReceiveFunctions

Adapters

Orchestrations

MessageBoxes

ServerHosts– Test Environment


Components in each layer

• Service layer– IS01, IS02, IS03, IS04, BP01, BP02 etc.

• Application layer– RuleExec, Janus Web, HOPPA etc.

• Database layer– NDTRefData, Alerter, HAT etc.

• Biztalk layer– Queue - Workqueue, Retry, Scheduled etc.– ReceiveFunctions - Failed Messages, Receiver– Adapters - FTP, HTTP, SOAP etc.– Orchestrations – BP01, BP02 etc.– MessageBoxes – MsgBox DB, BAM archive etc.– ServerHosts – Orchestration host, Tracking host, SendHost adapters etc.

• Test Environment– DEV, Systest 1, Systest 2, CATE 1, CATE 2, little CATE etc.


Test basis N

• This is the documentation that supplies the Expectations, Requirements and Expected Results for ANY given piece of testing effort

• Prime source of these here is the SUMMIT methodology deliverables – i.e. Project Charter, System Prospectus, System Delivery Spec (SDS), Tech System Design (TSD), Tech Procedure Development (TPD) etc.

• Test basis should (ideally) consist of SIGNED-OFF DOCUMENTATION ONLY. However – real-world here and it may include draft docs but increased risk of inaccurate Requirements and/or Expected Results

• All pertinent test basis documents collated as “KeyDocs” and are secured into Quality Center


Dependency matrix N

• This matrix will highlight the implicit and explicit relationship between IS components on each abstract layer

• It is used for impact analysis and, hence, to identify likely candidates for regression testing here in IS

• Where change is known to have occurred to one component in the abstract layer model, the reader can quickly follow the implicit and explicit relationships to other components that are identified in this matrix. As the reader navigates through the matrix, further likely regression test candidates (and hence Test Packs) are identified

• Change to one particular component may involve regression testing “most” of IS


Testing Big Picture N

• This is, in essence, a amalgamation of all of the useful diagrams identified as part of the test basis

• This model offers a tabular view of the testing and will involve the decomposition of those diagrams in the KeyDocs repository

• Data from the diagrams is dropped into a two-dimensional table and the information from the diagram (Names in boxes, names against lines, where the lines come from, where the lines go to etc) is then accessible AND, more importantly, reusable from a testing perspective

• Impact analysis is greatly facilitated by using this Big Picture…You can not only prove the Dependency matrix – which means you can accurately describe what related, regression tests need to be run but you also have a pictorial view of everything testing. INVALUABLE


Test Packs N

• The management will usually have a view at what level of granularity a Test Packs should be designed. Integration Services (IS) have chosen “layers” within platform. i.e ISGM Biztalk, or IS2004 Database, or IS01 service etc.

• The FEWER Test Packs you choose to have, the MORE has to go into each Test Pack. This increases complexity, which increases the chance of failure in a Test Pack, BUT the MORE Test Packs you have, the more resources required to create, modify, prepare and execute them!

• If you “spread the load” by having more Test Packs (higher level of granularity), then you have a more agile way of testing change BECAUSE you wont need to regression test ALL Test Packs.


Summarising N

• Test Packs are used to regression test• KeyDocs repository contains the (signed-off and master) test basis documentation which

will facilitate proof of test coverage (i.e. HAVE we covered everything in the documentation?)• Dependency matrix helps you to decide (accurately when used in conjunction with the

Big Picture) exactly what else is candidate for test• Big Picture gives pictorial representation of ALL things testing on a project/programme.

The data/information contained therein is ultimately reusable because it also exists in two-dimensional form!

• If all this seems VERY COMPLEX, then wait till something CHANGES. i.e. documentation or process. THEN it gets complex because you don’t know what impact that change will have!!

• The beauty of the above is that I have tools that will automatically decompose documents/diagrams so “change” merely means re-extraction of the data from the changed documents or diagrams (text from documents or text/context information from diagrams) into KeyDocs or Big Picture……. Next slide elaborates more

• ….. But this is all talking the talk, now lets walk the walk


Walk the walk N

• DocIndex provides the engine that drives the creation of the KeyDocs (test coverage and Expected Results).

• InternetMiner provides the engine that assists the creation of KeyDocs and operates on Intranet/internet pages and hyperlinks to extract similar information to DocIndex. This is more an ancillary tool that complements DocIndex and is useful for gaining insight into specific areas of expertise.

• VisioDecompositer provides the engine that drives diagram information (process flow and data flow) into the Big Picture.

• Here’s some screen shots to give you a better flavour….


DocIndex 1 of 10N

• DocIndex (viewer) main menu


DocIndex 2 of 10

• KeyDocs main menu


DocIndex 3 of 10

• Display a selectable list of KeyDocs


DocIndex 4 of 10

• Now display a summarised list of the sections in the selected KeyDoc


DocIndex 5 of 10

• Select one of the sections in the document and the text is displayed


DocIndex 6 of 10

• The power of this tool lies in the fact that ALL relevant project/programme-level documentation is captured into the tool


DocIndex 7 of 10

• The contents of all of the KeyDocs are stored in two tables: tblSectionHeaders – which is a summarised “table of contents” for a KeyDoc


DocIndex 8 of 10

• and File_Decomposition – which is textual body for all KeyDocs


DocIndex 9 of 10

• There is a keyword/key phrase Search facility for the group of KeyDocs, which allows the user to quickly become acquainted with “project specific” terminology


DocIndex 10 of 10

• and the reader can, very quickly, become acquainted with the context in which the specific search terms are used.


InternetMiner 1 of 10 N

• InternetMiner main menu


InternetMiner 2 of 10

• Enter a suitably relevant URL - in this case, found by doing a “biztalk performance test” search string in Google UK



• And the page(s) are decomposed into InternetMiner, in much the same way as DocIndex



• The list of decomposed URLs can be viewed (and refreshed either manually or automatically) so that the information contained therein is always up to date



• And the content from ALL of the decomposed URLs can be viewed at a glance



• And relevant search keywords or key phrases can be entered



• And the result be displayed (in various forms) - so that the user can see, contextually, where the search term is used



• Alternative views of the results (and search strings) can be made



• And you can quickly build up a picture of the various (in this case technical Biztalk) phrases



• And how they relate to the overall topic (Biztalk Performance testing)


VisioDecompositer 1 of 2N

• VisioDecompositer takes a straightforward Visio diagram…..


VisioDecompositer 2 of 2N

• And converts it into two-dimensional information that can be “mapped” into test cases or test scripts quite easily

test strategy utilising mc useful tools

Technology