Certification Test Tool Update

Toby Nixon

Program Manager

Microsoft Corporation

Assumptions

Familiar with UPnP™ v1 architecture and terminology

Interested in evaluating a device’s compliance against the UPnP™ standard

Pursuing the UPnP™ Logo

Overview Certification Test Tool purpose

What is tested

Test capabilities

Using the tool and interpreting results

Update on the next release

Purpose of Cert Tool

Drive the certification of UPnP™ devices

Execute device independent protocol tests against the UPnP™ v1 standard

Support device dependent syntax and semantic testing

What is Tested

Device-independent protocol tests

Device-specific syntax tests

Security Device tests

Device-specific semantic tests

Protocol Tests

Validate SSDP and HTTP packets and their sequence

Test the following stages: Addressing, Description, Discovery, Security, Control, Eventing

Execute regardless of device type

Syntax Tests Actions to be tested for each device type are

specified in an XML configuration file

Syntax tests are created by editing the <ServiceId>.SyntaxTests.XML file Working groups provide the XML syntax test

configuration files

Tests are run during the Control and Eventing test stages

Actions are invoked in the order in which they appear in the XML file

Device Security Tests

Available in test tool versions 1.4 and above

Device Security tests as part of the service syntax tests (3.*)

The test configuration file is ‘DeviceSecurity1.SyntaxTests.xml’

Semantic Tests Verify specified output parameters, error

conditions and events Create semantic tests by editing the

<ServiceId>.SemanticTests.XML file Working groups provide the XML syntax test

configuration files

Actions must be within the same service Results of one action cannot be used as

input parameters in another action

What is Not Tested

Does not test performance specifications

Does not certify device extensions external to the UPnP™ standard

Does not test incorrect inputs, boundary conditions

Does not test product capabilities

Tests on-the-wire interoperability

Test Environment Test tool requires Windows XP

Recommend XP SP2 (for IPv6 and HTTP/1.1 testing)

Run only one UPnP™ device and a single instance of the test tool on an isolated network

Turn off LAN connection settings (i.e. proxy settings) in Internet Options control panel on Windows XP

Test Environment Diagram

Isolated Isolated Network HubNetwork Hub

Cert Tool running on WinXPCert Tool running on WinXPDevice Under TestDevice Under Test

DHCP Server for DHCP Server for Addressing testsAddressing tests

Network Sniffer Network Sniffer (optional)(optional)

XML Configuration Files

<DeviceId>.xml Lists the implemented services and embedded devices

<ServiceId>.xml Lists the implemented Actions and State Variables

<ServiceId>.SyntaxTests.xml Lists the Action names and the ‘In’ arguments for each Action

<ServiceId>.SemanticTests.xml Lists the Action names, ‘In’ arguments and the expected ‘out’

arguments or error code.

Using XML Config Files Working Committees

Write the XML config files for standardized DCPs Distribute XML config files on the UIC web site

The test tool uses the XML files to verify Devices have all required services Services implement required Actions and State

Variables Devices respond correctly for each Action

invocation

Proprietary devices can be tested but require the respective XML config files

Device Description<device>

<deviceType>urn:schemas-upnp-org:device:WANDevice:1</deviceType>

</device>

<device>

<Optional/>

<deviceType>urn:schemas-upnp-org:device:LANDevice:1</deviceType>

</device>

Service Description<serviceStateTable>

<stateVariable>

<name>EthernetLinkStatus</name>

<dataType>string</dataType>

<allowedValueList>

<allowedValue>Up</allowedValue>

<allowedValue>Down</allowedValue>

</allowedValueList>

</stateVariable>

</serviceStateTable>

Syntax Tests XML File<TestCase>

<Id>1</Id>

<Category>Valid Action And Valid InArgs</Category>

<ActionName>GetEthernetLinkStatus</ActionName>

<ExpectedReturnCode>ACTION_AND_INARGS_ARE_VALID</ExpectedReturnCode>

</TestCase>

Semantic Tests XML File

<ManualAction>

<ActionInstruction> Remove cable. </ActionInstruction>

<VerificationInstruction> Verify link light is off.

</VerificationInstruction>

<ExpectedSuccessResponse/>

</ManualAction>

Test Tool UI

Select Test Options

Step Mode / Run Until Failure

Step Mode Pauses between each test Click the “Next” button to continue Useful when debugging a failure

Run Until Failure Pauses the test execution at the first failure

Interpreting the Test Log

Test Case 2.0.0.1 Passed

Test Case 2.1.0 Failed to RunFailure Reason :Precondition Failure - Unable to

Construct Device Discovery Message List.

Reason : Unable to Acquire Root Device Description Url.Badly formed Notify Alive Packets Header Value.

Reason : NT Header Value = TestDevice is Not Compliant to UDA V1.0

Helpful Hints

Review the test log for failure details

Use a network sniffer UPnP™ parser available for Microsoft Network

Monitor Useful for reporting problems

Use Step Mode to help isolate an issue

Refer to the Test Tool User documentation for descriptions of each test case

Helpful Hints Continued

A full certification pass takes about 25 minutes to complete

For testing purposes, set the CACHE-CONTROL in the device to be less than 60 for SSDP alive notifications

In your final device implementation, set the CACHE-CONTROL according to UDA recommendations

Update on next release v1.5

Planned for 2nd Quarter 2005 New features

Using MSEARCH to initiate tests instead of NOTIFY for faster test runs

Improved logging messages for better diagnostics Testing on non isolated networks and presence of

multiple root devices Embedded devices testing Comparing arguments and passing parameters

across different actions Presence of extra header information in Discovery,

Description, Control and Eventing tests

Q&A Certification Test Tool

Is made available by UIC Tool is posted on www.upnp-ic.org Documentation is included with the tool For debugging, Cert Tool in UPnP™ Forum website has

sources

Support UIC members receive one free support call Newsgroup: microsoft.public.upnp

Questions Email: upnpcert@microsoft.com

Foundation of the Connected Home