Certification Test Tool Update

Toby NixonProgram Manager

Microsoft Corporation

Assumptions

Familiar with UPnP™ v1 architecture and terminology

Interested in evaluating a device’s compliance against the UPnP™ standard

Pursuing the UPnP™ Logo

Overview Certification Test Tool purpose What is tested Test capabilities Using the tool and interpreting results Update on the next release

Purpose of Cert Tool

Drive the certification of UPnP™ devices Execute device independent protocol

tests against the UPnP™ v1 standard Support device dependent syntax and

semantic testing

What is Tested

Device-independent protocol tests Device-specific syntax tests Security Device tests Device-specific semantic tests

Protocol Tests

Validate SSDP and HTTP packets and their sequence

Test the following stages: Addressing, Description, Discovery, Security, Control, Eventing

Execute regardless of device type

Syntax Tests Actions to be tested for each device type are

specified in an XML configuration file Syntax tests are created by editing the

<ServiceId>.SyntaxTests.XML file Working groups provide the XML syntax test

configuration files Tests are run during the Control and

Eventing test stages Actions are invoked in the order in which

they appear in the XML file

Device Security Tests

Available in test tool versions 1.4 and above

Device Security tests as part of the service syntax tests (3.*)

The test configuration file is ‘DeviceSecurity1.SyntaxTests.xml’

Semantic Tests Verify specified output parameters, error

conditions and events Create semantic tests by editing the

<ServiceId>.SemanticTests.XML file Working groups provide the XML syntax test

configuration files Actions must be within the same service Results of one action cannot be used as

input parameters in another action

What is Not Tested Does not test performance specifications Does not certify device extensions external

to the UPnP™ standard Does not test incorrect inputs, boundary

conditions Does not test product capabilities

Tests on-the-wire interoperability

Test Environment Test tool requires Windows XP

Recommend XP SP2 (for IPv6 and HTTP/1.1 testing)

Run only one UPnP™ device and a single instance of the test tool on an isolated network

Turn off LAN connection settings (i.e. proxy settings) in Internet Options control panel on Windows XP

Test Environment Diagram

Isolated Isolated Network HubNetwork Hub

Cert Tool running on WinXPCert Tool running on WinXPDevice Under TestDevice Under Test

DHCP Server for DHCP Server for Addressing testsAddressing tests

Network Sniffer Network Sniffer (optional)(optional)

XML Configuration Files <DeviceId>.xml

Lists the implemented services and embedded devices

<ServiceId>.xml Lists the implemented Actions and State Variables

<ServiceId>.SyntaxTests.xml Lists the Action names and the ‘In’ arguments for each Action

<ServiceId>.SemanticTests.xml Lists the Action names, ‘In’ arguments and the expected ‘out’

arguments or error code.

Using XML Config Files Working Committees

Write the XML config files for standardized DCPs Distribute XML config files on the UIC web site

The test tool uses the XML files to verify Devices have all required services Services implement required Actions and State

Variables Devices respond correctly for each Action

invocation Proprietary devices can be tested but require

the respective XML config files

Device Description<device>

<deviceType>urn:schemas-upnp-org:device:WANDevice:1</deviceType>

</device><device>

<Optional/> <deviceType>urn:schemas-upnp-

org:device:LANDevice:1</deviceType></device>

Service Description<serviceStateTable> <stateVariable> <name>EthernetLinkStatus</name> <dataType>string</dataType> <allowedValueList> <allowedValue>Up</allowedValue> <allowedValue>Down</allowedValue> </allowedValueList> </stateVariable></serviceStateTable>

Syntax Tests XML File<TestCase>

<Id>1</Id><Category>Valid Action And Valid InArgs</Category><ActionName>GetEthernetLinkStatus</ActionName>

<ExpectedReturnCode>ACTION_AND_INARGS_ARE_VALID</ExpectedReturnCode>

</TestCase>

Semantic Tests XML File<ManualAction>

<ActionInstruction> Remove cable. </ActionInstruction><VerificationInstruction> Verify link light is off.

</VerificationInstruction><ExpectedSuccessResponse/>

</ManualAction>

Test Tool UI

Select Test Options

Step Mode / Run Until Failure Step Mode

Pauses between each test Click the “Next” button to continue Useful when debugging a failure

Run Until Failure Pauses the test execution at the first failure

Interpreting the Test LogTest Case 2.0.0.1 PassedTest Case 2.1.0 Failed to RunFailure Reason :Precondition Failure - Unable to

Construct Device Discovery Message List.Reason : Unable to Acquire Root Device Description

Url.Badly formed Notify Alive Packets Header Value.

Reason : NT Header Value = TestDevice is Not Compliant to UDA V1.0

Helpful Hints Review the test log for failure details Use a network sniffer

UPnP™ parser available for Microsoft Network Monitor

Useful for reporting problems Use Step Mode to help isolate an issue Refer to the Test Tool User documentation

for descriptions of each test case

Helpful Hints Continued A full certification pass takes about 25

minutes to complete For testing purposes, set the CACHE-

CONTROL in the device to be less than 60 for SSDP alive notifications

In your final device implementation, set the CACHE-CONTROL according to UDA recommendations

Update on next release v1.5 Planned for 2nd Quarter 2005 New features

Using MSEARCH to initiate tests instead of NOTIFY for faster test runs

Improved logging messages for better diagnostics Testing on non isolated networks and presence of

multiple root devices Embedded devices testing Comparing arguments and passing parameters

across different actions Presence of extra header information in Discovery,

Description, Control and Eventing tests

Q&A Certification Test Tool

Is made available by UIC Tool is posted on www.upnp-ic.org Documentation is included with the tool For debugging, Cert Tool in UPnP™ Forum website has

sources Support

UIC members receive one free support call Newsgroup: microsoft.public.upnp

Questions Email: upnpcert@microsoft.com

Foundation of the Connected Home