© 2012 Autodesk

Automated Testing with the AutoCAD® .NET API

Scott McFarlaneSenior Software Engineer, Woolpert, Inc.

© 2012 Autodesk

Learning Objectives

At the end of this class, you will be able to: Recognize the obstacles to doing software testing, particularly with

programs written for AutoCAD Understand the goals, philosophy and principles of test automation Appreciate the benefits of test automation Use the correct Terminology: What is a mock, a stub, a fake, a dummy? Use abstraction and dependency inversion to write testable code

© 2012 Autodesk

Automated Testing

© 2012 Autodesk

Why Am I Not Doing Automated Testing?

My programs always work correctly I never need to modify or refactor my code I enjoy spending hours tracking down bugs I haven’t yet realized the benefits I haven’t learned to do it well

© 2012 Autodesk

Code-Driven Testing Frameworks

The growing family of code-driven testing frameworks have come to be known collectively as xUnit

Based on a design by Kent Beck, originally implemented for Smalltalk as SUnit, then ported to Java as JUnit.

Wikipedia lists over 360 testing frameworks for over 70 different languages and environments.

© 2012 Autodesk

Basic xUnit Architecture

Test Case Also called Testcase Class Implemented as an interface, abstract base class, or class attribute Contains one or more Test Methods Typically shares the same Test Fixture

Test Method Contains the logic for each test

© 2012 Autodesk

Basic xUnit Architecture

Test Suite A set of tests that share the same Test Fixture

SUT (System Under Test) The actual class/method we are testing

Test Fixture All the things we need to have in place to run a test

4-Step Test Execution 1. Setup 2. Exercise 3. Verify 4. Teardown

© 2012 Autodesk

Topics

The Goals of Test Automation Philosophy and Principles of Test Automation Terminology: What is a mock, a stub, a fake, a dummy?

© 2012 Autodesk

© 2012 Autodesk

Goals of Test Automation

Tests should help us improve quality Tests should help us understand the system under test (SUT) Tests should reduce (and -not introduce) risk Tests should be easy to run Tests should be easy to write and maintain Tests should require minimal maintenance as the system evolves

around them

© 2012 Autodesk

Tests Should Help Us Improve Quality

What is Quality? 1. Is the software built correctly? 2. Have we built the correct software?

Tests act as an executable specification Tests help prevent bugs Tests help localize bugs But… Quality is not just testing

“Trying to improve the quality of software by doing more testingis like trying to lose weight by weighing yourself more often.”

- Steve McConnell

© 2012 Autodesk

Tests Should Help Us Understand the SUT

Rather than pouring over the code to answer: What is this code supposed to do? What would the result be if . . . ?

Tests should show the test reader how the code is supposed to work. Tests are less likely to become out-of-sync with code than

documentation

© 2012 Autodesk

Tests Should Reduce (Not Introduce) Risk

Tests provide a safety net for incrementaldevelopment and refactoring Without tests, we are reluctant to make

changes or try different approaches – risk is too high. Writing good tests is not easy – It takes lots of practice

Keep tests simple, clean, easy to read Don’t be afraid to use Test Utility Methods.

Dangers (Risks) of Automated Testing Putting test logic in production code Testing too many things at once Testing too little

© 2012 Autodesk

Tests Should Be Easy to Run

Tests must be… Fully automated Self-checking Independent Repeatable Fast

© 2012 Autodesk

Tests Should Require Minimal Maintenance as the System Evolves Around Them Test only public interfaces and outcomes when possible Avoid testing private methods directly (implementation details) Use Mocks only when necessary Minimize duplicate code in tests

Tests should facilitate (not hinder) our ability to make changes to the production code.

© 2012 Autodesk

Summary of Goals

Trustworthiness Are you able to run your test and trust the results?

Maintainability Treat your test code with the same care as your production code.

Readability Can you look at a test and quickly understand what is being tested and why it

failed?

© 2012 Autodesk

Testing Philosophy

Test first versus test after Test-by-test versus test all-at-once Outside-in versus inside-out Behavior verification versus state verification

© 2012 Autodesk

Definitions (Meszaros)

Test Double – Any object that is used in place of a real object for the purposes of testing. Mocks, stubs, fakes and dummies are all test doubles. Mock – an object that can monitor the indirect outputs (behavior) of the SUT. Your

tests assert against the mock object (rather than the SUT itself) to see if it has passed.

Stub – an object that can control indirect inputs into the SUT. A test can use multiple subs, but can only have one mock object.

Fake – an object used by the SUT that behaves exactly like the real object, but may be used to isolate the SUT from external resources. An in-memory database is an example of a Test Double.

Dummy – an object that is required by the SUT to run, but plays no active role in the test.

© 2012 Autodesk

Behavior Verification vs. State Verification

Behavior Verification State Verification

Class under test Mock

Test

Communicate

Assert

Class under test Stub

Test

Communicate

Asser

t

DependentComponent

DependentComponent

© 2012 Autodesk

Unit Tests vs. Integration Tests

Unit Tests Should run w/o config Isolated No teardown required Run in memory Fast Repeatable Run often

Integration Tests May require config Use external resources May require teardown May use disk or database Slow May not be repeatable Run less often

© 2012 Autodesk

Unit Tests vs. Integration Tests

Do not mix - Keep in separate projects/assemblies

© 2012 Autodesk

Unit Testing “Thorns”

MessageBox.Show() Timer WebClient System.IO (File, etc.) DateTime.Now Singletons/Statics

© 2012 Autodesk

Testing Tools

Testing Framework (xUnit) MSTest NUnit MbUnit/Gallio

Isolation Framework NSubstitute Moq FakeItEasy

Other Fluent Assertions NCrunch

© 2012 Autodesk

Obstacles to Testing AutoCAD Code

© 2012 Autodesk

Obstacles to Testing AutoCAD Code

Our programs are hosted by AutoCAD Heavy user interaction Tightly coupled with AutoCAD API

© 2012 Autodesk

Example

© 2012 Autodesk

Abstraction and Dependency Injection

© 2012 Autodesk

Why is Abstraction Important?

The Problem Classes often contain dependencies on other classes to do their work. If a class makes assumptions about how its dependent classes are

implemented, the class becomes difficult to reuse in combination with other components.

Such classes are also very difficult to unit test because they cannot be isolated from their dependencies.

© 2012 Autodesk

Why is Abstraction Important?

The Solution Establish a common set of protocols by which classes interact, separately

from the classes themselves. Promotes the use of software design patterns (particularly dependency

injection) that result in code that is more testable, extensible, maintainable, scalable, and reusable.

© 2012 Autodesk

Typical Application

Business Logic

Logger

FILE

User Interface

Exception Handler

Data Access

Web Service Client

Component

Component

Configuration

© 2012 Autodesk

Business Logic

User Interface

Logger

FILE

Exception Handler

Data Access

Web Service Client

Component

Component

Configuration

IDataRepository

ILogger

IMessageBox

IProgressBar

IExceptionHandler

ISomeWebService

Interface

Interface

IConfiguration

IHostApplication

Host Application

Typical Application

© 2012 Autodesk

Business Logic

Test Double

Test Double

Test Double

Test Double

Test Double

Test Double

Test Double

Test Double

IDataRepository

ILogger

IMessageBox

IProgressBar

IExceptionHandler

ISomeWebService

Interface

Interface

IConfiguration

Test DoubleIHostApplication

Typical Application

© 2012 Autodesk

Example 1public class Example1{

public void DoTheWork(){

DataRepository dataRepository = new DataRepository();Logger logger = new Logger();

logger.Log("Getting the data");

DataSet theData = dataRepository.GetSomeData();

// Do some work with the data...

logger.Log("Done.");}

}

© 2012 Autodesk

Example 2public class Example2{

private readonly IDataRepository _dataRepository;private readonly ILogger _logger;

public Example2(IDataRepository dataRepository, ILogger logger){

_dataRepository = dataRepository;_logger = logger;

}

public void DoTheWork(){

_logger.Log("Getting the data");

DataSet theData = _dataRepository.GetSomeData();

// Do some work with the data...

_logger.Log("Done.");}

}

© 2012 Autodesk

Software Engineering Principles

Separation of Concerns – This class is now only responsible for the specific job it was designed to do.

Abstraction – By using interfaces, we have established a set of protocols by which the components interact, separately from the classes themselves.

Inversion of Control – The class has relinquished control of the creation and initialization of its dependencies.

Dependency Injection – This pattern is based on Inversion of Control, and describes the way in which an object obtains references to its dependencies.

© 2012 Autodesk

Application Architecture

AutoCADMain Plug-

in Code (Bootstrapper)

AutoCAD Specific

LogicBusiness Logic

AcDbMgd.dllAcCoreMgd.dll

AcMgd.dll

Shared Abstraction

s

User Interface

Other Dependencies

LoggingConfigurationDatabasesWeb ServicesEtc…

Depends on AutoCAD API

No dependency on AutoCAD API

InterfacesDomain Objects

Implements

AcCoreConsole.exe

Implem

ents

Unit Tests

© 2012 Autodesk

Example

© 2012 Autodesk

Autodesk, AutoCAD* [*if/when mentioned in the pertinent material, followed by an alphabetical list of all other trademarks mentioned in the material] are registered trademarks or trademarks of Autodesk, Inc., and/or its subsidiaries and/or affiliates in the USA and/or other countries. All other brand names, product names, or trademarks belong to their respective holders. Autodesk reserves the right to alter product and services offerings, and specifications and pricing at any time without notice, and is not responsible for typographical or graphical errors that may appear in this document. © 2012 Autodesk, Inc. All rights reserved.