ios unit testing and code coverage
DESCRIPTION
iOS Unit Testing and Code Coverage. Amit Rao Mint iOS Developer [email protected]. Key Takeaway. Move Quality close to the developer desktop with unit testing, code coverage, static analysis, code reviews and warning free builds. Assumptions and Pre-Requisites. - PowerPoint PPT PresentationTRANSCRIPT
Key Takeaway
• Move Quality close to the developer desktop with unit testing, code coverage, static analysis, code reviews and warning free builds.
3
Assumptions and Pre-Requisites• Basic working knowledge of XCTest• Xcode 5.1.1• Jenkins (1.564 or above) installed and basic working knowledge of Jenkins job
creation and configuration• At least a couple of unit test already written • Unit tests can be invoked successfully from command line
High Level Steps
• Turn on code coverage flags in Xcode Project• Add Logic to generate code coverage files in source code• Install Code Coverage Tool and Plugin for Jenkins Server• Configure Unit Test Job with Code Coverage
Unit Testing Resources
• http://qualitycoding.org/
• OCMock (http://ocmock.org/reference/)
• OCHamcrest/OCMockito
• CoverStory
Unit Testing CI Methodolgy
• Develop unit tests while developing app code• Use CoverStory to view coverage on desktop• Post check-in verify unit tests and code
coverage report via CI system.
7
Step 1
Turn on Code Coverage Flags in xCode Project
8
Turn on Code Coverage Flags in xCode Project
Set the following flags for the main target to ‘Yes’:• Generate Test Coverage Files• Instrument Program Flow
9
Step 2
Add Logic to Generate Code Coverage Files in Source Code
10
Add Logic to Generate Code Coverage Files in Source Code
• As of Xcode 4.x, code coverage files are no longer auto generated by xCode.
• __gcov_flush(); needs to be called in the application delegate after all tests are run
11
Add Logic to Generate Code Coverage Files in Source Code
• To do so:– Create ‘test observer’ to notify application delegate when tests are done. *The test
observer should be a member of the unit test target– Eg:
12
Add Logic to Generate Code Coverage Files in Source Code
• Add observer when application delegate loads
• Enhance applicationWillTerminate of application delegate to call __gcov_flush();
• Note might be a good idea to use #ifdef to only call this during unit testing
13
Step 3
Install Code Coverage Tool and Plugins to Jenkins Server
14
Install Code Coverage Tool and Plugins to Jenkins Server
• Gcovr– generating summarized code coverage results from code generation files. – produces either compact human-readable summary reports, machine readable XML
reports or a simple HTML summary.• Cobertura plugin
– calculates the percentage of code accessed by tests. It can be used to identify which parts of the program are lacking test coverage
– Renders the xml report generated by Gcovr to a navigable code coverage dashboard
15
Install gcovr• Download the latest gcovr installation on server hosting Jenkins from
https://github.com/gcovr/gcovr/releases
• Get the tar.gz file of the latest release and uncompress it in a temp directory
• Copy the gcovr-X.X/scripts/gcovr to an accessible directory e.g. /usr/local/bin or /usr/bin or /Applications
– Note: Gcovr is installed on /Applications/Scripts on the TT iPad build server
• Run gcovr –-version to ensure installation> <your path>/gcovr --versiongcovr 3.1
Note: if gcovr is in your path, you can just type gcovr --version
16
Install Cobertura plugin
17
Add Code Coverage Plugins to Jenkins- Install Cobertura plugin
Start typing the word Cobertura in the Filter text box and the list below will expand and show the Cobertura (do not hit return after typing the word Cobertura
18
Add Code Coverage Plugins to Jenkins- Install Cobertura plugin
19
Step 4
Configure Unit Test Job with Code Coverage
20
Configure Unit Test Job with Code Coverage
• Creating a separate job from main build is a good idea while configuring and fine tuning the process
21
Configure Unit Test Job with Code Coverage
22
Configure Unit Test Job with Code Coverage
23
Configure Unit Test Job with Code Coverage
• Note: at this point, we will be writing a lot of instructions that can be executed on the command line• it would be a good idea to run these commands on the build server which
hosts Jenkins to make sure they work before configuring them onto Jenkins
24
Configure Unit Test Job with Code Coverage – run unit test
• *Change directory to location of xcode projectcd /Users/devmac/Developer/Mint/trunk_svn1.7/Gala
• Run xcodebuild command linexcodebuild –project Gala.xcodeproj -scheme ’Mint.com Debug-Internal' -sdk iphonesimulator7.1 clean test | ocunit2junit
• *Note: You don’t have to change directory and instead use an absolute path but I’d like to break things up a bit so the commands are shorter and more readable
• Ocunit2junit is a ruby script that generates the unit test summary report
25
Configure Unit Test Job with Code Coverage – run gcovr to generate Cobertura xml report
• Change directory to location of code coverage filescd "`xcodebuild –project Gala.xcodeproj -showBuildSettings -sdk iphonesimulator7.1 test | grep OBJECT_FILE_DIR_normal |grep -oEi "/.*" | awk '{print $0"/i386"}'`”
Note: The code generated files are placed in a xCode DerivedData folder which is xCode assigns. The script above is one way to determine the DerivedData folder. The method I use is to run a script called “exportenv.sh” which generates a file called “env.sh” with an environment variable to the DerivedData folder. For more details please refer this link.
26
Configure Unit Test Job with Code Coverage – run gcovr to generate Cobertura xml report
• Run gcovr and create the coverage.xml file/Applications/scripts/gcovr --object-directory . --exclude 'NS*’ --exclude '.*Frameworks.*' --exclude '.*include.*’ --xml > ${WORKSPACE}/coverage.xml
• Note exclude what you don’t need!• Note {WORKSPACE} is a Jenkins env variable that
indicates the Jenkin user’s workspace
27
Configure Unit Test Job with Code Coverage
• Putting it together …
28
Configure Cobertura plugin to render coverage report
• Add a post build action
29
Configure Cobertura plugin to generate coverage and unit tests report
30
Run unit test build jobNow lets give our unit test job a run by selecting ‘Build Now’If successful, an entry will show up under Build History, for our example, lets look at #45 Jun 5, 2014 3:04:37
31
Run unit test build job
32
Run unit test build job
33
Run unit test build jobDrilling down to individual files
Green section indicates code that are covered by unit testPink section indicates code that are not covered
34
Add step to invoke unit test build from main build
Demo