devops on ms cloud: o365 & azuredevopsonms-cloud.azurewebsites.net/devopsdocs/devops on ms...

17
1 DEVOPS ON MS CLOUD: O365 & AZURE An Overview Contributors: Arnab Bhowmick Soumen Pal Build & Deploy Test Release Monitor Plan HCL Technologies Enterprise Productivity Service Line – Modern Apps

Upload: others

Post on 15-Mar-2020

10 views

Category:

Documents


0 download

TRANSCRIPT

  • 1

    DEVOPS ON MS CLOUD: O365 & AZURE An Overview

    Contributors:

    Arnab Bhowmick Soumen Pal

    Build & Deploy

    Test

    Release

    Monitor

    Plan

    HCL Technologies Enterprise Productivity Service Line – Modern Apps

  • DevOps on MS Cloud: O365 & Azure

    2

    Contents Introduction: ........................................................................................................................................................................... 3

    Challenges of Release management of provider hosted add-ins without DevOps ................................................................ 4

    Continuous deployment and release management with DevOps .......................................................................................... 5

    DevOps process for SharePoint Provider Hosted App ............................................................................................................ 6

    Process Steps ...................................................................................................................................................................... 6

    Implementation of DevOps for MS Cloud (Provider Hosted Add-In) ..................................................................................... 7

    Prerequisites: ...................................................................................................................................................................... 7

    Implementation Steps: ........................................................................................................................................................ 7

    Conclusion – DevOps on MS Cloud ....................................................................................................................................... 16

    Additional Articles ................................................................................................................................................................. 17

    Abbreviation.......................................................................................................................................................................... 17

    Figures Figure 1: DevOps process for Provider Hosted Add-In ........................................................................................................... 6

    Figure 2: Create new Build Definition ..................................................................................................................................... 7

    Figure 3: Select Source Repository ......................................................................................................................................... 8

    Figure 4: Default Build Activities ............................................................................................................................................. 9

    Figure 5: Configure Build Solution Activity ............................................................................................................................. 9

    Figure 6: Build Console for Successful Build ......................................................................................................................... 10

    Figure 7: Build Summery ....................................................................................................................................................... 11

    Figure 8: Build Artifacts ......................................................................................................................................................... 11

    Figure 9: Create Release Definition ...................................................................................................................................... 12

    Figure 10: Configure Release Definition ............................................................................................................................... 13

    Figure 11: Empty Release Definition ..................................................................................................................................... 14

    Figure 12: Adding Activities for SIT Environment ................................................................................................................. 14

    Figure 13: Add Tokens as Environment Variables ................................................................................................................. 15

    Figure 14: Configure Multiple Environments for Release Management .............................................................................. 16

  • DevOps on MS Cloud: O365 & Azure

    3

    Introduction:

    “DevOps (a clipped compound of development and operations) is a term used to refer to a set of practices that

    emphasizes the collaboration and communication of both software developers and other information-technology (IT)

    professionals while automating the process of software delivery and infrastructure changes.” - Wikipedia

    DevOps has been conceptualized to overcome challenges of application development life cycle management. There are

    various challenges prevails for any ALM.

    Different but interdependent IT worldviews

    o Enterprise world vs Agile world

    o Developers vs Operations staff

    Business demand is increasing

    Pace of Software change is increasing

    Issue with release process

    o Slow, Painful, Manual, Buggy, Unverifiable, Inconsistent

    DevOps practice addresses these above-mentioned challenges and helps mitigate those by:

    Improved communications and working culture between Developers & Operations

    Faster release process (TTM)

    Better Software Quality

    Predictable processes ensuring consistency

    Better Metrics and Insights

    DevOps is not just adoption of any particular tool or technology for automation. DevOps is a culture or practice and is a

    long transformation journey. It is a combination and collaboration of 3 P’s:

    People

    Processes &

    Products (Tools)

    Successful implementation of DevOps in an organization depends on right people and correct processes more than

    adopted tools. Right minded Developers and Operations people need to work side-by-side with great collaboration

    breaking the traditional barrier. A huge number of tools available in market to establish the following processes:

    Source Control Branches

    Continuous Integration

    Automated Testing

    Continuous Deployment

    Infrastructure as Code (IaC)

  • DevOps on MS Cloud: O365 & Azure

    4

    Configuration Management

    Release Management

    Application Performance Monitoring

    DevOps can be very beneficial for MS cloud based development and release management. Microsoft is one of the leaders

    in Gartner’s magic quadrant for ADLM (February 2015).

    Visual Studio Team Services provides complete DevOps framework for implementing all the necessary processes. It uses

    both Microsoft products or can be integrated with various other Tools to implement those processes.

    We will discuss a scenario based example here. We will describe what are the challenges of release management process

    of a provider hosted add-ins and how those can be mitigated by implementing DevOps on MS cloud.

    Challenges of Release management of provider hosted add-ins without DevOps Deployment and release management of SharePoint Provider Hosted Add-ins is tricky. The add-in (app) package is

    intended to be created separately for every environment because it should contain a client id and secret specific to the

    environment (O365 tenant). Also, the remote web need to be deployed to azure app service. Same client id and secret

  • DevOps on MS Cloud: O365 & Azure

    5

    need to be configured to the remote azure web as well so that the app from host/app web can communicate with the

    remote web. Following are the challenges if this deployment and release process need to manage manually.

    Separate build and deployment effort is required for each environment

    Separate web.config file need to be maintained to have environment specific client id, secret and other tokens.

    Environment configuration may vary for different environment leading to some error

    Slower, unreliable release process and higher time-to-market for any feature.

    Continuous deployment and release management with DevOps Continuous deployment and release pipeline of SharePoint provider hosted add-in can be created using visual studio

    team services (VSTS) and some PowerShell scripts. It can fully automate the deployment of provider hosted add-in on

    SharePoint Online (O365) and Azure. Similar process can be set for on premise deployment as well.

    Continuous integration will help to build the code after every check-in. This helps in early detection of any build

    break.

    Setting up automated unit testing will help to catch any code error during build. QUnit can be used to achieve

    this.

    Continuous deployment helps to deploy the remote web on an azure web app after successful build. Also,

    SharePoint .app package will be installed on SharePoint Online developer site or any test site for further

    functional testing.

    Automated functional testing can be configured through VSTS using Selenium.

    Release definition will help to deploy the app in multiple environment without any manual work.

  • DevOps on MS Cloud: O365 & Azure

    6

    DevOps process for SharePoint Provider Hosted App

    Figure 1: DevOps process for Provider Hosted Add-In

    Process Steps 1. Developed code checked in to TFVC

    1a. User may pull the code from proper branch of TFVC

    2. Based on build definition configured in VSTS, automated build will fire once code is checked in.

    2a. Unit testing code is written in Visual Studio and QUnit. Automated Unit Testing will run during build.

    3. App package is copied to configured drop location as part of build.

    4. Replace Client Id, Client secrets and other tokens for a particular environment using PowerShell

    5. Deploy remote web to Azure web app.

    6. App package is deployed/installed to SIT /Testing environment (SharePoint Online) using PowerShell.

    7. Automated functional testing is carried out using Visual studio/MTM and/or Selenium.

  • DevOps on MS Cloud: O365 & Azure

    7

    8. After testing, the app can be deployed to UAT or other environment based on the release definition as part of

    release management process.

    8a. Optionally UAT approval and QA control approval can be configured before moving to next environment

    deployment.

    9. & 10. Based on the release management process, the app can be deployed to other environments. Same

    PowerShell script can be used in VSTS for deployment. Release can be configured as automated or manual.

    Implementation of DevOps for MS Cloud (Provider Hosted Add-In)

    Prerequisites:

    An IDE for building the code (Example: Visual Studio 2015)

    Subscription to Visual Studio Team Services (VSTS)

    Team Foundation Version Control (TFVC) or GIT

    Azure subscription

    Implementation Steps:

    1. Check-in Provider hosted add-in solution to TFVC or GIT from Visual Studio.

    2. Create a new build definition. Choose Visual Studio build template as shown below.

    Figure 2: Create new Build Definition

  • DevOps on MS Cloud: O365 & Azure

    8

    3. Next select the team project as shown below. Set the default agent queue as “hosted”. (If you want to use your

    own server for deployment you can select the agent queue as “Default”) Click on create. Make sure to check the

    “Continuous integration…” check box. It will enable automated build for every code check-in. Various types of

    code repository can be selected like TFVC, Git, GitHub, Subversion. Also, other repository tools can be hooked.

    Figure 3: Select Source Repository

  • DevOps on MS Cloud: O365 & Azure

    9

    4. Above step will create the build definition with following Build steps:

    Figure 4: Default Build Activities

    5. Build solution activity need to be configured for correct solution. Also, in order to package the .app and web site

    following build arguments need to be set.

    a. “/P:IsPackaging=True /p:DeployOnBuild=true /p:PublishProfile=Package”

    Figure 5: Configure Build Solution Activity

  • DevOps on MS Cloud: O365 & Azure

    10

    6. For Visual studio based unit testing, configure “Test Assemblies” activity with “*test*.dll”. QUnit can be used for

    unit testing of JavaScript code. For details, please refer to Additional Articles section.

    7. Save the build definition file.

    8. Queue a manual build to verify the build definition working well or modify some code and check-in to test

    whether continuous integration is set and automated build is fired. Build console will show the progress of

    activities and final result.

    Figure 6: Build Console for Successful Build

    9. Clicking on the build number after completion will open the build summary page.

  • DevOps on MS Cloud: O365 & Azure

    11

    Figure 7: Build Summery

    10. We can download the app and web packages from “Artifacts” tab.

    Figure 8: Build Artifacts

  • DevOps on MS Cloud: O365 & Azure

    12

    11. Now, create a release definition for continuous deployment. Choose an “Empty” template.

    Figure 9: Create Release Definition

  • DevOps on MS Cloud: O365 & Azure

    13

    12. Choose the project and build definition. Also remember to check “Continuous deployment” check box.

    Figure 10: Configure Release Definition

  • DevOps on MS Cloud: O365 & Azure

    14

    13. It will create an empty definition with one environment.

    Figure 11: Empty Release Definition

    14. Provide a name for the environment and add the following activities. Other activities can be added based on the

    requirement.

    Figure 12: Adding Activities for SIT Environment

  • DevOps on MS Cloud: O365 & Azure

    15

    15. PowerShell activity for “Replace Token” will replace the client id and secret for this environment before

    deployment. Token values need to be added as environment variables in release configuration.

    Figure 13: Add Tokens as Environment Variables

    16. For “Deploy Azure Web App” step, select AzureRM subscription and the web app where the compiled remote

    web can be deployed. Azure web app need to be created separately before this deployment.

    17. PowerShell can be used to install .app package to SPO.

    18. “Deploy Test Agent” activity will deploy the test agent on a VM. An Azure VM can be used for this purpose.

    19. “Run Test” activity will execute Selenium functional test. For further details on Selenium functional test, please

    refer to Additional Articles section.

    20. Add multiple environments and configure required activities based on the organization policy and requirements.

  • DevOps on MS Cloud: O365 & Azure

    16

    Figure 14: Configure Multiple Environments for Release Management

    21. Release management can be fully automated using VSTS. Also, various approval process can be set for deploying

    to any particular environment.

    Conclusion – DevOps on MS Cloud Implementing correct DevOps processes with right people and toolset will reduce the deployment/release cycle greatly.

    But, if not implemented properly, it may incur overhead instead of reducing effort. Also, people tend not to set up

    DevOps as it takes more time and effort in the initial phase. But, once it is set properly, it reduces the effort a lot and

    increases productivity. With MS Cloud environment, this even helps any organizations further as the effort towards

    environment creation is already reduced. Therefore, DevOps and MS Cloud together ensure rapid feature delivery to

    market with minimal chances of failure. Just what the software development community is looking for!

  • DevOps on MS Cloud: O365 & Azure

    17

    Additional Articles Name Article

    Qunit White Paper

    Selenium White Paper

    * additional articles could be viewed after downloading this whitepaper

    Abbreviations TTM Time To Market

    VSTS Visual Studio Team Services

    TFVC Team Foundation Version Control

    MS Microsoft

    VS Visual Studio

    SPO SharePoint Online

    ADLM Application Development Life Cycle Management

    SIT System Integration Test

    UAT User Acceptance Test

    QA Quality Assurance

    MTM Microsoft Test Manager

  • Version No Date Prepared by Reviewed by

    1.0 22nd Dec 2016 Soni Arora Soumen Pal

    QUnit Testing with Visual Studio Team

    Services

  • 1 | P a g e

    Contents QUnit Testing ..............................................................................................................................................................3

    Introduction ............................................................................................................................................................3

    Pre-Requisites .........................................................................................................................................................3

    Approach ................................................................................................................................................................3

    Creating QUnit Test Project ................................................................................................................................3

    Run Unit Test Case on VSTS ................................................................................................................................9

    Challenges............................................................................................................................................................ 15

    Abbreviations ...................................................................................................................................................... 16

  • 2 | P a g e

    Table of Figures

    Figure 1: Add Unit Test Project ...................................................................................................................................4

    Figure 2: Add Chutzpah from Visual Studio Extension ...............................................................................................5

    Figure 3: Add Chutzpah from NuGet Package ............................................................................................................5

    Figure 4: Solution File with JS Files .............................................................................................................................6

    Figure 5: Test Cases JS File ..........................................................................................................................................6

    Figure 6: JS File to run test cases ................................................................................................................................7

    Figure 7: Test Explorer ................................................................................................................................................7

    Figure 8: Test Passed ..................................................................................................................................................8

    Figure 9: Test Failed ....................................................................................................................................................8

    Figure 10: Add Solution to VSTS using Team Explorer ...............................................................................................9

    Figure 11: Create New Build Definition ................................................................................................................... 10

    Figure 12: Select Team Project for the build ........................................................................................................... 11

    Figure 13: Build Steps .............................................................................................................................................. 12

    Figure 14: Test Assemblies for JS ............................................................................................................................. 13

    Figure 15: Path to Custom Test Adapters ................................................................................................................ 13

    Figure 16: Successful Build ...................................................................................................................................... 14

    Figure 17: Test Passed Result .................................................................................................................................. 14

    Figure 18: Build Failed ............................................................................................................................................. 15

    Figure 19: Test Failed Result .................................................................................................................................... 15

  • 3 | P a g e

    QUnit Testing

    Introduction Writing unit tests for client-side code is the lack of any actual units. JavaScript code is written in any page and

    can be combined with HTML as inline event handlers as well as intermixed with back-end logic. So maintaining

    JavaScript methods become tedious each day. To improve the code design need to extract JavaScript methods

    into a separate JavaScript file. But it requires unit testing to check if the application is behaving same. Now, until

    we have solid coverage with unit tests, we need to continue manually testing to minimize that risk. But still

    there is risk in manual testing like: Misunderstanding the method output, lack of skilled testers, time

    management for the testing, a testing team handle multiple projects. The problem is, buggy JavaScript now

    deployed in the released environments.

    It will be good if there is any way, we can track the issue in JavaScript method during build then it will be easier

    for the developers to check instantly in place of waiting for testers reply. Here comes QUnit.

    QUnit is a JavaScript Unit testing framework. It's used by the jQuery, jQuery UI and jQuery Mobile projects and is

    capable of testing any generic JavaScript code.

    Pre-Requisites

    Visual Studio Online / Visual Studio Team Service account.

    Visual Studio 2012 / 2013 / 2015

    Chutzpah, an open source JavaScript test runner.

    Approach

    Creating QUnit Test Project

    Step 1: Open Visual Studio

    Step2: Create New Project of Type Unit Test Project.

  • 4 | P a g e

    Figure 1: Add Unit Test Project

    Step 3: Add Chutzpah to the project. It is available both as Visual Studio Extensions & Nuget Package. It Contains

    PhanthomJS as dependency and contains 3 JavaScript unit test libraries i.e. Jasmine, QUnit and Mocha.

    Add

    Chutzpah Test Adapter for the Test Explorer

    Chutzpah Test Runner Context Menu Extension.

  • 5 | P a g e

    Figure 2: Add Chutzpah from Visual Studio Extension

    Figure 3: Add Chutzpah from NuGet Package

    Step 4: Add JavaScript test file to add unit test cases.

  • 6 | P a g e

    Figure 4: Solution File with JS Files

    Figure 5: Test Cases JS File

    Step 5: Add JavaScript test file containing the source to be tested.

  • 7 | P a g e

    Figure 6: JS File to run test cases

    Step 6: Run the unit test in Visual Studio test explorer. All the test cases will be visible in Test Explorer.

    Figure 7: Test Explorer

    Run the test case and check the output.

  • 8 | P a g e

    Figure 8: Test Passed

    Figure 9: Test Failed

  • 9 | P a g e

    Step 7: Open Team Explorer and connect to Visual Studio Online account where the Project is created.

    Figure 10: Add Solution to VSTS using Team Explorer

    Step 8: After successful running of test cases, check –in the code to VSO.

    Run Unit Test Case on VSTS

    Step 1: Create a new Build Definition. Add Visual Studio build template as shown below

  • 10 | P a g e

    Figure 11: Create New Build Definition

    Next select the team project as shown below. Set the default agent queue as “hosted”. (If you want to use your

    own server for deployment you can select the agent queue as “Default”) Click on create.

  • 11 | P a g e

    Figure 12: Select Team Project for the build

    This will add following Build steps:

  • 12 | P a g e

    Figure 13: Build Steps

    Step 2: Add another Test Assembly task to the build definition that matches the JavaScript test files.

    Search Test Assembly with this criteria: **\*test.js;-:**\obj\**\*test.js. So it will only search JS files that are added

    for unit tests.

  • 13 | P a g e

    Figure 14: Test Assemblies for JS

    In Advance Execution Options Set Path to Custom Test Adapters. This will point to the Chutzpah Adapter added

    to the project using Nuget package. Its path will be like ‘$(Build.SourcesDirectory)\SolutionName\packages’

    Figure 15: Path to Custom Test Adapters

    Step 3: Save and Run the build. If the build is successful, go to Test Run and check the test run result

  • 14 | P a g e

    Figure 16: Successful Build

    Figure 17: Test Passed Result

    In case build fails, test run result will be as follows

  • 15 | P a g e

    Figure 18: Build Failed

    Figure 19: Test Failed Result

    Challenges 1. ##[error]System.Management.Automation.CmdletInvocationException: Could not find a part of the path

    'C:\a\1\s\Source\Packages'. ---> System.IO.DirectoryNotFoundException: Could not find a part of the

    path 'C:\a\1\s\Source\Packages'.

    2. Warning: The path 'C:\a\1\s' specified in the 'TestAdapterPath' does not contain any test adapters,

    provide a valid path and try again.

    3. Warning: The path 'C:\a\1\s/QUnitTestProject/packages' specified in the 'TestAdapterPath' does not

    contain any test adapters, provide a valid path and try again.

  • 16 | P a g e

    Resolution:

    After installing Chutzpah from “Extensions and Update” from Visual studio, make sure that Chutzpah is also

    added using Nuget package.

    In build steps “Visual Studio Test”, in Advanced Execution Options

    Set Path to Custom Test Adapters: $(Build.SourcesDirectory)/SolutionName/packages

    Abbreviations VSTS Visual Studio Team Services

    JS JavaScript

    arnab.bFile AttachmentQunit WhitePaper.pdf

  • Version No Date Prepared by Reviewed by

    1.0 22nd Dec 2016 Soni Arora Soumen Pal

    Selenium Testing with Visual Studio Team

    Services

  • 1 | P a g e

    Contents Selenium Testing ........................................................................................................................................................3

    Introduction ............................................................................................................................................................3

    Benefits ...................................................................................................................................................................3

    Pre-Requisites .........................................................................................................................................................3

    Approach ................................................................................................................................................................3

    Creating Selenium Test Project ..........................................................................................................................3

    Run Selenium Test Case on VSTS ........................................................................................................................9

    Download and Configure Agent .......................................................................................................................... 15

    Release Management .......................................................................................................................................... 16

    Abbreviations ...................................................................................................................................................... 17

  • 2 | P a g e

    Table of Figures

    Figure 1: Add Unit Test Project ...................................................................................................................................4

    Figure 2: Add Selenium Drivers from NuGet Package ................................................................................................5

    Figure 3: Code File with Test cases .............................................................................................................................5

    Figure 4: Files in Project Solution ...............................................................................................................................6

    Figure 5: Add Test Settings file ...................................................................................................................................6

    Figure 6: Configuring Test Settings .............................................................................................................................7

    Figure 7: Run Test in Test Explorer .............................................................................................................................7

    Figure 8: Test Failed ....................................................................................................................................................8

    Figure 9: Add Solution to VSTS using Team Explorer .................................................................................................9

    Figure 10: Create New Build Definition ................................................................................................................... 10

    Figure 11: Select Team Project for the build ........................................................................................................... 11

    Figure 12: Build Steps .............................................................................................................................................. 12

    Figure 13: Copy File Task for test settings ............................................................................................................... 13

    Figure 14: Path to Custom Test Adapters ................................................................................................................ 13

    Figure 15: Success Build .......................................................................................................................................... 14

    Figure 16: Test Passed Result .................................................................................................................................. 14

    Figure 17: Build Failed ............................................................................................................................................. 15

    Figure 18: Test Failed Result .................................................................................................................................... 15

    Figure 19: Create PAT .............................................................................................................................................. 16

    Figure 20: Create Release Definition ....................................................................................................................... 16

  • 3 | P a g e

    Selenium Testing

    Introduction Software Testing is a way to validate and verify the working of a particular product or application. It can be

    incorporated at various points of time in the development process depending upon the methodology and tools

    used. Automation testing process involves testing with the help of automation scripts and executing the scripts

    to run the application with the help of some automation tool. Once the script is ready then these tests can run

    quickly and efficiently. Selenium is an automated testing tool. Selenium has the ability of recording the actions

    and playback the recorded actions.

    Benefits

    Selenium is compatible with Web Applications.

    Selenium supports all versions of IE, Firefox, Safari and Opera and a few more browsers.

    Selenium uses Selenese, a high-level, cross platform language to write Selenium commands which is a domain specific language. There are 3 basic categories for the commands - named actions, assessors and assertions. To write tests there are a lot of programming languages like C#, Java, Perl, PHP, Python or Ruby.

    Pre-Requisites

    Visual Studio Online / Visual Studio Team Service account.

    Visual Studio 2012 / 2013 / 2015

    Selenium Webdriver

    Approach

    Creating Selenium Test Project

    Step 1: Open Visual Studio

    Step2: Create New Project of Type Unit Test Project.

  • 4 | P a g e

    Figure 1: Add Unit Test Project

    Step 3: Add Selenium references for the drivers to your Unit Test Project. The drivers are how the browser is

    selected for executing the test.

    Add the reference using Nuget Package.

    Drivers to be installed

    Selenium.webdriver (It will include Firefox driver by default)

    Selenium.webdriver.Chromedriver (For Chrome Browser)

    Selenium.webdriver.iedriver (For IE browser)

    Selenium.webdriver.phantomJS (For PhantomJS)

    Note: All browsers will work in interactive mode.

  • 5 | P a g e

    Figure 2: Add Selenium Drivers from NuGet Package

    Step 4: In the class file of the project put all the Test Cases. Different Tests can be written for different browsers

    using the drivers.

    Here we are using Chrome Driver.

    Figure 3: Code File with Test cases

  • 6 | P a g e

    Note: When Chrome driver is installed, the ‘chromedriver.exe’ is added to bin. Copy it either to a folder in the

    solution or at the root.

    Open properties of ChromeDriver.exe, set “Copy to output directory” as “Copy if newer”

    Figure 4: Files in Project Solution

    Step 5: Right click on the solution and add a “Test Settings” File. This is important in order to run the code using Test Explorer (Test -> Windows -> Test Explorer)

    Figure 5: Add Test Settings file

    Step 6: Configure “Test Settings” to include chromedriver.exe as a deployment item

  • 7 | P a g e

    Figure 6: Configuring Test Settings

    Step 7: Run the test in Visual Studio test explorer. All the test cases will be visible in Test Explorer.

    Figure 7: Run Test in Test Explorer

  • 8 | P a g e

    Figure 8: Test Failed

    Step 7: Open Team Explorer and connect to Visual Studio Online account where the Project is created.

  • 9 | P a g e

    Figure 9: Add Solution to VSTS using Team Explorer

    Step 8: After successful running of test cases, check –in the code to VSO.

    Run Selenium Test Case on VSTS

    Step 1: Create a new Build Definition. Add Visual Studio build template as shown below

  • 10 | P a g e

    Figure 10: Create New Build Definition

    Next select the team project as shown below. Set the default agent queue as “hosted”. (If you want to use your

    own server for deployment you can select the agent queue as “Default”) Click on create.

  • 11 | P a g e

    Figure 11: Select Team Project for the build

    This will add following Build steps:

  • 12 | P a g e

    Figure 12: Build Steps

    Note: If test runs using Chrome, FireFox or IE, then need to create own agent. This is because the hosted agent does not run in interactive mode and it does not have the various web browsers installed on it. Running tests that spawn Chrome using the hosted agent will result in error in build output:

    Unknown error: can’t find Chrome binary

    On other hand PhantomJSDriver works with the hosted build agent.

    Step 2: Set up custom build agent on the server for browser. (Download and Configure Agent)

    Step 3: Add one more Copy File to task that will copy test settings file to the drop folder.

  • 13 | P a g e

    Figure 13: Copy File Task for test settings

    Step 4: In Publish Artifacts select Artifact Type as File Share and put working folder path with

    BuildDefinitionName and BuildNumber.

    Note : The working folder should be a shared folder.

    Figure 14: Path to Custom Test Adapters

    Step 3: Save and Run the build. If the build is successful, go to Test Run and check the test run result

  • 14 | P a g e

    Figure 15: Success Build

    Figure 16: Test Passed Result

    In case build fails, test run result will be as follows

  • 15 | P a g e

    Figure 17: Build Failed

    Figure 18: Test Failed Result

    Download and Configure Agent Step 1: Create a personal access token from VSO

  • 16 | P a g e

    Figure 19: Create PAT

    Step 2: Click Download agent from VSO on the machine.

    Step 3: Run PowerShell as Administrator.

    Step 4: Run the commands under Create the agent

    Step 5: Run config.cmd and respond to the prompts.

    Here need to enter the PAT obtained from VSO and run the agent in interactive mode.

    Set the working folder

    Step 6: Run ./run.cmd to start the agent for listening the request.

    Release Management

    Once the build is successful and all the test assemblies found in artifacts folder, it can be released.

    Step 1: Create New release definition and link it to the successful build.

    Figure 20: Create Release Definition

  • 17 | P a g e

    Step 2: Add an environment and add following three tasks

    Azure App Service deploy

    Visual Studio Test Agent Deployment

    Visual Studio Test

    Configure each task and then release.

    Abbreviations VSTS Visual Studio Team Services

    PAT Personal Access Token

    VSO Visual Studio Online

    IE Internet Explorer

    arnab.bFile AttachmentSelenium WhitePaper.pdf