Continuous Delivery with TFS / VSTS – Enhancing a CI Build to Help Bake Quality In

In the previous instalment of this blog post series on Continuous Delivery with TFS / VSTS we created a basic CI build. In this post we enhance the CI build with further configurations that can help bake quality in to the application. Just a reminder that I’m using TFS to create my CI build as it’s the lowest common denominator. If you are using VSTS you can obviously follow along but do note that screenshots might vary slightly.

Set Branch Policies

Although it's only marginally related to build this is probably a good point to set branch policies for the master branch of the ContosoUniversity repository. In the Web Portal for the team project click on the cog icon at the far right of the blue banner bar:

This will open up the Control panel at the team project administration page. Navigate to the Version Control tab and in the Repositories pane navigate down to master. In the right pane select Branch Policies:

The branch policies window contains configuration settings that block poor code from polluting the code base. The big change is that the workflow now changes from being able to commit to the master branch directly to having to use pull requests to make commits. This is a great way of enforcing code reviews and I have more detail on the workflow here. In the screenshot above I've selected all the options, including selecting the ContosoUniveristy.CI build to be run when a pull request is created. This blocks any pull requests that would cause the build to fail. The other options are self explanatory, although enforcing a linked work item can be a nuisance when you are testing. If you are testing on your own make sure you Allow users to approve their own changes otherwise this will cause you a problem.

Testing Times

The Contoso University sample application contains MSTest unit tests and we want these to be run after the build to provide early feedback of any failing tests. This is a achieved by adding a new build step. On the Build tab in the Web Portal navigate to the ContosoUniversity.CI build and place it in edit mode. Click on Add build step and from the Add Tasks window filter on Test and choose Visual Studio Test.

For our simple scenario there are only three settings that need addressing:

• Test Assembly -- we only want unit tests to run and ContosoUniversity contains other tests so changing the default setting to **\*UnitTests*.dll;-:**\obj\** fixes this.
• Platform -- here we use the $(BuildPlatform) variable defined in the build task.
• Configuration -- here we use the $(BuildConfiguration) variable defined in the build task.

With the changes saved queue the build and observe the build report advising that the tests were run and passed:

Code Coverage

In the above screenshot you'll notice that there is no code coverage data available. This can be fixed by going back to the Visual Studio Test task and checking the Code Coverage Enabled box. Queueing a new build now gives us that data:

Of slight concern is that the code coverage reported from the build (2.92%) was marginally higher than that reported by analysing code coverage in Visual Studio (2.89%). Whilst the values are the same for all practical purposes the results suggest that there is something odd going on here that warrants further investigation.

Code Analysis

A further feedback item which is desirable to have in the build report is the results of code analysis. (As a reminder, we configured this in Visual Studio in this post so that the results are available after building locally.) Displaying code analysis results in the build report is straightforward for XAML builds as this is an out-of-the-box setting -- see here. I haven't found this to be quite so easy with the new build system.There's no setting as with XAML builds but that shouldn't be a problem since it's just an MSBuild argument. It feels like the correct argument should be /p:RunCodeAnalysis=Always (as this shouldn't care how code analysis is configured in Visual Studio) however in my testing I couldn't get this to work with any combination of the Visual Studio Build / MSBuild task and release / debug configurations. Next argument I tried was /p:RunCodeAnalysis=True. This worked with either Visual Studio Build or MSBuild task but to get it to work in a release configuration you will need to ensure that code analysis has been enabled for the release configuration in Visual Studio (and the change has been committed!). The biggest issue though was that I never managed to get more than 10 code analysis rules displayed in the build report when there were 85 reported in the build output. Perhaps I'm missing something here -- if you can shed any light on this please let me know!

Don't Ignore the Feedback!

Finally, it may sound obvious but there's little point in configuring the build report to contain feedback on the quality of the build if nobody is looking at the reports and is doing something to remedy problems. However you do it this needs to be part of your team's daily routine!

Cheers -- Graham

Continuous Delivery with TFS / VSTS – Configuring a Basic CI Build with Team Foundation Build 2015

In this instalment of my blog post series on Continuous Delivery with TFS / VSTS we configure a continuous integration (CI) build for our Contoso University sample application using Team Foundation Build 2015. Although the focus of this post is on explaining how to configure a build in TFS or VSTS it is worth a few words on the bigger picture as far as builds are concerned.

One important aspect to grasp is that for a given application you are likely to need several different builds for different parts of the delivery pipeline. The focus in this post is a CI build where the main aim is the early detection of problems and additional configurations that help bake quality in. The output from the build is really just information, ie feedback. We're not going to do anything with the build itself so there is no need to capture the compiled output of the build. This is just as well since the build might run very frequently and consequently needs to have a low drain on build server resources.

In addition to a CI build a continuous delivery pipeline is likely to need other types of build. These might include one to support technical debt management (we'll be looking at using SonarQube for this in a later post but look here if you want a sneak preview of what's in store) and one or more that capture the compiled output of the build and kick-off a release to the pipeline.

Before we get going it's worth remembering that as far as new features are concerned VSTS is always a few months ahead of TFS. Consequently I'm using TFS to create my CI build as it's the lowest common denominator. If you are using VSTS you can obviously follow along but do note that screenshots might vary slightly. It's also worth pointing out that starting with TFS 2015 there is a brand new build system that is completely different from the (still supported) XAML system that has been around for the past few years. The new system is recommended for any new implementations and that's what we're using here. If you want to learn more I have a Getting Started blog post here.

Building Blocks

The new build system (which Microsoft abbreviates as TFBuild) is configured from the Web Portal rather than Visual Studio, so head over there and navigate to your team project and then to the Build tab. Click on the green plus icon in the left-hand pane which brings up the Definition Templates window. There's a couple of ways to go from here but for demonstration purposes select the Empty option:

This creates a new empty definition, which does involve a bit of extra work but is worth it the first time to help understand what's going on. Before proceeding lick on Save and provide a name (I chose ContosoUniversity.CI) and optionally a comment for the version control history. Next click the green plus icon next to Add build step to display the Add Tasks window. Take a minute to marvel at the array of possibilities before choosing Visual Studio Build. This gives us a skeleton build which needs configuring by working through the different tabs:

There are many items that can be configured across the different tabs but I'm restricting my explanation to the ones that are not pre-populated and which required. You can find out more about the Visual Studio Build task here.

On the Build tab:

• Platform relates to whether the build should be x86, x64 or any cpu. Whilst you could specify a value here the recommendation is to use a build variable (defined under Variables -- see below) as Platform is a setting used in other build tasks. An additional advantage is that the value of the variable can be changed when the build is queued. As per the documentation I specified $(BuildPlatform) as the variable.
• Configuration is the Visual Studio Solution Configuration you want to build -- typically debug or release. This is another setting used in other build tasks and which warrants a variable and I again followed the documentation and used $(BuildConfiguration).
• Clean forces the code to be refreshed on every build and is recommended to avoid possible between-build discrepancies.

On the Repository tab:

• Clean here appears to be the same as Clean on the Build tab. Not sure why there is duplication or why it is a check box on the Build tab and a dropdown on this tab but set it to true.

On the Variables tab:

• Add a variable named BuildPlatform, specify a value of any cpu and check Allow at Queue Time.
• Add a variable named BuildConfiguration, specify a value of release and check Allow at Queue Time.

On the Triggers tab:

• Continuous Integration should be checked.

That should be enough configuration to get the build to go green. Perform a save and click on Queue build (next to the save button). You will see the output of the build process which should result in the build succeeding:

It's All in the Name

At the moment our build doesn't have a name, so to fix that first head over to the Variables tab and add MajorVersion and MinorVersion variables and give them values of 1 and 0 respectively. Also check the Allow at Queue Time boxes. Now on the General tab enter $(BuildDefinitionName)_$(MajorVersion).$(MinorVersion).$(Year:yyyy)$(DayOfYear)$(Rev:.r) in the Build number format text box. Save the definition and queue a new build. The name should be something like ContosoUniversity.CI_1.0.2016019.2. One nice touch is that the revision number is reset on a daily basis providing an easy way keep track of the builds on a given day.

At this point we have got the basics of a CI build configured and working nicely. In the next post we look at further configurations focussed on helping to bake quality in to the application.

Cheers -- Graham