Azure, TeamCity and OctopusDeploy - Part 1

Recently, Mohamed (a fellow Readifarian) and I spent some time creating a complete build and deployment pipeline. Our main goal was to create a fully functional CI server in Azure, with TeamCity for builds and OctopusDeploy for deployments.

Why? Firstly, although we both had some experience in parts of the technologies involved, neither of us had built a complete end-to-end environment, from scratch. Just sounded like too much fun!

Secondly, during my career as a software developer one of the things I’ve noticed is that many sites do not have a fully automated build and deployment environment. This just makes life incredible difficult, for a large number of reasons, including:

• builds are not repeatable,
• merging long lived feature branches becomes a nightmare,
• creating a release is often a manual and error prone exercise, and
• deployment out to production is a cross-your-fingers, roller-coaster ride of disappointment.

This post includes:

• the key learnings we had while setting up the environment, and
• links to some more detailed information we found useful.

This post won’t include:

• detailed step-by-step instructions on setting up Azure, TeamCity and OctopusDeploy (there are plenty of those, see below), or
• automation with PowerShell (that’s for another post!).

tl;dr #

The executive level summary is to go read this post by Andy French. It’s what we used as a guide and it provides great step-by-step instructions for getting a build server up and running (including Azure VM and TeamCity).

From there you just need to get Octopus installed and you’re away!

Of course, there were a number of little (and not so little) issues we encountered along the way, which the rest of this post will cover.

Creating the Azure VM #

Creating a VM on Azure is remarkably simple. The basic order of things is to:

1. Create a Virtual Network
2. Create a Storage Account
3. Create a SQL Server DB
4. Create a Cloud Service
5. Create a Virtual Machine

Step 1 - Virtual Network #

For our needs, setting up a Virtual Network was overkill, thus we skipped this step. However, if you would like to setup a multi-machine environment on the one sub-net in Azure, then you will need to do this. For us the stand-alone VM option was perfectly suitable.

Step 2,3 - Storage and Database #

Steps 2 and 3 are optional. A Storage Account will be automatically created for you when you create the VM if you don’t already have one defined.

As we’re using TeamCity, we chose to create a Azure SQL Server DB. TeamCity has it’s own, built-in database (HSQLDB), which you could use as a trial, but it’s not for anything else. From the TeamCity documentation:

The internal database may crash and lose all your data (e.g. on out of disk space condition). Also, internal database can become extremely slow on large data sets (say, database storage files over 200Mb). Please also note that our support does not cover any performance or database data loss issues if you are using the internal database.
In short, do not EVER use internal HSQLDB database for production TeamCity instances.

Note that the emphasis above is from the TeamCity doco, not mine.

Lastly, if you’re going to be using TeamCity for non-Windows builds, it’s strongly recommended to select a collation type ending in ‘CS_AS’. From the TeamCity SQL Server documentation:

As the primary collation, it is recommended to use the collation corresponding to your locale. We also suggest using the case-sensitive collation (collation name ending with ‘CS_AS’), which is mandatory for the certain functionality (like using non-Windows build agents).

And here’s a screen capture of that setting in the classic portal:

And here’s what the setting looks like in the preview portal:

Step 4 - Cloud Service #

A Cloud Service is a (Azure processing unit) container for one or more virtual machines, giving you the ability to load balance your service. The image below is from the linked Cloud Services article above and gives a brief description of the different hosting models for you application:

In this instance, we wanted to create our own VM, but we did not need the added features (and complexity) of the Cloud Services model. Azure makes this easy, as when you create the VM you can either:

• select an existing Cloud Service, or
• one will be automatically created for you.

We took the latter option.

Also note that Azure will automatically name the Cloud Service with the same name as the VM, which means they will have the same DNS name (eg, the service and the VM will both be at myvm.cloudapp.net). There were some articles on SO that indicated this may be a problem, but we never had an issue (except for when I forget to open the appropriate ports on the VM’s local firewall, but that’s another story …). In more complex, multi-VM environments this may be something to consider.

Step 5 - Creating the Virtual Machine #

When creating a VM, you really just need to select:

• an image name,
• a name for the machine,
• the tier and size, and
• the region.

If you’re going for a build server, as we were, when selecting the image name be sure to pick one with VisualStudio installed … perhaps too much coffee that day. In the end we had success with both Windows Server 2012 and Windows 10 VM’s.

The easiest way to do this is to go to the Azure Marketplace:

Then just enter ‘Visual Studio’ in the filter at the top of the page. Note that you may only see entries for Visual Studio 2013 in the drop-down:

Just hit Enter in filter box and wait for the full list to display. From there just pick the edition and release that suits your needs:

Also, when you first create a VM be sure to allocate some decent resources, at least multi-core and plenty of RAM. If you don’t, your RDP sessions will drive you mad with the lag. It won’t take long to configure (well, not if you follow this advice!) and you can down-grade the machine once your done.

There is also an option to select an Availability Set. I believe this allows you to configure fail-over into other regions / data centres, but for our purposes we just selected the default of ’(None)’.

If you’re using the new Azure management portal, you will also see an option for the Deployment Model. This option is not available in the classic view and allows you to select:

• Classic (also known as Service Managed) and is the same as using the old portal, or
• Resource Managed this is a new option, allowing deployment of groups of resources as a single entity.

Here’s what it looks like in the new portal:

Follow the link to more detail on the differences between Resource and Service Managed deployment models. Note that they are not entirely compatible. From the article:

The Resource Manager deployment model provides a new way to deploy and manage the services that make up your application. This new model contains important differences from the classic deployment model, and the two models are not completely compatible with each other. To simplify the deployment and management of resources, Microsoft recommends that you use Resource Manager for new resources, and, if possible, re-deploy existing resources through Resource Manager.

Lastly, be sure to update the Endpoints configuration to add any port exceptions you may need (this was especially true for OctopusDeploy, which we put at the non-standard port 8080). This can be done when you create the VM as well as later on in the configuration settings.