Choosing between Visual Studio Team Services and On-Premises TFS

A few days ago I bit the bullet and upgraded our on premises Team Foundation Server from TFS 2015 to TFS 2015 Update 4. As I sat there, after hours, watching the update spinner with the IT department on speed dial, it made me reflect on whether putting in on premises TFS was the best idea. If we had gone for Visual Studio Online (now Visual Studio Team Services) I wouldn’t have to do upgrades – ever.

Flashback 2 years: we had a mishmash of ALM tools that didn’t talk to each other, a git repository that worked at a snail’s pace and a deployment process that involved manually editing csproj files but never checking them in. We had to change. We discussed online vs on premises and after some pen chewing, fingernail biting and some proper pondering we went for on premises. Did we do the right thing? Here is why we took the decision to go on premises and on reflection here’s if I think we did the right thing.

1. Code stored offsite

With Visual Studio Team Services (VSTS) your code is clearly stored offsite on someone else’s machine – your crown jewels are stored under someone else’s bed. This shouldn’t be a problem. Microsoft bunkers are secure, more secure than what I could come up with. Surprisingly this was an issue and various stakeholders where very much happier with code being stored by us. I’ve had very experienced IT staff visit us since then and they were also happy to see we had complete control over our code. So on-premises won out on that score even if it was perception over reality.

Was I right?

Yes and no. Our source code is probably better off in Microsoft’s super secure bunkers but if the decisions makers want the code with us then that’s fine. I didn’t push for it to be stored offsite and I wouldn’t push for it now.

2. High levels of Customisation

This wasn’t a Greenfield project and we had pieces that needed to be integrated into TFS.  With on premises we could break it open and code against it. At the time with Team Services you got what you were given. Our two pain points were

  1. MSpec tests. We could only get these under continuous integration with a lot of on-premises custom tinkering
  2. Labelling build versions. I put a piece in to code against the TFS object model to get the build versions automatically populating and integrating this into continuous build. Again custom on premise tinkering but well worth it.

Was I right?

Yes and No – we ultimately retired MSpec but for a year about a third of our test coverage was using this so needed to be included. Labelling build versions is still really important to us and works well. But VSTS has come on a lot and would now do most of this if not all.

3. Infrastructure

With on-premises you are responsible for your own servers – you are masters of you own bits of tin. You want more infrastructure then you will need to find it. I didn’t think this would be a problem

Was I right?

No. Infrastructure became a big issue. Disk space ran out and I had to keep scavenging around for more. Our builds proliferated and our build server became very under powered. It took a long time to get another – we just didn’t have the rack space. Don’t underestimate the infrastructure – if the TFS implementation is successful them people will want more and more of it.

4. Quality of Internet access

2 years ago our internet was powered by a very tired hamster on a wheel. It wasn’t good – our existing online tools were causing a headache. No-one had much appetite for further reliance on our poor internet connectivity. On-premises seemed the better option.

Was I right?

Yes but not for long. We got a better Internet connection after 6 months so we could have ultimately used Team Services. But I think an initial 6 months of a poor user experience would have led to a rejection of the implementation. Not everyone in the world has great Internet so Team Services is always going to battle against that.

5. Upgrading

With Team Services your upgrade is done for you. You are always on the latest version. With on premises you do your own upgrades when you have the required levels of emotional resilience to embark on it. I didn’t think this would be a problem.

Was I right?

No. Upgrading is a pain. I’ve done it twice and it worked well both times but there is a lot of thought that needs to go into rollback plans and minimising outages. Now I’ve gone from TFS 2013 to 2015 R4 the next is 2017 and that’s going to take an upgrade to SQL Server as well. It will be much more difficult next time.

6. Amending Bug  and Work Item definitions

At the time Team Services didn’t offer customisation of the process templates i.e. you couldn’t put in extra field, dropdowns and process flows into your project tracking and defect management. This was a deal breaker for us – we had absolute requirements around this. Over two years we have done a lot of customisation of the templates and we really benefit. Once again customisation is king.

Was I right?

Yes and No. Team Services has added some customisation but it is still work in progress. We couldn’t and still wouldn’t wait for a feature that may or may not happen. We needed that level of customisation – it wasn’t just a nice to have. That said this feature is definitely on the road map for VSTS so any concerns I have will continue to evaporate.

7. Availability

Team Services promises 99.9% availability.  On-premises is going to struggle to complete with this particularly when factoring in downtime for upgrades. I didn’t think this would be an issue for on-premises.

Was I right?

Yes – it wasn’t an issue. Over 2 years we didn’t get anywhere near 99.9% for availability but it didn’t cause problems. Outages were planned and developers could keep developing, testers keep testing and project managers could keep doing whatever it is that project managers do. It was fine.

8. Other Issues

There are a few other issues that didn’t come up at the time but I would probably consider if I was to take the decision today

Costing

The costing model for on premises and VSTS is comparable currently.  You can get monthly users of on premise TFS with an online subscription. From the costing page

Each paid user in your Team Services account also gets a TFS client access license (CAL). This lets you buy monthly access to TFS for your team.

This really helps out when you are ramping up testing and need a whole bunch of new users. I’m just a little wary if this will always be the case – monthly users are baked into VSTS but I could imagine them being quietly dropped from on-premises. Paranoid maybe (maybe not).

Support for extensions

I was interested in a decent wiki for TFS but the extension is just for VSTS. The extension ecosystem seems better supported for VSTS. I suspect it always will be.

The future

I don’t need to climb inside the mind of the Microsoft Board to understand that there is a big focus on cloud offerings. VSTS fits nicely within this. TFS on-premises feels a little bit like a relic now. Perhaps not but I would be wary about ongoing support for the on premise version for small shops. I wouldn’t want to be chopping and changing my ALM solution every couple of years so I would have an eye on what the future may hold.

So … on-premise or online?

From my own experience I would say that for

Greenfield project with flexible ALM requirements
Pick VSTS – it’s easier to get going

Established project with requirements for high levels of customisation
Consider on premises but do review VSTS – it might do everything you need. It probably does by now.

For large shops with big IT departments and deep pockets
Go for on premises if it suits and you feel better with your code in house. Infrastructure and upgrading will be no problem. You’ll eat it up

For small shops with a small/non-existent IT department
Be realistic about your IT capacity and willingness to upgrade. Even if stakeholders are screaming for on-premises be certain about your own capacities.

And if I was to implement a Microsoft ALM today?
I would go for Visual Studio Team Services though on-premise was the right decision at the time.

As ever this is just my opinions. Feel free to disagree and if you post a comment telling me why I’m terribly wrong – that would be great.

Useful Links

Visual Studio Online home page
https://www.visualstudio.com/vso/

Customising process templates
https://www.visualstudio.com/en-us/docs/work/reference/process-templates/customize-process

Team Foundation Server object model – which I used to do a fair bit of customisation to our on-premises installation
https://www.nuget.org/packages/Microsoft.TeamFoundationServer.ExtendedClient/14.89.0

 

 

 

Conditional Deployment with WIX

Often when deploying an application you need guaranteed access to a certain area of the file system. This can change depending on the OS. The challenge is to produce an msi that can detect that and supply the correct values.

Summary of WIX

Wix is an xml based tool used to generate msi files. To use

  1. Create your WIX file – MyMsi.wxs
  2. Run the tool candle.exe to generate the wixobj file
  3. Run the light.exe tool to generate the msi

WIX has the advantages of being

  1. Free
  2. Complete – does everything that I have asked of it
  3. Fairly well documented
  4. Command line runnable so it can be easily integrated into continuous builds

It’s not all sweetness and light though. I find handcrafting the XML hard going and a bit obscure. Projects such as WixSharp and WixEdit can take the sting out of it. However when going past the basics it’s often easier to start to handcraft the XML.

The Problem

Often when deploying an application you want an area with guaranteed access to a certain area of the file system. The ProgramData folder fits the bill nicely. This folder is used for application data that is not user specific so it’s a good candidate as an area for downloads and so forth from an installed application. Unfortunately many of us need to support older OS such as XP which don’t have the ProgramData folder. In this instance a good candidate could be

We install our application using an msi generated by WIX.  What I really don’t want to do is maintain more than one wix source file (.wxs). Really, in my life I don’t want to maintain more than one of anything (DVD player, car, daughter etc…) and I really don’t want to maintain more than one wxs file. So there needs to be a way of

  1. Detecting the OS version in WIX
  2. Using that information to change a property within the wxs file

WIX is nothing if not complete so there is such a way.

Detecting OS version

The operating system is exposed by the Window Installer Property – VersionNT. So

  1. Windows 2000 is 500
  2. Windows XP is 501
  3. Windows Vista 600

And so forth as detailed here.

So to detect Windows XP you would need to include a condition element such as

Of course this blocks the install completely rather than changes a property so it’s only half the story.

Changing a Property Setting

To change a property conditionally on OS you need to reach for the SetPropertyElement. There are a few wrinkles with this

Action attribute

My first pass at this was

So you have a property. The two set property elements target the property element because of its common id. However if you run this then you get this error

So you need to deploy the Action attribute defined in the WIX documentation as

By default the action is “Set” + Id attribute’s value. This optional attribute can override the action name in the case where multiple SetProperty elements target the same Id (probably with mutually exclusive conditions).

So the wix elements become

Which gets over that issue

Before and After Attributes

Unfortunately making the installer with the revised XML generates this error

Which has the merit of being straightforward. It needs to know when in the install sequence the property should be set. The windows installer has a sequence of standard actions detailed here for example

  • InstallFiles
  • InstallExecute
  • InstallFinalise

And many others. (To me) it makes sense to set the property just before the files installs so …

And the WIX XML now becomes

Equally the After attribute can be used to set the properties after the InstallFiles install action.

SequenceType Attributes

Now when we make the WIX file we now get a more cryptic error

It turns out that there are 2 install sequences

  1. UI
  2. Execute

Looking again at this standard action table you can see there are two sections

  1. Wizard Dialog Stage (UI)
  2. Install Execution Stage (Execute)

The InstallFiles installer action only exists on the execute sequence and the WIX compiler needs to know that. So you now reach for the Sequence attribute and the WIX XML becomes

It now compiles and creates an msi that behaves differently on XP and later versions of windows. Job done!!

Uses

Once you have the flexible property it can be used to redirect output to a different location or used as a source for a config file transform such as

Which can be very useful.

So well done if you read this far. All the documentation exists on the internet but I found it puzzling so I hope this might help in a general demystification effort.

Useful links

About the ProgramData folder
http://stackoverflow.com/questions/9518890/what-is-the-significance-of-the-programdata-folder-in-windows

WIX documentation
http://wixtoolset.org/documentation/

Windows Installer Operating System Property Values
https://msdn.microsoft.com/en-us/library/windows/desktop/aa370905(v=vs.85).aspx

Windows Installer Standard Actions
http://www.advancedinstaller.com/user-guide/standard-actions.html

WixSharp
https://wixsharp.codeplex.com/
A C# API for WIX. Good for generating a base XML file to start with. Not complete though so will need to fall back to direct XML editing for more advanced scernarios

WixEdit
http://wixedit.sourceforge.net/
A simple UI for the XML file. Useful.

Deleting from Team Foundation Server

It’s non-obvious how to delete items from Team Foundation Server. Things can be removed from the web interface or from team foundation explorer but they can lurk around still and bother you. It’s surprising difficult to delete. These are the ways that I’ve cleaned up after myself in Team Foundation Server 2013 and 2015.

Workitems – Bugs and Backlog Items

The straightforward way is just to set the status to removed.

Removing work item from TFS

But depending on how you construct your queries they’ve a habit of shining through again. To me, there is a difference between “a bug has been created and I had a bit of a think about it and I think it’s not relevant” to “I’ve just completely created the wrong thing and I want to get rid of it”. So if you never want to see the workitem again then it’s down to the command line

e.g.

and it’s gone. Careful now – once it has gone then it’s gone forever.

Builds

On the face of it, it is straightforward to delete builds. Just go to team explorer find the build, right click and press delete.

Deleting build from TFS web UI

The problem with this is that the build still exists in some sense and is included when calculating build version number.

Example

Build definition is

Visible in the advance tab of the build definition

Build Label

The build includes the build revision token (Rev:.r) which increments by 1 for every build

If we want to delete the send build then fine – go to the web UI and press delete. If the build is requeued we will get

The build label jumps to 2 missing out 1. The deleted build still exists and counts towards incrementing the build revision token. This might not be an issue. However if you use the build label to drive out the version of the application as we do then this can become a significant issue. Suddenly a version has been missed which could cause various shades of confusion, upset and terror.

To avoid the TFS related terror scenario then the build needs to be deleted AND destroyed. Once again jumping to the command line

To delete the build (i.e. the same as through team explorer)

Now destroy the build so it is completely gone

Example

Now destroy the build so it is completely gone

Now when the build is deleted the build versions work as expected and the revision number is reused i.e.

Work Item History

Work Item History

This is trickier still, the situation where you put some errant information in a Bug or PBI and want to remove it. It could be innocuous as posting in the wrong bug post (which I have done many times). It could be as eye wateringly bad as three paragraphs of invective targeted at a colleague. Sadly the cmd line doesn’t expose a way to delete work item history.

The only way I know is to code against the TFS object model and delete from there. The object model isn’t that well documented but it’s not bad to code against. One for a future post perhaps,

A note on the command line

The command tools used here (TFSBuild and witadmin) along with other helpful tools such as tf.exe, are found in program files at

I would recommend putting this filepath on your PATH environmental variable if you find yourself doing much TFS admin. That way you won’t have to continually navigate to the program files folder. You’ll soon get sick if doing that if you have do delete something from TFS more than a few times.

Useful Links

A full list of build tokens for XAML based builds
https://www.visualstudio.com/en-us/docs/build/define/general

Details of programming against TFS object model
https://msdn.microsoft.com/en-us/library/bb130146(v=vs.120).aspx

Amending PATH variable on a windows machine
http://www.howtogeek.com/118594/how-to-edit-your-system-path-for-easy-command-line-access/