Sounds completely sensible.
Why would you do anything else? That would just incur unnecessary risk surely?
No, we would definitely not do that.
Yes you would, and from what I see, in surprisingly large numbers too.
Why Amber is Different
What allows this to happen? Red is Stop. Green is Go. Simple. Everyone gets this concept, and generally speaking most obey these rules.
Amber – that’s where actions can become a bit blurred. Depending on where you are, Amber is treated differently.
Take traffic signals as an example. On the roads of Asia Amber is definitely a shade of green. There are many statistics that show that this is not always a great approach. At the other end of the scale my limited travel experiences in Scandinavia have clearly indicated that Amber in those countries is more of a red hue. Strangers will take the time to loudly tut at you should you venture onto the road as a pedestrian when the green walk light is not clearly on.
Why this digression into Stop / Go semantics? Well for their perpetual sins Microsoft made it possible to end up with an Amber build result state in TFS. Why would you do this? What does Amber signify? Something didn’t quite work. What didn’t work? Presumably it must have been important otherwise it wouldn’t be in there. Right? So what the hell am you supposed to do with an Amber build?
Personally I want to know if a build either worked or it didn’t. That is the Red / Green bit. If the build goes Red we stop what we are doing, use the information in the build logs and identify the cause of the failure. Next we kick off a bug cycle to document, fix and test any changes necessary to get that build back to a healthy Green state.
A digression into the importance of Logs and Build Speed
Key to making this step work is getting enough diagnostic detail out of those logs. Normally I don’t need this, in fact I’d like those logs to be as short as possible. When the spanner falls into the machinery though I want to dial-up the logging to OCD levels of detail. To be successful with this approach requires that those builds are on the snappy side. As a guide anything over 30 minutes and I am really not interested. Again, if you are the unhappy worker on a project with long running build times I would suggest that you have more fundamental problems to solve in your build process than I will be addressing here. There are always exceptions to this. Those releasing safety critical or banking software systems then please take as much time with the build and especially testing phases of your releases as you need to, ensuring that you cover literally everything.
Amber is Indeterminate
In many programming languages we are provided with the ability to have NULL values. Tony Hoare, the computer scientist who implemented null references in ALGOL calls it his “billion-dollar mistake.” Amber build states are a similar GOTCHA. Some indeterminate thing that you now have to deal with. Why do this to your builds?. Seasoned developers armed with good intent and a reasonable smattering of modern tooling can avoid the pitfalls associated with not handling nullable types correctly. But this takes effort and is not always performed successfully. An Amber build is the same deal for Project Managers and Team Leads. At best a wasteful discussion will ensue about whether we dare to ship, accepting risks that we don’t yet have solutions for and a myriad of other distractions. This encourages us to not face up to the fact that we have a buggy build, not wise.
Why would you be tempted down the Amber Release path?
There must be situations where it’s surely ok to release on Amber? Some pressing business need to get a feature into a build or simply running out of resource on a sprint perhaps?
But look closely at your logs when you get an Amber build. There are likely warnings and errors in those logs that do not trip a build abort. These are like the bugs fossilized in that piece of Amber shown at the start of this post. Buried and baked in artifacts that you are now considering unleashing on your customers.
Green Builds for the Win
My belief is that it is simply not wise to release off of anything other that a nice shiny green build. What that build contains I am much more pragmatic about.
Obviously the build must have compiled and packaged up its outputs successfully. If the build fails to do this you are in the clear. There is nothing to release. If your release process does not have a build mechanism that automates compilation from your source control system then you should probably stop reading right here and go fix that issue first.
From here on out we are just adding risk reduction automation to what we are releasing. Me personally, I can’t get enough of this stuff. Unit tests, static analysis, IL weaving, security scanning, etc are all good examples of what you could add to your build scripts to get a better handle on the quality of the product you are considering releasing. The economies of automating repeating tasks should be obvious. We’ve been doing this since the Industrial Revolution with some fairly proven results. These automated tasks are giving us greater confidence in the quality of what we are releasing, but only if the build is Green. In an Amber state their value drops measurably and can even prove to be a distraction to adding business value to your product.
The Bottom Line
What it comes down to is I really don’t want to ask someone, anyone in fact, whether a release is fit to go out the door. Nor should anyone else on the team, including project management. The team’s build status should accurately capture this No / NoGo decision point and your source control system will unambiguously document what went into that release.
Like the 1950’s washing machine advert states ‘Simple, Thorough, Speedy and Safe‘.