Agile development methods are well suited to plan and execute near term release cycle. For instance, the tools we developed and processes we’ve adopted help us plan and steer a release to completion with a good level of accuracy and repeatability. However, there are instances when the time horizon needs to be further out than the current cycle. The need to create a budget, synchronize a roadmap with a partner or determine future hiring needs, make it necessary to have an effective mechanism for long term planning.

Fortunately, the metrics gathered during each Agile release cycle can be very helpful for that purpose. Once we gain a good understanding on what is being worked on, for how long and by how many people, we should be able to extrapolate this to forecast future releases.

Let’s take a look at what activities take place during a typical release cycle:

1) Plan release
2) Write code
3) Test
4) Fix bugs

Then repeat ad nauseam.

In that context, the programming tasks can be categorized as follow:

1. Planned work

This is the body of work identified during the planning stage. This is the raison d’être of the release. For the most part, this covers new features or less tangible things such as performance improvements. This is what we’ll want to talk about when the product gets released. It’s an easy planning trap to think that this is the only work required.

2. Unplanned work

As the name implies, this is work that was unforeseen at the beginning of the release but is required to be completed before the product can ship. It can be further refined as follow:

a) Change in requirements
This should not be unexpected. In fact, any Agile methodology assumes that there will be changes down the road. This is not a problem per se as long as there is a mechanism to trade features, extend duration or increase resources.

b) Omissions in planning
Because the planning period is relatively short, it is assumed that not everything is fully specified or researched upfront. As development proceeds, new pieces are discovered and introduced into the plan.

3. Bugs

Defects can be considered a side effect of software development. They can affect the product in different ways:

a) Regression
These are defects introduced when working on new code. They usually impact unrelated functionality that used to work before.

b) Defect in new feature
These are problems in a newly coded feature. The feature does not work quite as expected.

c) Existing bug
These are bugs present in the previous version of the software. They are either known or newly discovered during the course of the release and prioritized to be fixed now because they affect existing users.

Learning from the past to project in the future

A believable long term plan needs to layout new features on a timeframe against a set of resources. Because there is only a limited amount of time and people allocated to thinking through the issues upfront, the initial estimates are always very rough. It is then very difficult to predict how much time certain things will take or how many people will be required. To increase accuracy, you’d have to invest more time and resources, which is not practical, mostly because these are the very same resources that are critical to deliver against the current release plan.

You need a model to help forecast based on imperfect data. You could decide to simply pad all the current estimates, but you’d still need to figure out a factor that’s sufficient.

A better model is to find a way to anticipate the additional work that will be generated by the introduction of a brand new feature. With that model, we can layout a plan that should allow sufficient room in every release to accommodate for all the work, planned, unplanned and bugs.

Let’s take a look at historical data from previous Songbird releases:

The first row in the table contains the budgeted points for each release. That number represents the sum of all planned work as determined at the end of the planning phase for that release. The release schedule was set based on that number and the historical team velocity.

The second row represents the total actual points completed during the release, including features and bugs. This represents that amount of effort to get the release out.

The third row represents the actual completed feature points only. Next row computes ratio of planned vs. actual work.

It’s very tempting to compute a velocity based on total completed points and use it as a predictor for how much work the team can accomplish. While it’s true that it is a measure of total effort, it can’t be used against planned work only.

What we want to determine is that given 1 point of planned feature work, how much unplanned work and bugs will be generated and thus how long will it take to complete. Then normalize this for team size and you get a factor that can be used for long term planning.

To get there, I’ve computed a velocity for each release based on planned work divided by actual release duration (in days) and number of engineers. I then use the median value across release as a factor to turn estimated planned points into calendar days. With that in hand, I created a simple model where scope, duration and resources are linked. Set two values and the third one gets computed automatically. This model becomes very handy to run through what-if scenario during roadmap sessions. It provides answers to business questions such as: “How long would it take to do feature X?”, “Can you meet deadline Y?”, “How many more engineers would it take to do Z in Y time frame?”.

Hope you enjoyed learning more about our approach to the difficult task of long term planning. I’d be interested to hear about how you’ve approached the problem, so feel free to share your story.

Lastly, let’s remember the words of the famous Danish physicist Niels Bohr:

“Prediction is very difficult, especially if it’s about the future.”