Previously, we’ve examined the new development practices that the Songbird team adopted to plan and track a release. Everyone on the team was very eager to put them to the test. Unfortunately, at the time, we were still in the middle of the 0.3 release cycle and new work could only be started once that release was completed. During the 0.3 release, everything was still treated as a bug, but in fact, many bugs were stories and tasks in disguise. We decided to apply some of the newly defined tracking principle to help us guide and finish the cycle, so we could start fresh with our next release as soon as possible.

Cuánto es?

The first step was to add cost to everything. We introduced a new cost field in Bugzilla and put a cost value on everything according to our new scale of 1, 2 and 3 points. With costing in place, we were in a position to compute how much points the team was able to complete in a typical work week. That total, normalized per work day became our team velocity.

Below is a chart representing our velocity over many — one week — iterations during the 0.3 release (code name Bowie). The blue line is the number of points the engineering team completed, averaged per work day. The red line tracks the cost of new things being introduced, normalized per work day. The green line tracks the net velocity.

It quickly became apparent that as the team took things off the pile, new work was being identified and added. We had to keep track and take this into account. We named this intake to globally represent new functionality, regressions and newly discovered bugs.

The net velocity gave us an indication on how well we were doing overall. When it gets in negative territory, we are losing ground.

You can see below 3 events that had clear impact on intake, namely scope creep (some features were not well defined upfront), and bug intake due to public feedback from a blessed build and a release candidate.

Also noticeable is a week when the team was not as productive as usual. With that information in hand, we were able to have open conversations about the state of progress and try to determine the cause for it. Sometimes, a low velocity is simply because work gets accumulated in a week and does not get checked in until the next. We call this carry over. Other time, there are some inevitable distractions such as a move, interviews, equipment failure, etc. In other cases, the team is just having a bad week.

With this in place, we were able to understand the rate at which the team was completing work. We created a burn down chart that tracked the total points of known work left. Using the team velocity, we could forecast an expected release date with a simple best fit projection. When the line crosses the X-axis, you are done and can ship.

If you compare the two charts, you can see how much influence intake had on the release. This became a key component to take into account in our planning. By budgeting points towards intake, it allowed us to reserve some engineering capacity to take change into account upfront and have a more realistic schedule.

Where does intake come from?

By formally tracking our intake, we were able to better characterize the nature of change. Most of our intake comes from change introduced when features start to materialize in the product and can be tested. This is a desirable effect of adopting an Agile practice. Other contributors are defects being reported by existing users, which is a benefit of early releases. Less desirable intake come from regressions or new tasks that resulted from bad assumptions or misunderstandings. Those can usually be mitigated by increasing unit tests coverage and more upfront detailed planning.

Full Agile cycle

Let’s take a look at another release cycle. Dokken was our second release in which we used the new process from inception. The charts below represents velocity and burn down respectively. Note that the scale on the velocity has increased. There is more dynamic range. Because the release started from day 0, we noticed a ramp up in the velocity. We learnt not to be necessarly alarmed by this. In a new release cycle, the team needs some time to “prime the pump” of development. As the release progressed and we got better visibility of the team progress, we decided that we should defer feature work that was identified as nice-to-have. This is another thing we did during planning. We prioritized work in must have for the release, hope to have for the release and nice to have – mainly cosmetic changes, low risk bugs – buckets. This gave us a pre-negotiated way to easily shift features as the release progressed.

By actively tracking and managing the intake, we were able to steer the release and deliver within weeks of the projected date. Not great, but better than our previous releases. Dokken was a particular difficult release as we undertook a lot of device support work, which can lead to nasty device compatibility problems.

Embracing change

The next release, named Eno presented some interresting characteristics. The intake was relatively high thru the release but the team was also maintaining a higher velocity. This is a good example of the team achieving a good level of agility. Change is being introduced throughout the release and the team is well prepared to tackle it. Also notice that the spike in intake due to feedback from release candidate has a corresponding level of increase in team velocity. This is due to the shortening of feedback loop. With proper ownership, the developers are able to respond in real time to issues being discovered.

Notice how the burn down trend is converging almost linearly. This means that the team is achieving a sustainable pace, leading to more predictable ship date.

Achieving high velocity

Our latest release, Fugazi is another example of a successful cycle. This was the shortest release cycle the team ever undertook, with only 4 weeks of planned development work and 3 weeks of QA. In order to maintain our original release date, we had to defer some lower priority work in iteration 3. Despite the shorter release period, the team actually completed more points per iteration than any other release. We maintained an unprecedented high velocity, which in turn allowed for a high level on intake to be absorbed, so all the planned must have featured could be kept in the release and still hit our original target date.

DIY tools

In the last part of the series, we’ll cover the tools we created to make the tracking easier and how they are being used on a daily basis to help us steer the release. Stay tuned.