Useful as they are, Story Points are not uncontroversial nor is this problem of creating a “reference kilogram” for code trivial.
One problem we had on the WLC project was that we had two fundamentally different kinds of work to estimate: HTML Publishing and Software Development. Although we had attempted to have one reference Story Point for everyone, this proved to be elusive. After a few Sprints, I noticed that we had much less effort per publishing Story Point than per development Story Point.
A second issue is that some stories are intellectually complex while others are just time consuming. If Story Points represent complexity, then a simple but repetitive task should get a low number. This is unsatisfying, because such a story will put a higher load on the team, just because it takes so much time.
A third issues was that some developers just did not like Story Points and insisted on estimating in days.
After we had some experience with Story Points, we had an empircal measure of velocity, which we could also use to generate work days (“AT” for Arbeitstage) per Story Point. We discovered that the average of 3.5 AT / SP worked pretty well for us. Those 3.5 days included things like testing, code review, the Scrum-Master’s time, and everything that when into realizing that Story Point.
This allowed using the conversion when estimating stories. Having the stories somehow anchored in time was very reassuring to the developers who didn’t like Story Points. One they knew the could convert to time, they didn’t feel the need to.
This led to a different approach, let the team estimate hypethetical days, but on the Cohn scale. 1, 2, 3, 5….
I think people naturally estimate hypethetical days without being told to. This is the time it would take if they had nothing else to do, no email to read, no collegues asking for help, etc. Most Project Leaders add 50% to the developers estimate, and the difference between hypethetical and elapsed time is usually about this amount.
I have a certain fear talking about hypothetical days to customers. Just a matter of time before they only want to pay for the hypothetical time…
Still, this is not a bad reference kilogram. Something for the Scrum Team to fall back on, in case they don’t have any other brilliant ideas. At the same time, we call them Story Points, calculate velocity like if they were story points, and compare stories to each other for estimated purposes. So very quickly, they become real, live story points!
Update: There is no ultimate truth, and this proposal has its naysayers…