Last November, I started a quick poll on Story Size, Team Size, and Sprint Success. I wanted to explore that variables of team size, sprint duration, average and maximum story size, and whether there is any correlation between between these parameters and team success.
I received 81 responses (presumably from 81 different people and hopefully from 81 different teams, but I have no way of checking this). 78 claimed to be doing Scrum, one claimed XP, and two did not reveal their preference. This article presents a first look at the results.
Scrum recommends an Implementation Team size of 7+/-2, so 5 to 9. Based on the poll, 63% of the respondents are in the range of 5 to 10 (should have sized the response blocks differently!). 28% have 1 to 4 people and 9% have 11 of more.
These results were something of a surprise to me. More than half (63%) of all respondents indicated they did two week sprints. Only 1/3 that number reported 3 week sprints. In May 2008, I ran a quick poll on sprint length, and the answer came up even: almost exactly 1/3rd each were doing 2 and 3 week sprints, another third where spread around 1 week, 4 week, 1 month and variable.
Obviously the poll tells us that something has changed, but not why this change has happened. And in all honesty, I am not sure this represents a real change. The previous poll had only 34 respondents and even 81 is not a huge number.
On the other hand, the influence of XP on the Scrum community has grown, there has been a lot of discussion on “Scrum-But” since my original poll on the Nokia Test, and queuing theory tells us that shorter sprints with more stories per sprint should be more predicatable.
Number of Stories per Sprint
In my trainings, I recommend teams to take on at least 10 stories per sprint. This means that the average story size is around 10% of the team’s capacity. What do teams actually do? 74% of all teams take on between 4 and 13 stories per sprint, which puts the average around 7 or 8 and the average story size around 12% to 15% of the team’s capacity.
Largest Single Story
During my first Scrum Project as ScrumMaster, I made the observation that if my team took on a story that was 40% of their total capacity for the sprint, their chances of not completing the story were, uh, “excellent.” In the mean time, I have conversations with people who suggest 25% is better maximum. I was surprised to see that there are teams out there who will take on a story that represent 50% to 100% of the team’s capacity for the sprint!
According to this data, 54% or all teams insist on stories <= 20% of their capacity, and 79% insist or 35% or less.
Success is a tricky thing to measure. For this poll, I define success to mean, ‘the team delivers that which it commits to.’ This does not say anything about the productivity of the team. I once saw a 12 person team double its performance in story points after being split into two 6-person teams. But the team had been delivering what it promised, so by this definition it was a successful team.
I choose this measure for success for two reasons. 1) Delivering on commitments is a fundamental skill of a Scrum team. It builds trust between Implementation Team and Product Owner. 2) It is relatively easy (uh, possible) to measure in this context (“the light’s better“).
Looking at this graph, you might get the impression that a 70 to 90% success rate is OK. I don’t think so. The goal is to be a reliable partner. Yes, a team needs the safety to fail once in a while, especially when they just start out with Scrum, and if you focus too much on the goal of certainty, then other things, like performance and creativity, may suffer.
Scrum is Hard to do Well
So I would interpret this data differently. There are teams that are doing well, “successful” and those that need improvement:
…but I think we’re getting better!
It looks like 38% are doing pretty good Scrum, delivering on their commitments, and 62% could be doing better. This compares favorably with the 76% of all respondents who scored 7 or fewer points in the 2008 Poll on the minimal Nokia Test.
The interesting question is what are the signs of a good (or bad) Scrum team? I have been playing with Pivot Sheets and will post some analysis of the relationships between the various factors. (And if anyone is good with statistics and/or has access to better tools than Excel for this purpose, your help would be most welcome!).
Update 6.3.11: Here is the original Poll and the actual data in spreadsheet form. (Don’t use google’s graphical summary of the results. For reasons I do not understand it is not correct).
|cookielawinfo-checkbox-advertisement||1 year||Set by the GDPR Cookie Consent plugin, this cookie is used to record the user consent for the cookies in the "Advertisement" category .|
|cookielawinfo-checkbox-analytics||11 months||This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".|
|cookielawinfo-checkbox-functional||11 months||The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".|
|cookielawinfo-checkbox-necessary||11 months||This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".|
|cookielawinfo-checkbox-others||11 months||This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.|
|cookielawinfo-checkbox-performance||11 months||This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".|
|mailchimp_landing_site||1 month||The cookie is set by MailChimp to record which page the user first visited.|
|CONSENT||2 years||YouTube sets this cookie via embedded youtube-videos and registers anonymous statistical data.|
|_ga||2 years||The _ga cookie, installed by Google Analytics, calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognize unique visitors.|
|_gat_gtag_UA_42152348_1||1 minute||Set by Google to distinguish users.|
|_gcl_au||3 months||Provided by Google Tag Manager to experiment advertisement efficiency of websites using their services.|
|_gid||1 day||Installed by Google Analytics, _gid cookie stores information on how visitors use a website, while also creating an analytics report of the website's performance. Some of the data that are collected include the number of visitors, their source, and the pages they visit anonymously.|
|NID||6 months||NID cookie, set by Google, is used for advertising purposes; to limit the number of times the user sees an ad, to mute unwanted ads, and to measure the effectiveness of ads.|
|test_cookie||15 minutes||The test_cookie is set by doubleclick.net and is used to determine if the user's browser supports cookies.|
|VISITOR_INFO1_LIVE||5 months 27 days||A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.|
|YSC||session||YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.|
|yt-remote-connected-devices||never||YouTube sets this cookie to store the video preferences of the user using embedded YouTube video.|
|yt-remote-device-id||never||YouTube sets this cookie to store the video preferences of the user using embedded YouTube video.|
|COMPASS||1 hour||No description|
|cookies.js||session||No description available.|
|S||1 hour||No description available.|
I don't think that between 70 and 90% is bad. Yes one goal is to be predictable.
I had coached a team that made their sprints all the time. 3 other teams in the same organisation did not. Non of them used Burndown charts at that time. When we introduced burndown charts, we had some discussion if it was wurth using the burndown chart for the "good" team.
It was decided they would too.
The burndown chart showed that there was a big problem with the so called good team.
They had almost nothing done, untill the very last day.
This team was calling things finished, that weer in reality not finished (resulting in bad quality.)
I had another team that was always above 100%. This team always under committed. They did this because they received large bonusses for arriving at 100%.
I am totally with you that this is a very superficial definition of success.
I believe measurements of team performance can useful for helping the team understand its performance. They are worthless for 'driving' performance, setting targets, or comparing teams. The measurements are easily manipulated and may ignore important aspects, such as quality or exertion, as you correctly point out.
I don't think a team should regularly over-commit. It means the team does not understand its capacity or is being put/is putting itself under too much pressure.
So while I would never even suggest punishing a team for not meeting its commitment, as a coach, I would encourage a team to try to hit its commitment +/- 10% most of the time.