Nokia Test Results: Is anybody really doing Scrum?

Last week, I challenged the Scrum community to take the Nokia Test: a Scrum litmus test to deterimine whether a team is agile and doing Scrum. How did the Scrum community fare?

Protecting Team Members from Management is clearly the biggest challenge faced by Scrum Teams. Only 51% of the respondants claimed that their team members were well protected from Project Managers or others disrupting work.

Prioritized product backlog and burn down charts are second and third from the bottom at 72% and 74% respectively.

Interestingly, all three points are issues relating to managment and the product owner (who is likely part of management), suggesting that management buy-in is a challenge. Either Scrum adoption is coming bottom-up or management is “talking the talk” but not “walking the walk”.

On the total test score, only 26% of the respondants claimed an 8 on the Nokia scale. 36% claimed 7 points (total of 62% for 7 or 8 points). At least 7% did not meet the minimum definition of “agile”

Without question, doing Scrum is difficult. There was some discussion on the Scrum development group as to whether the Nokia test is hard enough. But even with this simple test, 3/4 of the teams in question are not doing Scrum. Or are they?

I have three issues with the Nokia test:

1. “Software tested” is too specific to software development. I would ask the question “Is there a definition of ‘done‘ which is consistently applied’?
2. Continuous improvement is completely missing. Iterations don’t necessarily imply retrospectives. I would add a question “Does the team perform retrospectives every sprint and implement top priority improvements?”
3. Intent is also missing – Is the team actively trying to use Scrum to better realize the project? I think even if a team had to compromise on some points for tactical reasons (e.g. “the customer said no”), it should earn points for trying.