Agile Assessment and Continuous Improvement
In a situation with 35 Software Development Scrum Teams, 260 team members from Engineering, QA, Product Management, and 1 Agile Coach, we were tasked with conducting 'audits' of the agile process.
At first we did our best to spread ourselves across all the teams and ceremonies (Grooming, Planning, Stand-ups, Retrospectives and Demos) but found we were not covering enough ground. Further, we wanted to know from the team members if they had absorbed the training we were offering.
So we came up with the idea of using an online survey tool to quickly gather as much data as we could. We needed to keep it short enough that it would get answered but cover enough topics to be meaningful. We settled between 15-20 questions, mostly multiple choice or true false with comments also allowed. The questions were grouped into 4 dimensions or categories so they could be rolled up into a neat graphic. Based on their answers we scored them on a unit scale.
Spider Web Chart of Survey Results
Because of the work environment specific to this business, we had to make the survey anonymous or nobody would answer it honestly. And we needed to be able to report the results to the role and team level but not both at the same time or it would give identities away. Even though we guaranteed anonymity some people refused to participate but we achieved more than a statistically valid sample.
To validate the results, we scheduled in-person reviews to confirm what team members had reported out. Based on our observations we awarded ribbons (graphics) like Olympic medals (gold, silver, bronze).
Olympic style medals
Based on our observations we also provided a feedback section for WWW (What Works Well) for positive feedback and WCBI (What Could Be Improved) for continuous improvement. We could then provide an overall team score, a score by dimension, and roll these up to department and overall levels to compare from assessment to assessment. We timed these to occur soon after each major product release. As a result we completed three per year.
This department improved dramatically in the two years I worked with them. Not only did their Agile maturity improve but the predictability of releases and product quality did along with it.