Process appraisals: from interviews to Socratic Circles, 10 years of results

Most process improvement initiatives start by a GAP assessment against a pre-selected process improvement model. During this GAP assessment, an expert is called upon to evaluate the state of the practice of the organization against the model. Usually, the expert selects an interview process with selected individuals from the organization. While still useful and applicable, this method
weakness lies in its incapacity to generalize and the validity of its results lies in the representativeness of the interviewed subjects. This method is also time consuming in large organizations, since there is a practical upper limit to the number of concurrent interviews an expert can handle.

Since 2003, we have been shifting the focus from the interview, which puts the interviewer at the center, to a group activity which resembles a Socratic Circle. In such circles, a set of questions is put
forth to a group and they are encouraged to discuss it until they reach a consensual answer. This method scales reasonably, and allows a medium sized organization to include every resource from
the beginning of an improvement initiative. Through this method we have managed to conduct process GAP appraisals in relative short periods of time (1-3 work days), with large groups (up to 30
concurrent participants) and in multi-model process environments (ISO 9001:2008, CMMI-DEV, ISO/IEC 20000-1:2011 and Enterprise SPICE) in traditional and agile settings.

This paper shares our results and the evolution of our assessment approach. We have been improving this approach for the last ten years after using it in several small to medium size organizations. In addition to the cost savings in effort and time during the GAP appraisal, we see benefits in terms of participation and reduced change resistance during the improvements design and deploy.