Donald rants on Kirkpatrick
Donald Clark has just posted another of his interesting rants on Kirkpatrick, titled amusingly Donald talks bollocks.
Here is my comment on it:
We said pretty much the same thing from some research and presentations a couple of years ago. The trouble is though that the KP levels and thinking has become lingua franca for the training community, and whilst individuals may question it, collectively they just accept it and continue to promote it.
I also believe this has been further magnified by the complicity of ASTD with Kirkpatrick and Philips in the US. Even ROI = level 5 now seems to be taken for granted even though ROI is just one format of presenting financial results. The biggest crime of many may be of perpetuating a model where the training industry thinks that "evaluation" is something that happens after the event rather than before and during. The effect of training's adherence to KP-thinking and systems for L1,2,3,4 evaluation is that it actually obscures rather than highlights the real issues of impact.
I would much rather trainers focus on understanding clearly what impact you are trying to achieve, whether it is a "learning" issue at all, and if so, in what way, how to design and construct solutions that make that impact, as well as delivering them to ensure they do make that impact. If you do this simply and clearly, understanding how to measure whether you are successful or not, becomes much more obvious.
David
P.S.One other quick though on L1. Why is the training industry the only one in the world that thinks it needs a detailed customer service questionnaire filled out by every customer on every transaction? I do see the need for business functions to understand their customer satisfaction, and encourage feedback, particularly on critical or challenging things. I do not understand why training thinks it is valuable to force it for every transaction. Ever hear of "sampling" guys!
2 comments:
I wholeheartedly agree with Donald's comments, and your take on it. Even if KP may have had some value if applied fully, how many organisations go beyond Level 1?
Barry, this is where I disagree with the ASTD stats which I think show an artificically rosy picture of how many are going beyond L1 systemically. E.g. the 2005 numbers showed 53% doing L2 assessment for all programmes, and 22% doing L3 for all programmes.
In our research, none of the core corporate participants did this. Yes they would do it for certain programmes, but not systemically. Most of their overall metrics were L1 or activity measures, what I call L0.
Although this has changed it more recent years with products like Metrics that Matter from Knowledge Advisors enabling much more extensive post-event follow-up.
But this is not a panacea, I think can actually cause more bad behaviour. e.g. focusing on evaluation after the event, not before and during. It can also remove the need to clearly understand the impact we are trying to achieve, substituting it for a generic post-course questionnaire. That is a huge issue.
Post a Comment