Rethinking Kirkpatrick Level 1 (Reaction)

Don Kirpatrick’s 4 levels of evaluation has been in place since 1954, when he originally did his dissertation. I don’t have any information on when the first smiley sheet or survey was put in place but I can say that even today many companies elude to smiley sheets or surveys to measure a learners’ reaction to a learning intervention (Kirkpatrick Level 1 Evaluation). After painfully chasing down the learners to accomplish a certain response rate, they may then optionally go through excruciating levels of pain to aggregate, slice/dice, diagnose, and sense making of all the data. As we sit here in the era of crowd sourced, democratized, and digitally social conversations, I wonder about the following:

  • Should we measure reaction (or) should we measure engagement?
  • Should we democratize the reaction process & make it transparent?
  • Should we re/imagine the way we look at the survey process?

Create a facility wiki, wordpress, or something similar

Drawing inspiration from Yelp, create facility profile and let your employees rate, and provide both qualitative and quantitative feedback. Just like Yelp. If people are truly either positively or negatively moved by the location, they will make it a point to let you know. In addition, there will be other tips that might come out of it that other learners could potentially use. Even simple things like bring an extra jacket, check out the trail can go a long way. This way your learners don’t have to wait till the end and remember to give their feedback. It can happen in the moment.

Create a faculty social profile

Drawing inspiration from LinkedIn’s recommendations & Twitter/Facebook based-customer service approaches, create faculty profile and let your learners rate, and provide both qualitative and quantitative feedback to the faculty members ondemand. Let that conversation flourish. It will help both the faculty member and the learners to build a better bond because at the end of the day that matters. You can do this with Yammer, Twitter, Facebook, LinkedIn, or even Poll Everywhere that just shows information to the faculty member real-time. You can pick an option based on the transparency levels you intend to achieve.

Let learners like/share/recommend a interactivity/interaction

The issue with current approaches to gathering reaction is they aren’t granular enough or not in the moment. If you are building a traditional eLearning module, embed link/share/recommend to various content interactions. If you are building it for the classroom, remind the learners that they can continue to share their reflections or feedback. You can achieve all this with either Yammer, Twitter, Facebook, LinkedIn, Poll Everywhere, or spending a bit more dollars on the content design/build.

Rethink the analytic

The human brain is wired to differentiate via numbers. That is how we have trained ourselves since eons. Hence the stigma and dogma towards surveys. If you rethink the analytic into series of helpful info-graph & keyword analysis, wordles, segregation by categories (e.g. Faculty/Facility/Interaction), along with minimum thresholds to flag you for quality, it can go a long way. Get away from survey and reports and get more visual.

Simplify experience

Finally, ensure that the learner can do all this simply from one spot. At the end of the day asking for engagement is a privilege into someone’s time and availability. You have to make this just simple.

Now are you wondering where have we moved the dial from reaction to engagement? Guess what the answer will lie in metrics that will come out of it. We will talk more about that in a subsequent post.

Do you do any of the above? Is it time to shake it up a bit?

Credits: Image sourced under Creative Commons License from Flicker. Click here for the actual source

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.