Measuring the Human Factor: Pitfalls of the ROI Mindset
A recent post on The ROI Dilemma forwarded to me by our PM College Business Development maven Johanna Mickel sparked a conversation about the complexities of determining the value of project management training.
Johanna told me, “One of the things that clients have said to me is that it’s nearly impossible to measure the impact of training on the overall business, but I disagree. If you stay in touch with participants, and require them to capture the story of how this training made a difference to you in your job, you gather, at the very least, anecdotal evidence of improvement.” Enough “stories” of the same type, and you should be able to look at related business metrics and gauge the impact.
For example, if multiple project managers within the PMO report that they have seen an improvement in schedule variance since their teams participated in training, you should be able to quantify not only a numerical or percentage improvement in projects delivered on time, but a financial metric associated with improved schedule targets. This would vary depending on the business, of course, but one example would be, in projects that are in response to regulatory demands, the avoidance of penalties.
Another hard-to-quantify, but equally important area involves opportunity costs, or loss avoidance. It’s hard to put a number on what didn’t happen, yet these impacts can be significant. Johanna told me about a PM College class where, when discussing risk, our instructor challenged participants to validate their assumptions about actual projects under way in their organizations. One participant worked for a firm involved in an acquisition that revolved around the firm obtaining a new technology. He discovered, in trying to validate the underlying assumptions, that the firm being acquired did not actually own the technology in question! “How do you capture ROI for something like that?”
The answer is framed in The ROI Dilemma as “shifting the narrative”: opening up a wider field of possible positive impacts beyond direct, causative, or financial measures. The author also urges training leaders to “play the long game” by constantly communicating with executives about how training value, although it may feel intangible, does show up in tangible impacts. To this we would add: baseline, baseline, baseline.