Are You Picking Effective CEOs and CFOs?

By Leslie S. Pratch

Picking good CEOs and CFOs for your portfolio companies is generally thought to be quite important to investment performance. You probably have a process that you use with some discipline. But does it work? And even if it does, can you improve it? You can do some research to figure out the answers to these important questions.

I’ve worked with some firms to try to understand:

• Whether some or all of the information they currently collect about CEO/CFO candidates in fact statistically predicts their performance outcomes.
• How the firms can make their hiring processes more efficient and more accurate in predicting success.

Many firms use a process of testing, assessment by outsiders, and interviews by firm members to select their CEOs and CFOs. If you looked at a firm’s hiring approach, you would hope to find that:

• The hiring process predicted success vs. failure of the hired candidates; that is, people who did better in the process in fact did better on the job, and
• Hired candidates were more highly rated/scored in the process; that is, hired candidates tended to do better in the process than candidates who were not hired.

In some recent research, the predictive validity of firms’ typical methods was not supported by statistical analysis. We found no evidence that any of the methods depicted above work to distinguish ultimately very successful leaders from those who performed considerably less well (or badly) after they were hired. The data that firms collected (and documented) couldn’t be shown to be helpful in predicting performance — although we cannot say for sure it was useless either.

Surprisingly, we also found that managers selected for positions appeared statistically indistinguishable (based on the data provided) from those the firms chose not to hire.

WHAT CAN YOU DO IF YOU ARE NOT SURE YOUR HIRING PROCESS WORKS VERY WELL

You can choose from at least three strategies. You can:

1. Duplicate the research to figure out if there are elements of your process you should definitely keep or get rid of, and/or

2. Take some “no regrets” actions to improve your process

3. Add some statistically valid components to your process

Duplicate the Research with your Own Data

Duplicating the research with your own data is possible if you have adequate data, and not too expensive if the data is accessible and complete. The data used is all the information you have on a set of candidates — scores, interview ratings, and other evaluations, plus the performance ratings of the CEOs and CFOs you ultimately hired. The research involves standard statistical analysis steps — such as getting the data in one place, cleaning the data, coding the data into discrete categories where required, and then running and interpreting the statistical results.

If you do your own research, you may discover that there are steps in the process that are adding no value — and you can save some money by eliminating them.

Take “No Regrets” Actions to Improve your Process

In the absence of data, you can still do some “good practices” that might be useful and that have minimal costs. These include:

• Understand, specifically for your firm, the competencies required of portfolio executives and how the operating and deal team can recognise, in an interview or assessment situation, that a manager either has or does not have the competency. In this process
• Focus on the competencies that you believes are most critical to success
• Use “behavioral anchors” (descriptions of the behaviors that demonstrate a competency) to consistently measure the competencies.
• Train interviewers in the firm to correctly execute the interview process so they efficiently get at what you are looking to understand
• Calibrate across interviewers to ensure that competencies are rated consistently across interviewers
• Consider using some kind of test (e.g., of cognitive ability) to screen out candidates before you spend too much time or money evaluating them. For example, if you do not want to hire a CEO who isn’t smart no matter how good they are on other dimensions, then give candidates a quick cognitive ability test upfront and you’ll probably eliminate quite a few.

Add the Right Kind of “Statistically Valid” Methods

You might want to add statistically valid measures, properly defined. There are many methods that describe themselves as “statistically valid”, but that term is used ambiguously.

Sometimes, the devisers/sellers of the method mean that the method reliably measures the thing it claims to measure, that is, a measure of “leadership” will accurately measure the specific concept of “leadership” defined by the method (which, by the way, may or may not have anything to do with your conception of leadership).

Other times, the devisers/sellers of the method can show that the method can predict outcomes based on the input. This kind of “predictive validity” is what you are looking for; however, you still need to sure that the prediction is relevant. In one case I know of, the studies about predictive validity were based on “managers”, and it turned out that being able to predict whether someone would be a good manager is quite different than being able to predict if someone would succeed as a CEO; the method was provide predictive validity but in a way that was irrelevant or even misleading for the purpose for which the private equity firm was applying the method.

Feel free to contact me if you want to talk about methods that have predictive validity including some that I use, or about methods that are claiming predictive validity if you’d like help in understanding the claims and what they mean.

About the Author

Leslie S. Pratch is the founder and CEO of Pratch & Company. A clinical psychologist and MBA, she advises private equity investors, management committees and Boards of Directors of public and privately held companies whether the executives being considered to lead companies possess the psychological resources and personality strengths needed to succeed. In her recently published book, Looks Good on Paper? (Columbia University Press, 2014), she shares insights from more than twenty years of executive evaluations and offers an empirically based approach to identify executives who will be effective within organisations — and to flag those who will ultimately very likely fail — by evaluating aspects of personality and character that are hidden beneath the surface.

« Germany — is Madame Merkel up for a Ruse? The Macron Effect: Taking France by Storm »

Originally published at www.europeanfinancialreview.com.