A Clinical Method for Predicting Leadership: Part 2
Methods and Procedures
What research methods did we use to determine the predictive validity of our assessments of leaders of private equity funded ventures? This entry describes the methods and procedures in the study featured in this series, designed to determine the accuracy of predictions regarding managerial performance that were based on a clinical psychological method.
We looked at assessments performed between 2002 and 2006 carried out on behalf of one particular middle-market private equity firm. These investors had actual performance data against which to compare indicators from our assessment. The firm made control equity investments in companies where its experience and resources can be applied to accelerate earnings growth through aggressive investment management plans. It typically did not make turnaround or venture investments. By limiting our analysis to assessments performed for this one private equity firm, we eliminated variability among investment styles of different private equity firms and focused on companies where management capability is an important factor in the success of the investment. Subsequent posts in this series formally compare our assessment findings to the operating executives’ actual performance. Firm and operating executive performance measures were provided by investors two, three, and, in two cases, five years after we had assessed the executive.
The circumstances of the assessments varied on two dimensions. First, at the time we made each assessment, the private equity firm may or may not have already invested in the company. Second, the executive may have been either part of an incumbent management or an outside candidate hired to fill a management role. In most cases, our assessment occurred in parallel with other diligence efforts prior to a deal. In two cases, our assessment took place when investors were contemplating adding an executive to an existing portfolio company.
Before we make an assessment, we gain an understanding the specific demands of the operating environment in which the executive is to perform. We ask what investors hope to achieve by making the investment — how they plan to create value — and the executive’s role in implementing strategy. This is important in understanding and interpreting the psychological assessment.
After we make an assessment, we prepare a formal written report. The report looks at competencies and capabilities organized around factors that bear importantly on the executive’s ability to perform as required. These factors are Judgment, Influence, Management, and Personality (I will briefly define these domains at the end of this post). To determine whether an executive has the resources to deploy these capabilities consistently and effectively under the proposed operating conditions, we assess his or her personality and psychological functioning.
The methodology we use to assess executives uses standard clinical methods adapted to address concerns of interest to investors. In particular, they need to be confident of an executive’s ability to function in a stressful environment. I will describe our particular assessment strategy and data collection techniques in the next post. For various reliabilities and validities of the measures, see, for example, Pratch and Jacobowitz (1996; 1998).
The executive consented to permit us to use the information gathered during the assessment for research into the relationships among personality, development, and leadership effectiveness. We have attempted to eliminate identifying information, including changing the names of the executives and characterizing the investments in general terms.
Each psychological assessment lasted approximately four hours with the executive and followed the methodology that will be described in the next post. The executive understood that the assessment was to provide information to the private equity firm regarding his or her managerial strengths and weaknesses and leadership style as these concerned his or her fit with the management role. (The executive consented to permit us to use the information gathered during the assessment for research into the relationships among personality, development, and leadership effectiveness. We have attempted to eliminate identifying information, including changing the names of the executives and characterizing the investments in general terms.) The executive’s test outcomes provided the basis for our conclusions and recommendations.
In the spring of 2008, we interviewed the investment professionals responsible for overseeing the deals. We had also monitored the executive’s performance through discussions with the investors at various prior intervals, occurring roughly one and two years after we had conducted our assessment.
Exhibit 1 details the protocol we used to collect data on the predictive validity of the assessments. Section 2 of this exhibit describes the particular deal. Section 3 documents investors’ evaluation of the executive’s performance. Section 4 details the assessment procedure and recommendations. Section 5 documents how well our assessment predicted the executive’s actual performance. In addition to this general evaluation of our assessment, this process allowed us to collected information on many specific components of our assessment process.
Exhibit 1: Follow Up Protocol
Next month’s post will describe the data that was collected.
About the Author
Leslie S. Pratch is the founder and CEO of Pratch & Company. A clinical psychologist and MBA, she advises private equity investors, management committees and Boards of Directors of public and privately held companies whether the executives being considered to lead companies possess the psychological resources and personality strengths needed to succeed. In her recently published book, Looks Good on Paper? (Columbia University Press, 2014), she shares insights from more than twenty years of executive evaluations and offers an empirically based approach to identify executives who will be effective within organisations — and to flag those who will ultimately very likely fail — by evaluating aspects of personality and character that are hidden beneath the surface.
