Methodological challenges in measuring leadership through quantitative and qualitative approaches

One of the simplest and most common methods to assess leadership attributes are questionnaires. Questionnaires, such as the Leadership Behavior Description Questionnaire (LBDQ) use quantitative survey techniques to assess the degree to which organizational actors display leadership behaviors (Bryman, 2011). It should be noted that the LBDQ asks not the leaders themselves whether they display specific leadership attributes, but members of the organization. The LBDQ “provides a technique whereby group members may describe the leader behavior of designated leaders in formal organizations. The LBDQ contains items, each of which describes a specific way in which a leader may behave” (Tepper, 2011). However, questionnaires submitted to followers or leaders have a number of constraints which limit their accuracy. Firstly, the data collection method design can bias results regarding behaviors which may be ambiguous, such as forcing respondents to answer whether they agree or disagree, or rate something on a Likert-type scale.
Attribute studies, in contrast, attempt to link specific personality attributes to leadership abilities, versus behaviors. They too often survey leaders and followers about qualities they associate with leadership. For example, in a study conducted by the staffing company Robert Half, leaders and followers were asked to rank the following attributes: “Accessibility, Collaborative Mindset, Competitiveness, Decisiveness, Fairness, Integrity, Strategic Mindset, and Transparency” (Lipman, 2016, par.5). Although the ranking varied depending on the individual’s position in the corporate hierarchy, all persons answering the questionnaire ranked integrity first. Although this was taken by the constructor of the survey to suggest that integrity, is the most important leadership attribute, attribute studies have similar problems to behavioral studies, in terms of the fact that they are quantitative-type instruments demanding a fairly narrow range of responses, which may not fully encapsulate the full range of feelings of the respondents. A further problem with quantitatively-driven attribute studies is that various respondents may define the attributes differently, based upon personal associations or cultural background, while at least with behavioral assessments, there is objective, observable behavior.
Qualitative studies are another way to measure leadership. Rather than data-driven analysis using large number sets, qualitative research is intentionally subjective, and based upon observations, personal interviews, and is not necessarily generalizable to a larger population. Examples of qualitative approaches include phenomenology, case studies, and grounded theory approaches. For example, a case study of a notable organization such as Google or Steve Jobs might be undertaken. Grounded theory approaches achieve a more generalizable result, by using inductive observations to construct a theory after the fact, grounded in research, such as one qualitative case study of mergers and acquisitions at a firm, which found a failure of leadership to successfully navigate a merger to be attributed to poor communication, different organizational cultures, a lack of trust in the leadership, and contested leadership positions (Smollan & Griffiths, 2020). While qualitative studies can be valuable in terms of bringing light to unexpected aspects of leadership on one hand, the narrowness of focus often make organizations reluctant to set policy or make generalizations based upon their results.
Thus, leadership, despite the general agreement of its importance, remains a very difficult attribute to study. First of all, there is widespread disagreement about how to define various behaviors and qualities associated with leadership.