Job Analysis, Competency Modeling, and 360° Feedback

Introduction to KSA Determination Methods

Literature has shown that, just as the skills, knowledge, and competencies required for success differ from position to position, the methodologies for determining the specific KSAs required for success also differ. For instance, whereas administrative and clerical positions require only a local job analysis to determine the required KSAs, supervisory and executive positions call for the development of a competency model in addition to a local job analysis.

For an administrative position, the methodology for KSA determination would essentially involve analyzing the requirements of the position under changing business environments — drawing on information presented in books and other business-related literature — and then determining the knowledge (such as a university degree or relevant certification) and skills (the minimum amount of relevant experience) that an incumbent would need at entry in order to execute the duties and responsibilities of that position.

This methodology is simpler than that used for managerial and supervisory positions, but its applicability is questionable given the difficulty involved in validating the requirements. For instance, managers will often find it difficult to determine whether to select a candidate with the relevant certification or one who lacks certification but possesses the relevant experience. Some managers will opt for the latter and others for the former, and there cannot therefore be a one-size-fits-all strategy for the full validation of requirements (Reed, McCloy & Whetzel, 2010).

Job Analysis for Administrative Positions

This simpler model may not work effectively with supervisory and managerial positions. In such cases, competency modeling would be used in addition to local job analysis to determine the required KSAs. The methodology in this context would be considerably more complex, involving: (i) analysis of the job or position under changing business environments, as is also the case with administrative positions; (ii) identification of effective and ineffective behaviors through leadership-based theory, including analysis of the cognitive and affective leadership skills required; (iii) analysis of gathered data and formulation of a competency model combining all relevant skills and competencies; and (iv) validation of the appropriateness of the developed model (Schmit & Strange, 2010).

Unlike the methodology used for administrative positions, this approach is competency-based, allowing the manager to assess a candidate's suitability for the position in a holistic manner — not solely on the basis of experience and educational qualifications. For this reason, the methodology is also suitable for determining KSAs for entry-level positions, where incumbents are new to the organization and it is more reasonable to assess them from a holistic perspective: that is, in terms of their suitability not just for the position at hand, but for the organization as a whole.

Competency Modeling for Supervisory and Managerial Roles

Generally, however, job analysis and competency modeling are most appropriate when an organization's jobs do not differ substantially from one another, since in that case fewer resources need to be spent on the development and validation of different competency models — a process that can be quite costly.

There is a continuing debate over whether multi-rater (360°) feedback (MRF) ought to be used for developmental purposes only or for administrative purposes as well, such as in determining who should be promoted. Proponents of the latter position argue that the benefits of MRF are not fully realized if it is restricted to development, which in practice means behavioral change (Bracken et al., 1997).

After reviewing the views presented by both sides, it is the position here that organizations are safer using MRF for developmental purposes only. The reasoning is straightforward: when information collected through MRF is used for assessment and other administrative purposes, it is no longer safe. For instance, if an individual receives negative ratings when the data is being used to determine their eligibility for a salary increase or promotion, they are likely to become defensive, reject the feedback, and begin to harbor negative feelings toward their raters. If, however, the information is not tied to any administrative function and is used purely for behavioral development, the individual is more likely to receive it positively and make changes where necessary, simply because there is no adverse consequence in doing so.

Multi-Rater Assessment: Development vs. Administrative Use

Using MRF for administrative purposes rests on the flawed assumption that information provided by raters is always accurate and that raters act in the best interests of the person being rated. A manager who holds employees accountable for underperformance is more likely to receive lower ratings than a more lenient one who prioritizes building favorable ratings for themselves. If information gathered through such ratings is then used for performance appraisal, organizations risk promoting individuals into leadership positions who are unsuitable but popular — rewarding those who simply do what staff want rather than what the organization needs.

To avoid these outcomes, managers are better off reserving MRF for developmental purposes. The developmental and administrative functions can only be responsibly combined if: (i) there are effective measures to hold raters accountable for their ratings as well as for follow-through action; (ii) raters are made part of the solution; and (iii) the organization has other means of assessing employees' suitability for advancement and uses MRF only as a secondary tool in that process (Bracken et al., 1997).

Conclusion and Conditions for Combined Use

Schmit, M. J., & Strange, J. M. (2010). Assessment for supervisory and early leadership roles. In J. Scott & D. Reynolds (Eds.), Handbook of workplace assessment (pp. 365–394). Jossey-Bass.