This paper investigates the role of security metrics in information security governance, arguing that while metrics are widely valued, they do not on their own guarantee improved security outcomes. Drawing on scholarly and professional literature, the paper defines key terms, outlines the characteristics of effective metrics, and surveys multiple dimensional frameworks. It then examines governance metrics — addressing board-level responsibilities, key performance indicators, and business goal alignment — alongside technical metrics, including tools such as NIST SP 800-53, Common Criteria EAL, and OWASP DREAD. The paper concludes by identifying significant limitations in current measurement methods, including the inability to capture real-time change and the difficulty of accounting for unpredictable adversarial behavior.
The paper demonstrates effective use of synthesized literature review: rather than simply summarizing sources one by one, it weaves together findings from multiple authors (Barabanov et al., Pironti, Stoddard, McQueen) to build a cumulative argument. This technique shows how different bodies of evidence converge on a single analytical conclusion — that current metrics carry inherent measurement limitations.
The paper opens with a research objective and motivating survey data, then works through definitions, characteristics, and dimensional frameworks before diving into the two focal metric types (governance and technical). Each substantive section follows a consistent pattern: define the concept, enumerate its components, and cite supporting evidence. The final section pivots to critique, identifying real-time measurement gaps and the challenge of unpredictable adversaries, before the conclusion restates the thesis.
The objective of this study is to examine the concept that the use of various metrics has tended to improve security; however, metrics alone may not necessarily improve security. This study focuses on two well-known metric types: governance metrics and technical metrics.
Barabanov, Kowalski, and Yngstrom (2011) state that the greatest driver for information security development in the majority of organizations "is the recently amplified regulatory environment, demanding greater transparency and accountability. However, organizations are also driven by internal factors, such as the needs to better justify and prioritize security investments, ensure good alignment between security and the overall organizational mission, goals, and objectives, and fine-tune effectiveness and efficiency of the security programs" (p. 1).
A survey conducted by Frost and Sullivan demonstrated "that the degree of interest in security metrics among many companies (sample consisted of over 80) was high and increasing (Ayoub, 2006); while, in a global survey sponsored by ISACA, dependable metrics were perceived to be one of the critical elements of information security program success by many security professionals and executives, though they were also deemed difficult to acquire (O'Bryan, 2006)" (Barabanov, Kowalski, and Yngstrom, 2011, p. 2).
The focus on governance also includes a "need for proper measurement and reporting on all the echelons within the organization, starting at the highest level. Another survey instigated by ISACA showed that organizations missing an information security governance project had identified metrics and reporting as the areas in their information security programs where the lack of quality was most noticeable" (Barabanov, Kowalski, and Yngstrom, 2011, p. 2). Barabanov, Kowalski, and Yngstrom further report that their study highlights the requirement to recognize "that measurement and reporting are connected with management on all organizational levels" (p. 2).
There is a great deal of ambiguity surrounding the precise definition of the term metric or security metric, according to Barabanov, Kowalski, and Yngstrom (2011), since the terms "security metric and measure tend to be used interchangeably" (p. 3). Definitions that have been proposed include the following:
(1) Measure — A variable to which a value is assigned as the result of measurement, where measurement is defined as the process of obtaining information about the effectiveness of Information Security Management Systems (ISMS) and controls using a measurement method, a measurement function, an analytical model, and decision criteria (ISO/IEC, 2009a).
(2) (IS) Measures — The results of data collection, analysis, and reporting, which are based on, and monitor the accomplishment of, IS goals and objectives by means of quantification (Chew et al., 2008).
(3) Metric — A consistent standard for measurement, the primary goal of which is to quantify data in order to facilitate insight (Jaquith, 2007).
(4) Metric — A proposed measure or unit of measure designed to facilitate decision making and improve performance and accountability through collection, analysis, and reporting of relevant data (Herrmann, 2007).
(5) Metrics — A broad category of tools used by decision makers to evaluate data. A metric is a system of related measures that facilitates the quantification of some particular characteristic. In simpler terms, a metric is a measurement that is compared to a scale or benchmark to produce a meaningful result (McIntyre et al., 2007).
(6) Security Metrics — The standard measurement of computer security (Rosenblatt, 2008).
Although the specifics of these definitions vary, certain common characteristics generally emerge (Barabanov, Kowalski, and Yngstrom, 2011, p. 20). Primarily, metrics and measures are "considered to be measurement standards that facilitate decision making by quantifying relevant data, where measurement refers to the process by which they are obtained" (Barabanov, Kowalski, and Yngstrom, 2011, p. 20).
Stoddard et al. (2005) report that the term metrics "describes a broad category of tools used by decision makers to evaluate data in many different areas of an organization. In its simplest form, a metric is a measurement that is compared to a scale or benchmark to produce a meaningful result" (p. 3).
The characteristics of good metrics are reported to include the following:
(1) Metrics should measure and communicate things that are relevant in the specific context for which they are intended, and be meaningful — both in content and presentation — to the expected target audience.
(2) The value of metrics should not exceed their cost. Measures should be cheap or easy enough to obtain so that potential inefficiencies in data collection do not drain resources needed for subsequent stages of measurement or for other parts of the organization.
(3) The timeliness and frequency of measurement must be appropriate for the rate of change in the targets of measurement, so that the latency of metrics does not defeat their purpose. It should also be possible to track changes over time.
(4) Good metrics should ideally be objective and quantifiable, derived from precise and reliable numeric values rather than qualitative assessments, which carry potential for bias. They should likewise be expressed using readily understood and unambiguous units of measure.
(5) Metrics must be consistently reproducible by different evaluators under similar circumstances; therefore, a sufficient level of formality is expected from defined measurement procedures (Barabanov, Kowalski, and Yngstrom, 2011, p. 21).
You’re 38% through this paper. Sign up to read the remaining 3 sections.
Sign Up Now — Instant Access Already a member? Log inAlways verify citation format against your institution’s current style guide requirements.