How DNPs Can Use Empirical Critical Thinking Tools

Evaluating and Applying Empirical Critical Thinking Tools

The purpose of this paper is to provide a comprehensive evaluation of two prominent empirical critical thinking assessment tools: 1) the California Critical Thinking Skills Test (CCTST) and 2) the Watson-Glaser Critical Thinking Appraisal (WGCTA), within the specific context of the DNP program. Validated critical thinking assessment instruments such as the CCTST and the WGCTA provide nursing leaders with objective metrics to quantify and compare the reasoning capacities of both current and prospective nurses across multiple healthcare settings as well as for ongoing, honest self-assessments. Following this evaluation, the paper describes the implications of empirical critical thinking tools for the nursing profession in general and nursing leaders in particular. Finally, the paper presents a summary of the findings that emerged from the foregoing evaluations and analyses in the conclusion.

Empirical Critical Thinking Tools

California Critical Thinking Skills Test (CCTST)

The CCTST is a validated, standardized instrument that was developed by a team of educational psychologists in order to measure critical thinking abilities. In sum, the CCTST provides an overall score for critical thinking skill as well as scores on cognitive skill subsets like deduction, interpretation, explanation, and self-regulation capacities that facilitate sound reasoning. The CCTST has been extensively tested for validity and correlates highly with other critical thinking metrics and academic success (Bycio & Allen, 2009).

The questions on the CCTST were intentionally designed to objectively assess capability in skills such as properly identifying assumptions, drawing accurate conclusions, assessing credibility, and avoiding logical fallacies. The CCTST quantifies strengths and deficiencies in both the cognitive and attitudinal dispositions underpinning exceptional critical thought. It has proven useful in both academic contexts to evaluate student development as well as professional environments to assess personnel capacities for complex decision making. By providing a norm-based measurement of critical thinking capabilities, the CCTST seeks to support the improvement of essential skills for ongoing analysis and evaluation (Facione, 1990). One of the major strengths of this instrument is its demonstrated validity over time as discussed further below.

History of the tool: Who created it. How was it created. Availability of the tool.

Developing during the early 1990s by a team of educational psychologists headed by Peter Facione at California State University, Fullerton, the CCTST has been evaluated by a large, nationwide Delphi consensus study which defined critical thinking and identified core critical thinking skills. Guided by this team of educational psychologists, consensus conceptualizations of critical thought were created based on the results of the Delphi study and Facione’s team developed and validated multiple choice test questions to objectively measure each of the key skills dimensions (Knox, 2013). The development process involved writing and empirically trial testing hundreds of test questions and the subsequent retention of just those properly calibrated items that demonstrated the requisite qualitative characteristics and quantitative measurement properties (Facione, 1990). This rigorous approach served to ensure that the final CCTST possessed content and face validity, reliably drew on all aspects of critical thought as conceptualized by the Delphi theoretical framework, and exhibited sensitivity to development changes in individual critical thinking abilities over time (Facione, 1990).

The CCTST first became commercially available in 1994 from Insight Assessment, a company that was co-founded by Facione specifically to market this critical thinking assessment tool. The company continues to publish the test for individual or bulk purchase for degree program evaluation purposes (CCTST, 2023) as well as for other testing purposes as described below.

Functionality: Questions list on tool; meaning of the results.

The test is comprised of 34 multiple choice questions that describe scenarios requiring the application of critical thought across areas like analysis, evaluation, inference, logic and reflection at increasing levels of complexity (Facione, 1990). According to Facione (1990), “Twenty of the questions otter four choices, fourteen offer five. For purposes of CT skill assessment, one answer has been designated the superior choice on each question” (p. 4). The evaluation of the instrument as noted above subsequently identified three core skill subsets that align with consensus understanding in the field: analysis, evaluation, and inference; these three subsets relate closely to overall critical thought capabilities (Facione, 1990).

In addition, two additional facets, deductive reasoning and inductive reasoning, were also shown to possess strong connections to the measured critical thinking skills. Performance on these category-specific competencies, as measured through appropriately designed test questions, largely correlates with total scores achieved on validated critical thinking assessments administered both before and after instructional interventions. By segmenting aggregate scores into percentile rankings across the five narrow skill types as well as a broad critical thinking score, normative benchmarks have been made available offering indicative interpretations at the subdomain level. These subgroup metrics allow identification of relative strengths and weaknesses that can inform personalized improvement in discrete critical thought realms tied to the generally recognized components framework. Ongoing assessment refinement aims to produce classifications that sustain reliable, useful guidance for nurturing comprehensive excellence in this fundamental capability (Facione, 1990).

DNP Project: How will this information (critical thinking) aid in the completion of your DNP project?

Improving critical thinking capabilities provides DNP students with an enhanced ability to thoroughly analyze the theoretical bases and research methodologies that support their clinical project proposals. Moreover, continuously honed analytical skills enable DNP candidates to better synthesize the findings that ultimately emerge from their project within the broader scope of evidence-based research.

Watson-Glaser Critical Thinking Appraisal (WGCTA)

The Watson-Glaser Critical Thinking Appraisal (WGCTA) evaluates competency to judiciously examine written content, comprehend any corresponding implications, and make sound determinations by testing for essential reasoning skills that are aligned with real-world decision contexts across continuing education, human resources, and workplace settings. In sum, the assessment provides a norm-referenced baseline metric of current critical thought capabilities while informing targeted improvement efforts (WGCTA, 2023).

History of the tool: Who created it. How was it created. Availability of the tool.

The first version of this instrument was developed a century ago and ultimately published in 1937 by Goodwin Watson and Edward Glaser. Since that time, the WGCTA has undergone many modifications and developments, including internationalization (WGCTA, 2023). The tool is currently available from Pearson as noted below.

Functionality: Questions list on tool; meaning of the results.

The features a sequence of readings and hypothetical situations posing issues for analysis. Test-takers must address a series of multiple-choice questions that correspond to each scenario, with compatibility for both proctored assessment and unsupervised self-administration (WGCTA, 2023). The assessment includes 80 or 40 items in the standard or short form respectively across five subareas checking for skills in drawing reasonable inferences, identifying unstated assumptions, properly evaluating lines of argumentation, demonstrating deductive logic, and interpreting semantic meaning. Test takers are asked to analyze components of passages and select the best fitting response option. Both forms of the WGCTA come in parallel variants allowing pre-post administration, with the short form containing a subset of standard test content. Extensive test scoring normative data stratified across industries (e.g. healthcare, technology), functions (e.g. managers, sales), and job levels (e.g. director, professional) facilitates meaningful performance comparisons. This inventory delivers standardized accountability for key facets of critical thought while offering efficiency in assessment through its concise alternative (Watson-Glaser Critical Thinking Appraisal, 2023).

DNP Project: How will this information (critical thinking) aid in the completion of your DNP project?

Besides providing an aggregate score reflecting overall proficiency, individual performance reports give examinees personal percentile ratings across the three “RED facets” (e.g., recognize assumptions, evaluate arguments and draw conclusions) that identify areas of relative strength versus areas that could use improvement. By framing results in relation to subgroup norms and breaking down capacities across core constructs of competent analysis, interpretation and deduction, those undergoing assessment using the WGCTA receive targeted guidance to improve skills that are essential for judicious evaluation and decision making in educational, workplace, and interpersonal contexts (WGCTA, 20323).

Evaluation of the Tools

Both of these tests require some time to administer, with the CCTST requiring about 45-50 minutes and the full version of the WGCTA requires approximately 60 minutes compared to 30 minutes for the abbreviated alternative. In addition, the results of both of these tests require careful evaluation according to the respective test’s scoring guidelines to ensure optimal accuracy of interpretations. Carefully administered and thoughtfully analyzed, though, the findings that result from both of these tests be used when formulating practice change recommendations that can enhance patient outcomes, nursing workflows, protocols and even an organization’s policies with broad implications for the nursing profession.