Evaluating Nursing Education Assessment Tools

Nursing Education Assessment Project Coursework early in a nursing education program covers a broad range of topics and extensive amount of details must be committed to memory. Assessments that are directly tied to coursework are primarily formative assessments, which demonstrate the ongoing learning over the period of the course. Formative assessments generally take the form of quizzes and clinical demonstrations of a particular knowledge set recently covered in during a class or classes.

Summative assessments are generally used at the end of a course to assess the overall learning that has taken place during the course; summative assessments include final exams or tests, practicum demonstrations, and capstone projects. The focus of this assessment project is a formative criterion-referenced test of general, fundamental nursing education knowledge. The items used in the test are included in Appendix A -- Nursing Education -- Fundamental Concepts.

Twelve individuals were approached to take the exam as though it was an actual assessment of their knowledge, and of the twelve candidates, ten people agree to complete the exam. The general exam is useful for students to gauge their learning and recall of basic concepts acquired during the early phase of their program. A collection of comprehensive tests at this and more advanced levels can serve as a basis for preparing for state board exams.

The results of the examination for the ten individuals are shown in aggregate form below. Planning and Development To arrive at a comprehensive test of basic nursing concepts, the planning and development process addressed several common areas in nursing education. The objective was to achieve a balance of question items across several different areas of the fundamental nursing knowledge base. A variety of resources were used to arrive at the item selection, including professional standards and competencies, learnings from previous coursework, and personal experience.

These areas are as follows: Examination Procedures -- 32% Nursing Actions -- 28% Nursing Terminology -- 4% Nursing Theory -- 8% Management & Supervision -- 4% Legal Liability -- 8% Patient Rights -- 4% Medical Records -- 4% Care Objectives -- 8% Levels of Blooms Taxonomy were used to organize the test items and to achieve a balance of test item difficulty and knowledge scope. The levels of Bloom's Taxonomy that were introduced in the original taxonomy in 1959 include the following: Application, analysis, comprehension, and knowledge (Bloom, 1969; Anderson, et al., 2001).

However, this assessment uses the revised taxonomy from 2001 (See Appendix B -- Bloom's Taxonomy Revised) and instead uses the following categories of the taxonomy, including further delineation of the knowledge category, as follows: Remember (Recognizing, Recalling); Understand (Interpreting, Exemplifying, Classifying, Summarizing, Inferring, Comparing, Explaining); Apply (Executing, Implementing) Analyze (Differentiating, Organizing, Attributing); Evaluate (Checking, Critiquing); Create (Generating, Planning, Producing) (Armstrong, 2015).

The six cognitive processes are based on knowledge, which is further divided in the revised taxonomy into types of knowledge utilized in cognitive processing, as follows: Factual Knowledge (Knowledge of terminology, Knowledge of specific details and elements); Conceptual Knowledge (Knowledge of classifications and categories, Knowledge of principles and generalizations, Knowledge of theories, models, and structures); Procedural Knowledge (Knowledge of subject-specific skills and algorithms, Knowledge of subject-specific techniques and methods, Knowledge of criteria for determining when to use appropriate procedures); Metacognitive Knowledge (Strategic Knowledge, Knowledge about cognitive tasks, including appropriate contextual and conditional knowledge, Self-knowledge) (Armstrong, 2015).

Content Outcomes Remember Understand Apply Analyze Evaluate Create Knowledge Examination Procedures 1 14, 17 13 12 -- Procedural; 23 -- Factual Nursing Actions 16 10, 19, 20, 22 15, 21, 25 3 -- Procedural; 18 - Conceptual Nursing Terminology 2 - Factual Nursing Theory 4, 5 24 - Conceptual Management / Supervision 6 Legal Liability 7, 9 Patient Rights 8 - Conceptual Medical Records 11 - Procedural TOTALS 4 6 3 3 1 0 25 Percentage of Each of Bloom's Taxonomy Cognitive Domains in the Nursing Fundamentals Assessment Exam Remember 16% Understand 24% Apply 12% Analyze 12% Evaluate 4% Create 0% Knowledge Conceptual 12% Factual 8% Procedural 12% Metacognitive 0% Evaluation The reliability of a test and the validity of test items are the basis for determining the dependability and trustworthiness of an assessment, with both of these measures playing a pivotal role in professional examinations (Billings & Halstead, 2009).

In order to consider a test reliable, there must be evidence of strong test-retest replication or some other demonstration that the test provides consistent scores within in an evidentiary bandwidth of test takers (Billings & Halstead, 2009). The test was administered to 10 individuals in a nursing education program at a major university, such that levels of experience fell within a range of one to two years of nursing education.

Raw Scores Data Number of students 10 Number of items 25 Maximum point value 100 Highest score 100 Lowest score 76 Median 88 Mean 86.8 Standard Deviation 8.527 Cronbach's Alpha (test reliability) .998 The internal consistency of the assessment was conducted using Cronbach's Alpha in IBM SPSS. The Cronbach's Alpha score attained was .998, which indicates that the question items do measure the same concept. Moreover, the inter-item correlation matrix displays all positive numbers, with the lowest correlation at .991 and the highest correlation, naturally, at 1.00. The Sum of Squares between people was 66963, with a mean square of 2575 (F=3.144, p = .001).

The Intraclass Correlation Coefficient (ICC) shows single measures at .980 (95% Confidence Interval, F = 525.4). This was conducted with a two-way random effect model in which both the effects of the people and the effects of the measures are random. And the intraclass correlation coefficients were measured using the definition of absolute agreement. The test reliability for this assessment instrument was found to be high using both Cronbach's Alpha and Intraclass Correlation Coefficient measures.

The high reliability and high F scores can be at least partially attributed to the basic nature of the question items, very few of which were troublesome to the nursing student taking the test. Indeed, the test items that showed the highest difficulty levels in the analysis were as follows: Questions 5, 19, 20, and 25 at P = .01; and, questions 9, 14, and 19 at P = .05.

Considering the question items that stood out in the item difficulty analysis, two types of questions were challenging: Questions that required discerning among several potentially reasonable options, and questions that required a substantive ability to recall a detail. Both of these item types are known to be used in state board exams, so the information nursing students learn about their relative strengths and areas of improvement via this fundamental nursing skills assessment can support their preparation for final tests and state board exams.

Conclusion The nursing education fundamentals exam was found to be both reliable and valid, showing utility for nursing student preparation for summative exams. The exam is also useful as a self-assessment tool for nursing students to learn more about their orientation to tests that cover a wide variety of topics after a substantive amount of time has passed. References Anderson, L.W., Krathwohl, D.R., & Bloom, B.S. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Longman. Armstrong, P. (2015).

Bloom's Taxonomy. Nashville, TN: Center for Learning, Vanderbilt University. Retrieved from http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ Billings, D.M. & Halstead, J.A. (2009). Teaching in nursing: A guide for faculty (3rd ed.). St. Louis, MO: Sanders Elsevier. Bloom, B.S. (1969). Taxonomy of educational objectives: The classification of educational goals: Handbook I, Cognitive domain. New York: McKay. Fundamentals of Nursing Exams. RNPedia. Retreived from http://www.rnpedia.com/practice-exams/fundamentals-of-nursing-exams/fundamentals-nursing-practice-exam-4/ Appendix A: Nursing Education -- Fundamental Concepts* 1.

A sudden redness of the skin is known as: A Flush B Cyanosis C Jaundice D Pallor 2. The term gavage indicates: A Administration of a liquid feeding into the stomach B Visual examination of the stomach C Irrigation of the stomach with a solution DA surgical opening through the abdomen to the stomach 3. A patient states that he has difficulty sleeping in the hospital because of noise.

Which of the following would be an appropriate nursing action? A Administer a sedative at bedtime, as ordered by the physician B Ambulate the patient for 5 minutes before he retires C Give the patient a glass of warm milk before bedtime D Close the patient's door from 9pm to 7am 4. Which of the following nursing theorists dveloped a conceptual model based on the belief that all persons strive to achieve self-care? A Martha Rogers B Dorothea Orem C Florence Nightingale D Cister Callista Roy 5.

Which of the following nursing theorists is credited with developing a conceptual model specific to nursing, with man as the central focus? A Martha Rogers B Dorothea Orem C Florence Nightingale D Sister Callista Roy 6. Which of the following questions is most appropriate to ask when interviewing a potential candidate fo an RN position? A What was your last nursing experience? B Are you willing to do overtime on weekends? C How many children do you have? D Do you plan to get pregnant? 7.

If a patient is injured because a nurse acted in a wrongful manner, which party could be held liable along with the nurse? A The private attending physician B The nursing supervisor C The hospital D All of the above 8. Which of the following may be considered a patient's right? A The right to euthanasia B The right to refuse treatment C The right to ignore hospital regulations D The right to refuse to pay for what the patient considers to be inferior service. 9.

If a patient sues a nurse for malpractice, the patient must be able to prove: A Error, proximal cause, and lack of concern B Error, injury and proximal cause C Injury, error and assault D Proximal cause, negligence and nurse error 10. Which communication skills is most effective in dealing with covert communication? A Validation B Listening C Evaluation D Clarification 11. Which of the following qualities are relevant in documenting patient care? A Accuracy and conciseness B Thoroughness and currentness C Organization D All of the above 12.

The usual sequence for assessing the bowel is: A Right lower quadrant, right upper quadrant, left upper quadrant. left lower quadrant B Right lower lobe, right upper lobe, left upper lobe, left lower lobe C Right hypochondriac, left hypochondriac and umbilical regions D Rectum, pancreas, stomach and liver 13. The nurse should take a rectal temperature of a patient who has: A His arm in a cast B Nasal packing C External hemorrhoids D Gastrostomy feeding tubes 14. Blood.