Testing Reliability and the Mitigation of Risk

Discussion 1: I have used many different types of assessments in my classes in the past. I would use student self-assessment, portfolio assessment, observational (informal) assessment, and formal assessment such as tests, quizzes and exams. I felt that the more varied your assessment methods, the more reflective of the student’s overall abilities the score at the end would be.

Student self-assessments allow students a chance to review their work on their own and judge their performance; it promotes active engagement with their work instead of passive engagement with whatever assessment the teacher gives. Portfolio assessments are good because they allow the student to gather the best work over a period of time and see how they are developing.

Observational assessment is good for developing a sense of the student’s skills in an informal way, the student’s participation level, the student’s methods of interaction and communication, etc. Formal assessment is good for measuring the student’s ability to problem solve in a specific way. As Peregoy and Boyle (2013) show, these assessments are effective and helpful in their own ways. The purpose of education is not just to pass a test or do well on an assessment.

Rather, the purpose of education is to grow and develop as a person. It is also important to recognize that learning should be rooted in Christ, as 2 John 1:9 says: “Everyone who goes on ahead and does not abide in the teaching of Christ, does not have God.

Whoever abides in the teaching has both the Father and the Son.” I believe it is important to stress to students that classroom assessments are but tiny reflections of the ultimate assessment that will be given at the end of one’s life, which is the final judgment. This can help reduce some of the stress that students feel about passing a test or performing well under pressure.

While knowing that these assessments are important, they are really not the most important thing in life: the most important thing is abiding in the teaching of the Lord, as John states. Thus, faith-based strategies can be used to assessments more effective by orienting students towards embracing the most important lesson and then aligning everything else within that framework.

As students learn to follow the Lord, they will also want to perform to the best of their abilities so that they can shine and use the talents and gifts given them by God to return glory to Him. Discussion 2: For a unit for grades 6-8, a set of 6 knowledge-level objectives would include Bloom’s knowledge levels: 1) remembering, 2) understanding, 3) applying, 4) analyzing, 5) evaluating, and 6) creating.

Remembering means to recognize and recall facts appropriately; understanding means comprehending what the data means; applying means taking the data and using it effectively in a problem-solving way; analyzing means to break down information into its component parts; evaluating means to gauge or measure the value or relevance of data; and creating means to take the parts and make something new. Following Bloom’s Taxonomy, the 1st learning objective for a unit for grades 6-8 in a reading class would be to memorize one Shakespeare sonnet.

The 2nd learning objective would be to communicate its meaning in a written essay. The 3rd learning objective would be to provide an example of how the ideas present in the sonnet could be applied in the real world. The 4th learning objective would be to identify the component parts of the sonnet in a separate essay. The 5th learning objective would be to determine the value or worth of the ideas in the sonnet.

The 6th learning objective would be to create a new sonnet using the ideas in identified earlier in the 4th learning objective.

Six objectives for grades 9-12 would be 1) to watch a performance of a Shakespeare play and detect non-verbal communication cues (perception); 2) to explain the process by which the plot is developed; 3) to copy the formula used in the play to produce an outline for a new play; 4) to assemble the parts (on paper) for the play to be produced; 5) to respond to a prompt given by the teacher in which a series of challenges are presented on opening night of the play and the student, as director, must overcome them; 6) to create a new set of recommended practices for a young director.

The rationale for using these objectives is that each challenges the student to engage with the material and lessons presented him or her. They enhance the teaching/learning experience by providing real world situations in which the student can apply the knowledge and higher learning obtained through the exercises in a manner that is consistent with real world application. As Bardovi-Harlig and Mahan-Taylor (2003) point out, teaching should have a pragmatic base to it so that students see the point and relevance of the exercises in their own lives.

These objectives supply that point. Discussion 3: The advantages of Mr. Williams’ approach in Scenario 1 is that it challenges the students to focus and prepare for the weekly tests. This will help to keep them engaged with the material so that they can pass. The disadvantage of the approach in the scenario is that the “A” for the most improved student is not necessarily reflective of actual improvement.

For instance, a highly talented and skilled student could “throw” his initial tests, knowing full well that he will show marked improvement later on, with the hope of getting the easy “A” at the end. Another disadvantage is that this is only one assessment method and teachers should use multiple assessment methods in order to ensure that the student’s acquisition of learning is actually taking place. One method is not necessarily reflective of the reality of acquisition levels, as Seraphine (1995) has shown.

The approximate weights that should be placed on Mr. Williams’ system should really only be about a quarter, with three other assessments being used to weight the students’ scores. The advantages of Mr. Bell’s approach in Scenario 2 are that he uses multiple assessments, which are more reflective of the overall ability of a student. He also recognizes competition as a motivating factor, which can indeed help students to focus more and engage more fully with their studies. However, the disadvantage of Mr.

Bell’s approach is that the type of competition he uses is not direct but rather indirect. Instead of students competing with one another directly in, for instance, board races, they are competing indirectly via the grading scale. This is not a very effective method because it can produce rather negative effects, particularly where a student’s self-esteem is concerned.

More direct competition is immediate and in the moment and the negative consequences are less likely to be manifested because losses are short-lived, whereas the shame of falling behind for everyone to see can be felt for a while. Mr. Bell should not use this grading scale method of competition but should instead use a fourth assessment—competition at the board—and make the assessments each worth a quarter of the grade.

This would help to create a more positive environment in the classroom and more than likely inspire students to be more actively engaged in their students because of the immediacy of the competition. Discussion 4: Test score reliability is impacted by group variability, scoring reliability, test length and item difficulty. Each of these variables should be considered when designing a test and the risks posed by each can be mitigated by following certain steps.

Group variability can be mitigated by formatting the test to meet the specific make-up of the group taking it. For instance, if a group consists of all ELLs, then giving a test that is designed for native English students might seem inappropriate. Formatting a test is helpful in reducing the impact of group variability. Knowing what the test is supposed to measure can help reduce the risk of scoring reliability.

If too much weight is given to superfluous items that are not really designed to measure the acquisition of knowledge provided in the class, then the test can be unreliable. The test should be designed to measure the student’s mastery of knowledge and skills taught in the class. If the skills have not been taught in the class, an assessment that measures for those skills is inappropriate. Test length can be mitigated by ensuring that the test is neither too long nor too short.

Tests can also be given in a range of lengths throughout a class so that students can become comfortable with various lengths of assessments. However, a good test.