¶ … shorten tests while still providing reliable information about student performance. It can also be used to clarify or improve questions students will be tested on in the future as well as to eliminate questions which are not reflective of students' real abilities. An item analysis is conducted after-the-fact, i.e., after a test is given and allows the teacher to improve and redesign the test based upon the feedback received from student responses. With the rise in the use of test analysis technology, now teachers in classrooms as well as professional test designers can use the method to improve tests. A typical score report offers data such as the average or mean response as well as the standard deviation from that average ("Understanding item analysis reports," 2015). Item difficulty is also assessed as well as the test's ability to determine how well the students understood the material that was being tested. A test with a high level of internal consistency in this area will be both more reliable and valid than a test that is not. Ideally, the difficulty level of a specific item should be slightly greater than the midpoint to eliminate the chance of random guessing resulting in a correct response ("Understanding item analysis reports," 2015).
A key concept behind item analysis is that of item discrimination: the extent to which a response to the item correlates with a high or low score on the test. For example, a difficult test question might have a high correlation of correct answers for students with high overall marks on the test and a correlation of incorrect answers for students with low marks. This would suggest an effective test question vs. A question with a relatively random series of answers (McDonald 2013: 231). Conversely, test questions which appeared to stump...
The least effective test takers would be problematic in terms of its efficacy in testing ability.
Arguments in favor of using item analysis
Testing time is finite and item analysis allows tests to be shorter and more carefully designed to reflect teacher's and district's needs. Teachers can also engage in classification of items to ensure a wide range of student needs and abilities are assessed, such as using Bloom's Taxonomy to rate various questions based upon the types of higher-level thinking required to answer the question ("Item analysis," 2015). Test answers which appeared to only be answered by the most sophisticated thinkers in the class might highlight potential skills deficits in the student population as a whole as well as problems with the test.
Teachers often use the same tests from year to year but testing can be -- and should always be -- a work in progress. Test items must constantly be screened for confusing wording which does not test the desired content area; for bias against a specific population (such as race or gender); and if the test answer is unclear (or too clear) in nudging the reader in the direction of a particular answer (Krishnan 2013: 7).
There are also a number of useful, peer-reviewed techniques to use to screen for potential biases and other problems with item analysis, including Classical Measurement Theory (CMT) or Classical Test Theory (CTT) versus Item Response Theory (IRT) otherwise known as Rasch. Theory (Krishnan 2013:2). CTT uses smaller sampling sizes and because it is sample-dependent, results are not generalizable; IRT estimates can be used to assess the overall accuracy for items for test-takers of different levels of ability. In other words, a test…
