Question answering and comprehension strategies

¶ … limitations of an evaluation are that the evaluation is biased in one way or another. Researchers guard against this by using reliability estimators. Each of the possible four reliability estimators has certain advantages and disadvantages. For example, inter-rater reliability is one of the better methods to estimate reliability when measuring an observation. For an evaluation it is better to use the reliability estimator of test-retest. This reliability estimator can be administered to the same test or a similar sample from different occasions. This approach assumes that there is no substantial change in the construct being measured between the two occasions. The amount of time allowed between measures is obviously critical. Measuring the same thing twice so that the correlations between the two observations are compatible depends partly on elapsed time between measurement occasions. The shorter the time gap, the higher the correlation; the longer the time gap, the lower the correlation.

The difference between probability and non-probability sampling is that non-probability sampling does not involve random selection and probability sampling does. Probabilistic or random is used more appropriately than the other because with non-probability samples, they may or may not represent the overall population well and it is difficult to know if it was represented. So, probabilistic methods are better to use than non-probabilistic.

The difference between systematic and random errors in measurement can be reduced to what causes the errors. Systematic errors are caused by all factors that systematically affect measurement of the variable across an entire sample. Random errors are caused by all factors that randomly affect measurement of the variable across the sample. An example of a systematic error would be if there a bomb blows up outside of a classroom where a class is taking a test, this noise and distractions affect all of the children's test scores by systematically lowering them because of the distraction. In the case of random error would be only two or three children have the sun brightly reflecting on their papers making concentration difficult.

The measures that can be taken to ensure measurement validity and reliability are reducing measurement errors, random or systematic. Techniques that work are:

pilot testing all external instruments of methods getting feedback on how easy or hard a measure was getting feedback about how the testing environment affected performance when gathering measures using people to collect data as either interviewers or observers, make sure they are trained thoroughly to avoid introducing errors double-check the data thoroughly