Fairness and accessibility in design and policy

Fairness

When data collection is reliant on teacher reporting and therefore the "...perception and rating of the Kindergarten teacher" (Guhn, Guderman, & Zumbo, 2007, p. 456), how would you suggest best controlling for or adjusting for perception bias?

Teacher reporting is understandably used in studies like that of Guhn, Guderman & Zumbo (2007). In the Guhn, Gederman & Zumbo (2007) study, the Kindergarten teachers rated their own students on the EDI. It is difficult to conceptualize other means by which the students could be evaluated. Researchers could use third-party observers, such as Kindergarten teachers from other schools. This might reduce perception bias in that teachers are somewhat likely to develop personal likes or dislikes during the course of instructing their students. A teacher from another school who has not been in contact with the students might have less personal bias toward the population being measured, but would be lacking in the ability to evaluate the students on the EDL survey parameters. Therefore, it would be preferable to retain the teacher reporting but try and adjust for perception bias in some way. One method would be to supplement the teacher reporting with a third party objective observer. This would create a burdensomely large data pool, but might help illuminate the presence of bias and therefore control for it. Another method would be to employ a longer-term period of data gathering or even a longitudinal research design. Tracking teacher reporting over time might do away with temporary biases or subjective perceptions, ruling out nuances of perception, cognition, and emotion.

2. Based on the Guhn, Guderman and Zumbo (2007), Oliveri, Ercikan and Zumbo (2013) and the text (and perhaps your experience at Tuesday's dissertation proposal defense) are there alternative considerations or ways you would suggest to provide a for fair assessment of pre-k children (or any children) from diverse populations?

Guhn, Guderman and Zumbo (2007), Oliveri, Ercikan and Zumbo (2013), and readings from the text all illuminate some of the ways diverse classrooms function. Early childhood educators face a multitude of issues during the course of their work. One of the greatest challenges to early childhood educators is assessment, and finding the most appropriate, least biased, and least invasive methods of assessing students from diverse populations. Assessments should certainly be comprehensive and refer to the whole child, as Guhn, Guderman and Zumbo (2007), Oliveri, Ercikan and Zumbo (2013), and readings from the text all show. Behavioral issues need to be taken into account, as to issues related to social learning and emotional maturity. Yet concrete learning tasks, ranging from language development to specific knowledge, also need to be measured in ways that are sensitive to different learning styles. One way to ensure a fair assessment is simply to use multiple methods of assessment for each child, and to assess for as many factors as possible to avoid overlooking key areas of strength or weakness. Moreover, administrators or coworkers can monitor assessments or perform independent ones to correct for and address potential biases related to gender or ethnicity. Ideally, observing children in a naturalistic setting over a long period of time would help provide a comprehensive picture of their development. Video recordings might also come in handy for review purposes.

3. Does the prospect and understanding of differential item functioning (DIF) change your perception of reported scores and rankings on tests such as the PISA? Explain how DIF may have changed your perception or did not and why?