Attribute Hierarchy Method for SAT Diagnostic Inferences
This paper critiques Gierl, Wang, and Zhou's (2008) study, which applied the attribute hierarchy method (AHM) to diagnose examinees' cognitive skills in SAT algebra. The critique examines the study's purpose, methodology, cognitive development assessment framework, and limitations. The AHM classifies student responses into structured attribute patterns derived from a cognitive model of task performance, offering a more nuanced alternative to simple correct-versus-incorrect scoring. The paper evaluates the study's four-step model construction process, its use of think-aloud protocols and psychometric analysis with 5,000 examinees, and ultimately questions whether the study's narrow scope, retroactive design, and limited generalizability undermine its ambitious research goals.
- Introduction: SAT's impact and need for fair question analysis
- Purpose and Background of the Study: AHM defined and study objectives explained
- Literature Review and Cognitive Development Assessment: CDA contrasted with conventional performance assessment
- Research Methodology and Model Construction: Two-step and four-step model procedures described
- Scope and Limitations: Narrow scope, retroactive design, and attribute issues
- Overall Critique and Conclusion: Ambition exceeds findings; generalizability questioned
✍️ How to write this paper — guide, tools & examples ▾
What makes this paper effective
- The critique follows the structure of the source article closely, moving logically from purpose and background through methodology to evaluation, making the argument easy to track.
- Frequent direct quotation from the source article grounds each analytical point in specific textual evidence rather than paraphrase alone.
- The concluding critique is appropriately measured — acknowledging the study's intriguing elements while clearly identifying weaknesses such as narrow scope, retroactive design, and limited generalizability.
Key academic technique demonstrated
The paper demonstrates disciplined source integration: it uses block quotations with precise page references to substantiate descriptive claims, then transitions to evaluative commentary. This technique — summarize, cite, evaluate — is the foundational move of an academic article critique and is executed consistently throughout.
Structure breakdown
The paper opens by establishing the significance of SAT testing, then introduces the AHM framework and its purpose. It moves through the source study's literature review rationale, two-step methodology, and four-step model construction before narrowing to admitted limitations. The final paragraph delivers an evaluative judgment covering scope, generalizability, and design feasibility, giving the critique a clear arc from description to assessment.
Introduction
One problem with evaluating the effectiveness of different types of test questions is that it is often unclear why students get particular exam questions wrong — or right. The SAT is a particularly controversial and challenging test that can have a long-lasting impact on a college applicant's life, depending on the score received. Thus, effective analysis of SAT questions for validity is essential to ensure fairness for the high school students who take the test.
Purpose and Background of the Study
The purpose of the study by Gierl, Wang, and Zhou (2008), entitled "Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees' Cognitive Skills in Algebra on the SAT," was to provide greater clarity about the test-taking strategies used by various students on the SAT, going beyond a simple correct-versus-incorrect perspective. According to the authors, the attribute hierarchy method (AHM) is "a psychometric method for classifying examinees' test item responses into a set of structured attribute patterns associated with different components from a cognitive model of task performance" (Gierl, Wang & Zhou, 2008, p. 4).
The authors' ultimate objective was to provide a more effective methodology for preparing students as well as evaluating potential test questions — not simply on the SAT, but across various other types of standardized examinations. As they explain, "cognitive models provide one method for representing and reporting the examinees' cognitive profile on diverse tasks, which could be used to link their weaknesses with instructional methods designed to improve the examinees' skills" (Gierl, Wang & Zhou, 2008, p. 6).
Create your account
Always verify citation format against your institution’s current style guide requirements.