Research methodology in educational psychology and testing measures

Educational Psychology: An Overview

The topic of educational psychology, or psychology as it is practiced within a school system, has become increasingly important in recent years as the number of special education demands on school systems and especially public school systems has increased. While educational psychologists tend to work in universities and other research settings rather than in the school system per se, they are continually affected by what is happening in schools as they seek to respond to the changing needs of student bodies. Thus the needs of school psychologists tend to be reflected at least in some measure in the research of educational psychologists, who in turn affect the practice of school psychologists. Thus one area of research that is becoming increasingly important to educational psychologists is one that is among the most pressing demands made on the school psychologist today. This topic is how to help autistic students benefit from the public education system without their needs diverting too many resources from general education classrooms and students. This paper examines current research on this topic while outlining a research methodology for a proposed future project in the area.

Educational psychology is an interdisciplinary field, for in addition to interacting with the practical concerns of school psychology (a field that is now developed within and maintained by school certification programs and so is attuned to the needs of school districts rather than research). It is based primarily in psychology, of course, but also draws on research from education, neurology, counseling, and organizational studies, among others. The research that educational psychologists perform, in turn, influences the work performed by many of the disciplines that provide direct support services in the school system. The research that educational psychologists perform is used to help design curricula, especially for special education programs; to help create new educational technology, especially assistive technology for students with a range of disabilities; and even classroom management strategies.

Educational psychologists are primarily concerned with the development of young people from young childhood through adolescence, although there are those who focus on adult education as well. They are generally concerned with stages of development, including social, emotional, and cognitive stages of maturation (Furth & Wachs, 1975). Educational psychologists also examine the ways in which moral development occurs as children age. All of these factors of development are inter-related and iterative, so that no one aspect of development can be understood without attending to the other aspects of development.

Before focusing on the specifics of this topic, it is important to limn the scope of the profession, which has expanded dramatically in recent years. A generation ago, and even a decade ago, the major task for the educational psychologist was to provide standard tests for schoolchildren and provide feedback on the results of these tests to administrators, teachers, and caregivers. This remains a central task for the school psychologist, but the scope of the job has changed because of the recent influx of a much larger special education student population. Of course, educational psychologists work with general education students as well and address far more broad-ranging concerns than simply testing and placement.

It is important to note that, while the job of the educational psychologist is in the process of adding new concerns and duties, it has also discarded tasks that were once assigned to this office. For example, Kraft (1970) discusses how it is vital that the educational psychologist help each student selected her or his course of study and then follow up with each student to ensure that such a study is the one that is most suited to that student.

It is hard to imagine that any educational psychologist today would be concerned with such a task. Not only do educational psychologists generally not interact with individual students today, but they are focused on more sophisticated areas of research rather than a more practical set of concerns such as career counseling. This is not to say that the tasks previously performed by educational psychologists were not valuable; rather, it is merely to comment on the fact that the role and function of the profession have shifted dramatically over time.

Just as educational psychologists have shed some of their previous foci, they have taken on others. For example, one area that educational psychologists have recently moved into, for example, is that of social justice. This might seem far afield from the traditional practice of the educational psychologist, but as Speight & Vera (2009) note, issues of access to appropriate education is in fact central to the concept of social justice, for education remains an essential aspect of long-term economic and emotional well-being in the United States as well as the possibility of class mobility. Specific issues such as the differential ways in which certain diagnoses are applied (such as the fact that the cohort that is most frequently diagnosed with ADHD is African-American boys) also touch on the core concepts of social justice. Consideration of such issues is an important aspect of educational praxis.

Theoretical Models for Educational Psychology

There are a number of different major theoretical schools within educational psychology. These, it should be noted, are not unique to this field but are used throughout the social and human sciences. This paper examines three of the primary theoretical models in this field that are most relevant to the writer. The first of these is applied behavior analysis. While this set of techniques is based on one of the first principles of psychology -- the kind of behavioral conditioning that was developed well over a century ago -- this particular version of behavioral analysis is an updated version that borrows significantly from the practice of cognitive behavioral therapy (commonly used as a form of therapy) and the current emphasis on evidence-based therapies.

Applied behavioral analysis is favored by a number of school psychologists, teachers, and school administrators because it often promises -- and sometimes delivers -- relatively quick results. This form of analysis is really more a technique than it is a theory since. It is based on the same kind of guidance that many teachers and parents already use on a consistent basis with children: Good behavior is rewarded while problematic behavior is not. (Note that problematic behavior is not punished but rather simply not rewarded.) There are theoretical elements to such a model, but the theory that there is quite thin on the ground. The psychologists who developed the models of operant conditioning were focused almost entirely on behavior. Modeling their own work on the hard sciences of biology and chemistry, they shied away from everything that could not be measured.

The result of such a model -- whether it be conceived of theoretically or in terms of praxis -- is that the focus must be on behavior. What goes on outside of the realm of observable, measurable behavior certainly existed (emotion, for example, as well as essential human attributes such as motivation and cognition), it was deemed peripheral and even irrelevant. Such a model has a real appeal for a teacher faced with a child whose behavior not only prevents him from benefiting from the classroom experience, but also prevents the other children in the classroom from gaining an education.

Thus many school programs, based in part on the work of educational psychologists, use various forms of applied behavioral analysis. Among the areas in which such programs are highly prevalent are the special education programs in which children with autism are enrolled. Because autism is -- for those around the person with autism -- defined primarily by out-of-control behavior, a program that limits such behavior is highly attractive.

However, there are distinct limitations to such an approach, and these derive from the same facet of applied behavioral analysis that makes its theoretical grounding so skeletal (Cameron, Pierce, Banko, & Gear, 2005). While human behavior is, of course, important, and especially so in the classroom environment, it is arguably much less important to understanding and re-directing human behavior than a deeper understanding of human cognition, motivation, and emotion. In other words, much of human life is subjective, and a method that focuses only on those aspects of human nature that can be easily objectified will be an extremely limited one, as this early advocate of applied behavioral analysis notes. That these concerns were already present in the earliest years of the popularity of applied behavioral analysis is striking. In fact, such a sensitivity may have been more present then than it is now (Alberto & Troutman, 2003, p. 29).

It is clear that a number of the most important concepts of our culture are subjective, perhaps even the most important. Martin Luther, as the story goes, was severely criticized for setting Protestant hymns to the popular melodies of songs and dances of the time. He replied, "Why should we let the devil have all the best tunes?" Well, why should we let the others have all of the best human goals and social problems? (Wolf, 1978, p. 210)

One of the problems that Wolf cites in both the theory and praxis of applied behavioral analysis (and its close cousin cognitive behavioral therapy) that has remained an issue in the intervening decades is that because this technique is focused on the observable, there is no way for either a clinician or a researcher to make an accurate assessment of the internal state of a subject.

Behavioral researchers have reported many examples of a lack of correspondence between client-reported data and observer-obtained data. Patterson (personal communication, 1974) for example, described discrepancies between parental reports of improvements in the child's behavior, while objective data obtained by observers did not support these claims. Conrad and Wincze (1976) reported that clients undergoing orgasmic reconditioning verbally reported favorable results that were not substantiated by the objective data. (Wolf, 1978, p. 212)

How significant a problem this is depends upon a researcher's (or clinician's) overall theoretical perspective and goals.

It is certainly true that teachers (or researchers in laboratory conditions) can significantly change the behavior of students to bring that behavior more in line with what the teacher (or researcher) wishes. However, without a continuous series of rewards, there is extinction of the desired behavior. Again, this may or may not be viewed as a problem depending on the circumstances. However, critics of this approach point to this aspect as being indicative of larger theoretical and even ethical issues in that any approach based on operant conditioning denies the complexity and urge toward self-determination of an individual (Lepper, Greene, & Nisbett, 1973; Cameron, Pierce, Banko, & Gear, 2005).

Applied behavioral analysis techniques have been shown to reduce the intrinsic motivation of individuals when it leads teachers and researchers to provide tangible rewards (Pierce & Cameron, 2002). However, in other versions of such techniques -- when rewards are given to children as they gradually but steadily increase their performance in a particular area -- intrinsic motivation does not seem to be as adversely affected (Cameron, Pierce, Banko, & Gear, 2005; Pierce & Cameron, 2002).

Pivotal Response Therapy

Pivotal response therapy is one of the preferred methods for treating children with autism spectrum disorders. It is derived from applied behavioral analysis, but it has become sufficiently recognized as a distinct mode of treatment that the author is examining it separately here. The key distinction between pivotal response therapy (in both theory and application) is that rather than focusing on a single behavior or type of behavior (such as screaming or hitting by an individual with autism), pivotal response theory focuses on a cluster of behaviors and motivations that educational psychologists believe are intrinsically linked to each other. Because these behaviors are linked on an a priori basis, researchers believe that there is a sort of contagion effect that can travel among each bundle of behaviors and beyond to affect a wide range of other behaviors as well.

A physical analogue to pivotal response behavior can be seen in an obese individual's losing weight. Weight loss is likely to trigger improvements in blood pressure, blood sugar, arthritis, kidney function, etc. Improvement in a key area in cognition, emotional control, and behavior -- according to the pivotal response model -- will similarly benefit a range of other areas for the individual with autism.

Pivotal Response Training (PRT) is a naturalistic behavioral intervention developed to facilitate stimulus and response generalization, increase spontaneity, reduce prompt dependency, and increase motivation while still relying on the principles of applied behavior analysis. PRT was designed based on a series of studies identifying important "pivotal" treatment components that affect a wide range of behaviors. Pivotal behaviors are those behaviors that are central to wide areas of a child's functioning, and when promoted, they are believed to produce improvement in many non-targeted behaviors. (Koegel & Schreibman, n.d.)

Koegel et al. (1999) found four pivotal behaviors: Motivation, self-management, self-initiation of behavior, and responsivity to multiple cues. As noted, this theoretical model assesses bundles of behavior as being the key level of assessment and treatment. This is a dramatic shift from other forms of behavioral analysis and treatment, which take a much more micro-level approach.

Koegel et al. (1999) found that when improvement is made in these three areas, children and adolescents with autism will experience improvement in overall function in areas that tend to be very detrimentally affected by autism, including the ability to generalize a new behavior or thought pattern from one specific to a broader range and a greater range of autonomy.

motivation, self-management, and child self-initiations. According to), when these pivotal behaviors are enhanced, improvement in autonomy, self-learning, and generalization of new skills will follow.

The educational psychologists that developed this model of intervention relied on an idea that is very simple, but that had been largely absent from the preceding models of behavioral analysis and modification: The idea that the interventions should be as naturalistic as possible. Rather than attempting to disrupt the behaviors of individuals with autism with cues that cut them off from their own subjective reality and understanding of their behaviors and motivations, PRT shifted the emphasis to a more intuitive approach:

PRT is a loosely structured, naturalistic intervention; in other words, it relies on naturally occurring teaching opportunities and naturally occurring consequences & #8230; Research has shown that naturalistic interventions are associated with more expressed positive affect on the part of the child as well as the parent implementing the treatment. PRT is child directed (as opposed to therapist or parent directed), giving children the opportunity to initiate learning events. & #8230; PRT works to increase motivation by including components such as child choice, turn-taking, reinforcing attempts and interspersing maintenance tasks. PRT has been used to target language skills, play skills and social behaviors in children with autism. These strategies are also used by some other naturalistic behavioral strategies such as milieu teaching and incidental teaching. (Koegel & Schreibman, n.d.)

The University of California at San Diego's Autism Intervention Research Program lists the following attributes of Pivotal Response Training:

1. The instruction should be clear, appropriate to the task, uninterrupted, and the child should be attending to the therapist or task.

2. The client can maintain previously mastered tasks.

3. Multiple cues should be presented as appropriate for the child's developmental level.

4. The child should be given a significant role in choosing the stimulus items.

5. Rewards should be immediate, contingent, uninterrupted, and effective.

6. Natural or direct reinforcers should be used the majority of the time.

7. Rewards should be contingent on response attempts. (Koegel & Schreibman, n.d.)

The above criteria demonstrate that PRT was a significant shift from applied behavioral analysis both in that it focuses on core bundles of behavior rather than immediate individual behaviors and in its emphasis on the subjective needs and perspectives of the client.

Self-Determination Theory

Both much of the research and the clinical interventions designed for individuals with autism assume that the key to improvements in behavior must be externally directed. Another way of describing this orientation is that the individual must be molded from the outside, which is true in large part because that individual is presumed not to have any internal motivation to change.

Educational psychologists who adhere to a self-determination theoretical stance reject this position. This might well seem counter-intuitive to anyone who has ever had any significant experience with an individual with autism. People who are diagnosed as having a condition somewhere along the autistic spectrum can seem from the outside to be entirely without motivation to change, and especially to be entirely without motivation to change their behaviors to meet the needs of their caregivers. Indeed, such a lack of understanding of the consequences of their actions and a lack of empathy for the consequences of their actions on others seems to be a key criterion of autism.

Researchers who focus on motivation within the perspective of self-determination reject such an idea, arguing instead that all humans are motivated toward growth and improvement, although this inclination towards growth can be significantly limited (to the point of being nearly derailed) by the social conditions in which the person is operating. For the child or adolescent with autism, these social conditions often include numerous adults constantly pressuring the client to act in ways that may not in any way accord with the internal needs of the client.

(This is not meant in any way to negate the difficulties faced by those who live and work with individuals with autism, whose behaviors can be so difficult as to be traumatizing. Rather, it is a theoretical point about the ways in which autism should be conceptualized and treated.)

The following summarizes the theoretical model of self-determination theory:

Human beings can be proactive and engaged or alternatively passive and alienated largely as a function of the social conditions in which they develop and function. Accordingly research guided by self-determination theory has focused on the social-contextual conditions that facilitate vs. forestall the natural processes of self-motivation and healthy psychological development. Specifically factors have been examined that enhance vs. undermine intrinsic motivation, self-regulation, and well-being. The findings have led to the postulate of three innate psychological needs: Competence, autonomy, and relatedness -- which, when satisfied yield enhanced self-motivation and mental health and when thwarted lead to diminished motivation and well-being also considered is the significance of these psychological needs and processes within domains such as health care education work sport religion and psychotherapy that is worthy of our most intense scientific investigation specifically social contexts catalyze both within and between-person differences in motivation and personal growth resulting in people being more self-motivated energized and integrated in some situations domains and cultures than in others. Research on the conditions that foster vs. undermine positive human potentials has both theoretical import and practical significance because it can contribute not only to formal knowledge of the causes of human behavior but also to the design of social environments that optimize people's development performance and well-being. Research guided by self-determination theory has had an ongoing concern with precisely these issues (Ryan & Deci, 2000, p. 68)