Research methodologies for assessing phonics awareness literacy screening effectiveness

¶ … employed by a researcher can positively or negatively affect the outcome of research as well as perceived applicability or usefulness of a study. Thus it is vital that the researcher adopt a research methodology that is relevant to the phenomena being examined, and the methodology that will provide the most comprehensive explanation for predicted patterns or observed behaviors.

There are two primary forms of research methodology employed by researchers engaged in a quest to uncover information about a given phenomena: qualitative and quantitative. There is a large body of evidence available that supports both forms of methodology for acquiring insight and information regarding given behaviors and experiences.

In some situations, both a qualitative and quantitative approach of research methodology have been adopted, though by and large most people in the research community agree that it is vital that a researcher employ the use of one or another research methodology, and that selection should be based largely upon their research goals and their intent.

For purposes of this study, the researcher is attempting to discern whether a qualitative or quantitative approach to research methodology is appropriate for examining PALS. PALS measures young children's knowledge of important literacy fundamentals, including phonological awareness, alphabet knowledge, knowledge of letter sounds, spelling concept of word, word recognition in isolation, and oral passage reading.

The major purpose of PALS is to identify those students who are below grade-level expectations in these areas and may be in need of additional reading instruction. The primary emphasis of the research will be an approach to determining the "what" "how" and "who" with regard to literacy problems in the classroom.

Meaning, the research will focus on determining what patterns are occurring in the classroom, how they are occurring and who they are occurring to. Bloland (1992) suggests that such relationships are best measured via a qualitative approach, which allows interpretive analysis of a given phenomena in a complex setting such as an educational environment.

There is also however, much support for a quantitative approach, which generally attempts to measure "why" something is occurring (Bloland, 1992). Within the field of education, an emphasis of late has been placed on conducting quantitative analysis that is analysis that incorporates the use of statistical analysis, fact collecting and data collecting to derive meaning or distinguish between cause and effect relationships (Lee & Poynton, 2000).

In fact, there are many that might argue that a quantitative approach is preferable to a qualitative approach in all situations regardless of the phenomena being studied (Myburgh, Poggenpoel & Van Der Linde, 2001) because it lends itself to less 'subjective analysis.' However further exploration into the matter suggests that qualitative analysis is not subjective in nature and may in fact be the preferred method for research analysis into complex behavioral issues, particularly when the researcher is attempting to determine patterns in behavior and conduct (Myburgh, et. al, 2001).

That there is a great deal of 'opinion' surrounding the issue of both qualitative and quantitative research methodology cannot be argued. Thus research will be examined from both perspectives, qualitative and quantitative, in order to ascertain which methodology is best suited to provide detailed analysis of the PALS program, and in order to determine which methodology should be adopted by the researcher to provide adequate insight into the given phenomena that the researcher is attempting to explore. No conclusions will be made with regard to the legitimacy of either research approach until the final portion of this analysis, where a determination will be made as to whether a qualitative or quantitative approach is more appropriate in this particularly setting.

History of Qualitative-Quantitative Research

Both qualitative and quantitative research according to Benz & Newman (1998) have foundations in philosophical roots and natural philosophies. Throughout time qualitative and quantitative research methods have been used successfully by researchers to explain patterns in behavior, given phenomena and to acquire a greater understanding of human behaviors. Over time one research methodology is often favored over another. Despite this both have been used almost equally with regard to frequency to examine sociological phenomena.

Though there are those that would argue in support of one method vs. another, a researcher would be hard pressed to prove that one method is ultimately better than the other; in fact there are those that would argue that the two should not be used independently from one another, but rather in conjunction with one another to provide adequate insight and understanding of social phenomena in complex settings (CSU, 2004).

Qualitative researchers vary with regard to theory, however a majority have reflected on individual phenomenological perspectives; quantitative researcher approaches however, no matter the theoretical foundation, have throughout time emphasized discovery of a common reality "upon which people can agree" (Benz & Newman, 1998:2). This is not to say that qualitative approaches are not universally agreed upon. There is however, a generally acknowledgement within the scientific community that qualitative research often lends itself to more interpretation or a wide range of interpretation from many different perspectives (Benz & Newman, 1998).

Traditionally the qualitative researcher will employ the use of a variety of techniques that can be reconstructed, much as is the case with quantitative methods. However the manner in which the qualitative researcher interprets the results of the information gathered may be more subjective in nature than the manner in which quantitative data is interpreted (Benz & Newman, 1998).

Reality in and of itself may be considered a social construct, and how one determines the best method for conducting and interpreting a study and the conclusions that are subsequently drawn are considerably different when one uses a quantitative philosophy, which assumes that "a common objective reality" is tangible for all individuals (Benz & Newman, 1998). However, there are different sets of belief and assumptions that are associated with both forms of research.

Qualitative and quantitative researchers have often debated based on their perspectives of what aspects of reality are measurable vs. which are not (Blumer, 1980; Douglas 1976; Benz & Newman, 1998). For example, Benz & Newman (1998) cite William Firestone (1987) who states that qualitative and quantitative research can be distinguished based on four identifying dimensions which are: assumptions, purpose, approach and the research role (p. 2).

The key factor to determine with regard to research, according to Firestone (1987), is whether reality is sought through facts (quantitative research) or whether reality is socially constructed (more qualitative in nature). However, despite this declaration it is vital to note that qualitative research and supporters of this form of research have argued successfully that qualitative research is just as likely to provide information that is reality based and factual in nature (Myburgh, et. al, 2001).

Qualitative information is simply more likely according to supporters, to provide information from a given or directed point-of-view, and it may be more narrowly focused, however it is also more subject to interpretation, and lends itself to more creative analysis and discourse about a given phenomena in some cases than quantitative research (Myburgh, et. al, 2001).

An experimental or correlational form of research is typically utilized when embarking on fact seeking or quantitative research, whereas more theoretical approaches are typically assumed for purposes of qualitative research (Shaker, 1990). Shaker (1990) supports a "naturalistic" or qualitative approach to interpret reality when the aim is developing theories to express a phenomena that is experienced, whereas a qualitative approach should be utilized more when the researcher has developed a certain hypothesis that requires that the researcher conduct experiments or tests to determine whether or not a given phenomena can be explained statistically.

Qualitative vs. Quantitative Research great debate exists with regard to qualitative and quantitative research. Since the dawn of time researchers have attempted to determine whether one method is better than the other. This is particularly the case among educators and educational research authorities, who at various times have argued in favor of one form of research vs. another. From a fact-based or 'scientific' perspective, logically one can only argue that the methods employed by the researcher must depend entirely upon the aims of research; that is if the researcher intends to provide statistical information and a numbers based approach to analysis, a quantitative approach is the most well suited to that particular environment (Potter, 1996).

If however, the intent of the researcher is to gather information about a given phenomena, with the intent of providing interpretive analysis of how something is occurring and the frequency with which it is occurring a qualitative approach may be warranted (Mills, 2000).

Likewise if the researcher is attempting to provide information about phenomena without determining a causal relationship, then qualitative research is certainly warranted. Historically there are many instances where a qualitative approach is much more suitable than a quantitative, as a causal relationship is not always the primary focus of research endeavors.

The type of research methodology employed might also be contingent upon the field being studied or the experiences of the researcher. There are those within the scientific community that always promote the use of quantitative analysis, whereas individuals engaged in the examination of sociological phenomena from a historical or cultural context might always employ the use of qualitative methods (Mills, 2000).

There are those that believe that qualitative research is the best form of research, whereas others insist that only quantitative methods are appropriate in a research environment (CSU, 2004). Still others argue that both approaches are useful and appropriate though one is often more indicated than the other depending on the exact phenomena being examined and the nature or intent of the research being conducted (Potter, 1996; Lee & Poynton, 2000).

Fred Kerlinger once exclaimed that "there is no such thing as qualitative data, everything is either one or zero," however his claim is countered by another researcher, Campbell, who asserted that "all research ultimately has a qualitative grounding" (CSU, 2004).

Given the great debate that exists, researchers often find it difficult to determine which strategy is best and which is most likely to be accepted by peers when presenting a research program. Most researchers would aggress however that qualitative and quantitative data and research methods are equally important (Myburgh, et. al, 2000; CSU, 2004).

Qualitative research methods more often employs the use of words and interpretive analysis, whereas quantitative data analysis involves the use of numbers and statistics (CSU, 2004). There are some researchers that feel that quantitative is better because it is more 'scientific,' or at least they assume so because quantitative research requires the use of numbers. However, the major difference between the two is that qualitative research is more inductive in nature and quantitative research is more deductive in nature (CSU, 2004).

Because quantitative research is deductive, it requires that the research develop a hypothesis prior to conducting any research. Quantitative researcher also assumes that the researcher is an "objective observer" that is capable of neither participating in the study nor influencing the phenomena being studied (CSU, 2004)

This is not necessarily the case in qualitative research, where the researcher is able to learn about a situation by being immersed in it (CSU, 2004). Both approaches have benefits and drawbacks. The most obvious drawback or limitation of quantitative research is that it often focuses too closely on "individual results" and therefore fails to "make connections to larger situations or possible causes of results" (CSU, 2004). Qualitative research on the other hand, is sometimes considered to subjective or interpretive in nature, and is criticized for not being more 'scientifically' grounded because it is less likely to produce numbers or generate statistical analysis of a given situation.

In some studies, a combined approach may be indicated. For example, a quantitative approach may be established whereby the researcher develops an experiment to test a given hypothesis. Once that experiment is carried out, it is feasible that the researcher might then also include the use of qualitative approaches to supplement the findings or provide observational analysis via interviews and subject interaction to supplement the information acquired from the experimentation (CSU, 2004).

Most researchers however, agree that more problems are likely to result when a combined approach is used, and a greater margin of error might be afforded in situations where both approaches are used. A better approach would be the use of qualitative studies to gather initial information about a given phenomena, and later development of quantitative analysis or experimental research to gather additional information or to determine the cause and effect of patterns uncovered during a qualitative approach to research (Myburgh, et. al, 2001). There is general agreement among researchers that qualitative approaches to research may be considered a suitable prerequisite to future quantitative studies. In addition qualitative research is considered a logical starting point when little information is available about a given phenomena to begin with. The information gathered from this type of study may provide guidelines for future quantitative analysis (Myburgh, et. al, 2001).

Education and Literacy

With regard to education and literary competency, as far back as 1844 during the time of August Comte researchers argued that methods of "naturalistic approach" to research could be utilized to collect and study facts and observe trends and patterns through sensory input (Benz & Newman, 1998; Vidich & Lyman, 1994). Thus phenomena can be studied and observed and accounted fro more using qualitative means rather than via quantitative methods.

However, these views have changed, and in fact through the 1960s a more quantitative approach to examining educational research predominated among behaviorists and organizational theorists who determined that empirical fact gathering and 'hypothesis testing" were critical elements for studying educational and social phenomena alike (Benz & Newman, 1998: 5).

In recent years however once again a qualitative approach has become popular, emphasizing the need to examine each phenomena on a case by case basis in order to determine the best research methodology for uncovering facts or patterns related to any given phenomena (Benz & Newman, 1998).

Cohen, Gass & Tarone (1994) suggest that the validity of any discipline including literacy and educational competency is "predicted upon the assumption that the research methods used to gather data are sufficiently understood and agreed upon" (p. xii). There are many research issues that relate to basic questions of competence vs. performance in a classroom setting, as well as issued related to quantitative vs. qualitative research methodology, including whether or not research methodology should be based upon statistical analysis or a more theory based approach (Cohen, Gass & Tarone, 1994).

Literacy and language acquisition draws on many fields including linguistics and child language, and researchers working within this area are traditionally concerned with issues related to reliability and validity (Choen, et. al, 1994). Research should be focused on one of two traditions, (1) whether or not the goal is to explain and predict how natural phenomena work or (2) to understand and interpret the manner in which phenomena might be organized (Cohen, et. al, 1994). The first assert that a single, "discoverable reality" exists that may be explained by laws of nature and an approach that is cause-effect related (Cohen, et. al, 1991). This supports use of a quantitative methodology which is capable of showing causal relationships (Cohen, et. al, 1991: 324). This suggests that a qualitative approach serves primarily the approach of exploring and identifying issues rather than proving causal relationships. However, supporters of the qualitative approach assume that there are multiple realities that exist that might explain a given phenomena, and under this assumption is the notion that human events can't simply be interpreted in terms of contexts and uses, or in an idealized form, but must be interpreted individually to varying degrees (Cohen, 1994).

Data that explains information from a learner's point-of-view or self reported data, via use of interviews and direct observation, are valid methods of providing an ethnography, or information about learner behavior, and typically encompass a qualitative approach to research methodology (Cohen, 1994). In some cases this approach is clearly preferable, because seeing out the learner's point-of-view will provide a better understanding of the phenomena being examined, which may include literacy acquisition. As Cohen et. al (1994) there are limitations and benefits to both approaches to review.

Lee & Poynton (2000) point out that in the last several decades more and more attention has been given the significance of language acquisition and problems associated with it. The researchers affirm that research tools can provide alternative approaches to understanding given phenomena. Further Lee & Poynton (2000) point out that with regard to educational research typically a conflict exists. While researchers attempt to explain phenomena as a function of individual behaviors and have traditionally intended to understand the "complexities around the construction of knowledge and practices in the context of the changing culture" via a qualitative approach, there is pressure from funding and governmental authorities to provide linear and rational descriptions of phenomena including communication and linguistic skills necessary for efficient work and training, suggesting an emphasis toward quantitative analysis (p. 117).

Most 'funding' authorities view research as a task that should be structured around product-oriented activity, meaning that it should aim at identifying relevant competencies and relevant 'facts' related to given phenomena (Lee & Poynton, 2000).

Cross, David, Graham & Thralls (1996) suggest that most people conceptualize methodology as a literal phenomena, related to concepts and 'procedural apparatus' utilized by researchers in order to gather an analyze information. This definition however, according to the researchers, is too narrow, because as they point out it overlooks the fact that "methodologies exist within the larger framework of research narratives - stories or discourses - that shape the way disciplines make knowledge" (Cross et. al, 1996: 131). Further quantitative methodology is typically utilized particularly with regard to business environments where a need exists for the researcher to apply statistical tests and analysis to gather information and results (Cross, et. al, 1996). In other situations, a qualitative methodology is a valid approach when the researcher is attempting to utilize interpretation; the quantitative approach therefore requires that the researcher conduct the interpretative step and analysis step, and often readers are "lulled into accepting statistical conclusions as valid because they supply numbers" (Cross, et. al, 1996).

Qualitative research is sometimes viewed by the public in a more 'skeptical' light because it does not provide the numbers or concrete foundational analysis that quantitative research does. However, a qualitative approach can provide many benefits. Because the researcher would be working in an "interpretist" manner, they have the ability to provide generalizable information that can be applied more universally, and is open to alternative 'reconstructions' due to the "provisional and fluid" nature of the research presented (Cross, et. al, 1996: 105).

Qualitative Approach

The qualitative approach adopts certain methodologies including: "ethnography, ethno methodology, reception studies, ecological psychology, symbolic interactionism, cultural studies and textual analysis" (Potter, 1996: 50). These dominant methodologies are utilized by many theorists and varied enough to be considered strategically different.

Ethnography is focused particularly on exploring exactly how communities are created "and held together with human interactions" (Potter, 1996:51). The focus is traditionally on long-term observation of behavior and may be used to examine religion or the arts or other aspects of culture (Potter, 1996). The data provided via this method of research are often open to many different forms of interpretation and review. It is also often referred to as the anthropological approach, drawing on philosophical and sociological ideas and phenomenology (Potter, 1996).

Ethno methodology focuses on how people "make sense of everyday life" and occurrences, and attempts to identify and analyze the methods that people use to construct practical explanation for common events (Potter, 1996). A person examining phenomena from this point-of-view will examine a subject's knowledge and idealizations as well as environment to determine how behavior is affected or influenced. This approach is also referred to as a common sense approach (Potter, 1996).

Reception studies focus on "how readers of texts construct meaning from an activity," and the research believes the key to explaining the audience's behavior is focusing on the context of the individual's interpretation of some event or text, rather than the information they are reading (Potter, 1996). The researcher attempts to examine how the ideas for example, in a text are "brought to life" in a reader's mind (Potter, 1996: 54). The foundation for this methodology is based in literary theory and criticism, and is very similar to ethnography because it focuses on "people as members of interpretive communities" (Potter, 1996: 55). A researcher employing this form of methodology must examine the reader and the text.

Ecological psychology is a form of methodology that "focuses on culturally patterned behavior" and incorporated use of biological constructs to determine the manner in which the environment might affect behavior, suggesting that behavior has both objective and subjective components (Potter, 1996). The goal of this type of methodology would be among other things to provide detailed and objective descriptions of "naturally occurring behavior" that are "amenable to quantitative analysis" (Potter, 1996: 57).

Symbolic interactionism focuses on "meaning in social settings" or how people assume the perspectives of others and learn the meaning of things using symbols and human interactions (Potter, 1996). This type of research might focus on a history of experiences. A researcher adopting this form of qualitative methodology would necessarily examine symbols and behaviors, and must assume the perspectives adopted by the subjects being examined (Potter, 1996).

Cultural qualitative methodology simply examines how a person would interpret their culture and how they would interact in their culture via the use of symbols and creation of symbols (Potter, 1996). It is primarily indicated for purposes of sociological or anthropological investigations of social phenomena, and may be used in an attempt to interpret individual behaviors, attitudes or personalities in a given social setting or context.

Textual analysis focuses on examination of texts and literary review (Potter, 1996). It may be used primarily to derive information regarding previous literary works, and by and large does not seem appropriate to examination of educational programs in general. However it may be suitable for examination of historical phenomena or literary difficulties directly related to textual factors.

The latter of these two approaches are less concerned with the field of education and examination of literary competence and trouble, thus they are not discussed or examined in great detail.

Qualitative research is considered beneficial because it can provide information about broad ranging patterns and behaviors. When a phenomena is observed as occurring among a specific group of people, in a complex setting, or over a long or extended period of time, qualitative research is often indicated as it can provide insight as to how something is occurring, who it is occurring to and what exactly is occurring (QRCA, 2003). It is not as concerned with examining why exactly something is occurring, but may open the door for future researcher that is more cause and effect oriented and that may answer this specific question.

However, the results of qualitative research cannot be "statistically projected" across any target population and typically qualitative research cannot be quantified (QRCA, 2003). Thus researchers often tend to go with a quantitative approach if they are attempting to generalize information or statistical findings across a larger group of people or a large variety of situations. Quantitative approaches generally lend themselves to less variance and less alternative interpretations, and are rather static in nature, whereas qualitative approaches are generally considered more flexible and more subjective in nature.

Traditionally qualitative research is a first approach used in research methodology in order to provide information for planning future quantitative studies or as a method employed to help interpret the results of a quantitative research or provide additional information on statistically relevant topics (QRCA, 2003).

Quantitative Methods

Qualitative studies are more often than not considered the pre-requisite for quantitative studies (Myburgh, Poggenpoel & Van Der Linde, 2001). Quantitative research is based on "positivism which takes scientific explanation to be nomothetic" meaning based on universal laws (Myburgh, et. al, 2001).

In addition it aims at objectively measuring phenomena that occur within the social world, and testing hypothesis to predict, analyze, alter or control human behavior (Myburgh, 2001; Schrunk, 1998). Qualitative is more interpretive in nature, with the main aim being an understanding of social life and meaning people attach to that life (Schrunk, 1998).

In quantitative research, certain steps are embarked on which include (1) observations of natural phenomena, (2) development of conclusions with regard to natural phenomena, (3) development of hypothesis or predictions pertaining to the "causal relationship between certain observations" (4) testing of the hypothesis, (5) attempts to develop theories that explain why a phenomena is occurring after a spiral of scientific investigation (Myburgh, et. al, 2001: 408).

There is within the research community a tendency to "over-emphasize" the quantitative approach "regardless of the demands of the phenomena explored" thus it is vital for the researcher to spend an adequate amount of time determining whether or not the quantitative approach is truly the best suited for the actual research being conducted (Myburgh, et. al, 2001).

A quantitative approach can result in a tendency toward using statistics and hypothesis testing to dictate research processes rather than using the research problem and actual phenomena being researched as a guideline for analysis and interpretation of behaviors (Myburgh, et. al, 2001). Such an approach may reduce research to a mechanical implementation of measurements and "statistical testing" (Myburgh, et. al, 2001: 408).

Then again there is still evidence to suggest that the quantitative or experimental research approach lends itself to more knowledge gathering (Myburgh, et. al, 2001).

Myburgh, et. al. (2001) cites Sax (1968) who points out that all scientific research and inquiry can be defined "as consisting of the following characteristics: reduction and control of bias, the quest for precision, verification, empiricism, and theory construction" (p. 408). NO matter the form of research methodology utilized, as long as the researcher keeps these vital elements in mind research can be carried out in a manner that is justified and true to scientific theory.

The strategies and methods adopted by the researcher should be those that are most likely to provide the research with knowledge and adequate evidence with regard to a given phenomena. The knowledge the individual provides should be available for questioning and further testing in order to verify its legitimacy.

Research in Education

Bloland (1992) suggests that a strict quantitative approach with regard to educational research would limit the information provided, and suggests that an understanding of students, culture and behavior by nature requires a more qualitative approach.

Further, Bloland suggests that more and more qualitative research is becoming an acceptable form or research methodology as theorists and larger research authorities and agencies become more cognizant of the complexity involved in student development and learning, in addition to the influence environment may have on student achievement (Bloland, 1992).

Qualitative approaches to research in education may yield information that will specifically provide insight into the social and cultural patterns of learning (Bloland, 1992). Research that is qualitative or phenomenological in nature relies on data to describe human experience or behavior rather than using data to established relationships between events (Bloland, 1992).

Qualitative and quantitative research approaches vary in a number of ways. Qualitative analysis traditionally relies on non-numerical meanings and interpretation of behavior or patterns, for "the purpose of discovering underlying meanings and patterns in relationships" (Babbi, 1983; p. 537). The quantitative approach alternatively, provides numerical evidence and representation of observations for the sole purpose of describing and explaining phenomena rather than establishing patterns and understanding them (Bloland, 1992).

Each approach requires that the research examine a problem from an alternative perspective, and seek out different solutions or answers. A qualitative approach lends itself to discovering meanings of behaviors and patterns that exist in given behaviors, whereas the quantitative method is more likely to seek cause and effect relationships that occur in a given setting that can be demonstrated from a statistical point-of-view, using positivism and theoretical perspectives (Bloland, 1992).

The quantitative view is more concerned with facts and numbers, predictions and causation; researchers viewing behavior from a qualitative approach are more likely to employ the use of inductive analysis, and move from a specific observation to a more general one, whereas quantitative researcher is more likely to employ the use of 'deductive logic' and more from more general phenomena to more specific or from "theory to experience" (Bloland, 1992).

Analysis

Qualitative research refers to the meanings, concepts, definitions, characteristics, metaphors, symbols, and descriptions of things (Chappell, 2000; Gall, 2003). Quantitative research refers to the counts and measures of things (Chappell, 2000). Quantitative designs are descriptive or experimental. A descriptive study establishes only the associations between variables, and an experiment establishes causality (Chappell, 2000; Gall, 2003).

PALS is a program in the public schools in Virginia that screens and tracks children and finds literacy problems that they have so that measures can be taken early on to correct these problems. The primary purpose that PALS serves is to help identify children's literacy problems. Thus one may conclude that for purposes of a study, the researcher is attempting to identify a cultural or sociological phenomenon. Specifically the researcher is seeking to understand patterns in children's learning or literacy problems. The purpose of PALS is to help identify problems.

From the preliminary literary review, the researcher initially concludes that a qualitative research methodology is most appropriate for examining the PALS program. A qualitative study would provide in depth information regarding the characteristics of student learning ability and the PALS program. It would provide a foundation for future research which may be quantitatively-based with the intent of discovering statistical information regarding why exactly students are having literacy problems.

Qualitative investigation will provide information related to the human experience or behaviors demonstrated in the classroom using data. Qualitative approaches are not typically as popular in education (Bloland, 1992) however the approach is necessary as a preliminary step in this circumstance, before researchers can engage in cause and effect examination of behavior. Rather the intent of this study is to focus primarily on observable behavior of children being studies as a principle source of data for analysis, which is the foundation for qualitative research.

The purpose of qualitative research is in effect to gain a greater understanding of a given phenomena from a very specific and targeted point-of-view, from the view of the people being studied, so that conclusions may adequately be drawn with regard to behavior and interpretative analysis can be made (Bloland, 1992).