Qualitative Research and Validity

¶ … Validating Qualitative Data

Dissimilar to quantitative research, establishing or ensuring validity and reliability in qualitative research can be a difficult task. This is mainly because of the inherent differences between the two types of research. Even so, validity and reliability are as important in qualitative research as they are in quantitative research. Accordingly, qualitative researchers must ensure they provide valid and quality accounts of the research phenomenon at hand (Creswell, 2013). So how can qualitative researchers achieve this? This paper discusses standards of validity and reliability in qualitative research, particularly phenomenological research. The aim of the paper is two-fold. In the first part, the paper identifies and discusses the strategies or techniques used to assess validity and reliability in phenomenological research. In the second part, a selected published phenomenological research is evaluated to determine how conventional standards of validity and reliability have been applied.

Part 1: Evaluative Criteria and Validation Strategies

Before proceeding further, it is important to understand what validity and reliability mean. The notion of validity essentially refers to the extent to which the findings of a research are a true reflection of the real world (Bryman, 2008). In other words, how accurate or well-supported are the findings? Reliability, on the other hand, denotes the extent to which a research is consistent (Bryman, 2008). In other words, would the study produce similar findings if it were replicated, or do the findings and interpretations resonate with the data? Validation and reliability are crucial in qualitative research, though some scholars view them as unimportant. In essence, there are differences in how the concepts of validity and reliability are viewed in qualitative and quantitative research. Rather than validity, qualitative researchers may often use terms such as credibility, authenticity, confirmability, dependability, and transferability (Creswell, 2013). For reliability, qualitative researchers may prefer to use terms such as stability. The use of alternative terms is often informed by the need to avoid dragging positivist or naturalist concepts into qualitative research. All the same, the motive is generally the same -- to enhance and guarantee the accuracy and consistency of research.

Generally, the quality of qualitative research can be evaluated using techniques such as triangulation (confirming with other sources), peer review (external assessment of the research design), member checking (asking participants for feedback regarding the credibility of findings), external audits (inviting an external examiner to assess the research design and findings for accuracy) (Creswell, 2013). Other techniques that may be used include negative case analysis, prolonged engagement, persistent observation, stating researcher bias from the very beginning, and providing rich descriptions. Creswell (2013) recommends the use of at least two of these strategies. Though standards for evaluating validity and reliability may be quite similar for qualitative research in general, it is important to note that each type of qualitative research has its own standards. For phenomenological research, the focus is to determine how well-grounded the findings are. In other words, "does the general structural description provide an accurate portrait of the common features and structural connections that are manifested in the examples collected?" (Creswell, 2013, p. 215).

Based on this premise, phenomenological researchers should ask themselves the following five questions (Creswell, 2013): Did the researcher influence the subjects' responses in a manner that the responses do not accurately mirror the subjects' real experiences? How accurate is the transcription, and to what extent is it consistent with the oral recording of the interview? In analyzing the transcription, did the researcher identify alternative conclusions? Is there a link between the general structural description and the transcription, and is it possible to explain connections in the original account? Is the structural description specific to the situation or can it be generalized to other situations? Further, the quality of phenomenological research may be evaluated by checking whether the research reflects the philosophical underpinnings of phenomenology, whether the research phenomenon is clearly articulated, whether data analysis procedures have been used, whether the researcher communicates the general essence of the participants' experiences, and whether the researcher demonstrates reflexivity throughout the study (Creswell, 2013).

In my research, two criteria would take priority. First, it would be important to ensure the research clearly reflects phenomenological tenets. At its core, phenomenological research is about understanding how individuals perceive or interpret a given phenomenon or situation (Bryman, 2008). Therefore, a study that does not provide individual experiences of the phenomenon in question loses its validity as a phenomenology study. Second, I would ensure transcription accuracy. It is possible for discrepancies to be present between transcriptions and the oral recording of the interview. In such a case, the reported findings would not be a true reflection of the real-world. Therefore, I would ensure I transcribe the recording as accurately as possible, keen not to leave any important accounts or misrepresent some accounts.

Part 2: Validity and Reliability as Reported in a Published Qualitative Research

With knowledge of standards for evaluating validity and reliability in qualitative research, one is able to apply the knowledge to any qualitative research they come across. In the following section, the above standards are applied to a selected phenomenological study. The selected study is Akim & Yildrim's (2016) study; a study conducted to explore how elementary teachers perceive classroom management, especially in terms of the challenges they experience and the strategies they use to overcome the challenges. Following semi-structured interviews with 15 respondents drawn from the Turkish context, respondents reported classroom management problems in five areas: relationship management, behaviour management, time management, planning, and the physical environment. Further, respondents reported using various strategies to manage the problem, such as effective lesson planning, seating arrangement, fostering team spirit, and allocating responsibilities to students.

One of the ways through which the quality of phenomenological research can be evaluated is by checking whether the study is based on phenomenological principles (Creswell, 2013). Evidently, Akim & Yildrim's (2016) study is phenomenological in nature as it highlights the individual experiences of elementary teachers with classroom management. Without a doubt, student behaviour and classroom management practices may differ from context to context, often depending on factors such as subject matter, age, grade level, socioeconomic background, emotional and behavioural characteristics, as well as the nature of the underlying education system. Classroom management practices may also vary with the personality and character of the teacher. For the researchers, therefore, it was important to understand teachers' lived experiences with regard to classroom management. In essence, the study brings out the interpretivism or subjectivism that is characteristic of phenomenological research. It conveys the deep meanings elementary teachers in the Turkish context attach to classroom management.

Conveying deep meanings of subjective experiences means that the researcher must provide a "rich, thick description" of the research phenomenon (Creswell, 2013, p. 209). This is evident in Akim & Yildrim's (2016) study. More specifically, the authors provide a detailed description of the participants, the setting of the study, as well as data collection and analysis procedures. For instance, the researchers present a clear description of the participants' demographic characteristics (e.g. age and gender), the sampling strategy used (purposive sampling), participants' teaching experience, and the duration of the interviews (30-45 minutes). Richness is also observed in the results section of the study. Incorporating transcriptions, the section clearly reports the findings of the study, providing an unambiguous picture of how the respondents perceive classroom management, the problems they experience in maintaining order in the classroom, as well as the strategies they employ to address the problems. On the whole, a rich description of the research phenomenon, the research design, findings, and interpretations is provided. With such a rich description, it becomes easier for the reader to determine whether the findings can be transferred to other settings (Creswell, 2013). Transferability is an important hallmark of validity in qualitative research.

The ultimate evidence of validity in Akim & Yildrim's (2016) study is the use of external experts, pilot testing, member checks, prolonged engagement, persistent observation, peer review, and external audit in the research process. Indeed, this is a major strength as far as the design of the study is concerned. The researchers used a researcher-designed instrument to collect data. Upon designing the instrument, the researchers sought the opinions of three experts to ensure the instrument would effectively collect the required data. This is a clear indication that the researchers understand the importance of ensuring validity in qualitative research. An external party provides honest feedback about the research process, enabling the researcher to make the necessary adjustments (Creswell, 2013). Based on expert feedback, the researcher revised the instrument accordingly. The researchers then conducted a pilot test to guarantee clarity, appropriateness, and effectiveness of the interview questions. With a pilot test, the researchers were certain that the instrument would be effective for the actual study.