Reliability Credibility and Validity

Reliability
a. Dependability means emphasis on the need for researchers to take into consideration, the ever-evolving context within research as it happens (Creswell & Miller, 2000). The responsibility of qualitative research is to describe the changes naturally occurring in the setting as well as how such changes researchers approach the study. To address dependability within a qualitative research study, a person can perform member checking of data interpretation, a pilot test, and a peer review. These techniques allow for accurate assessment of dependability regarding the information collected for the study.
When a person begins qualitative research, to determine reliability and dependability of the information collected, they may perform member checks. “Member checking, also known as participant or respondent validation, is a technique for exploring the credibility of results. Data or results are returned to participants to check for accuracy and resonance with their experiences” (Birt, Scott, Cavers, Campbell, & Walter, 2016, p. 1802). Such simplistic reporting may be criticized due to its interpretative stance concerning qualitative research, however, it is often a typical technique employed by researchers. The main reason it is used is because of the chance to understand and determine information the authors intends and assess whether the interpretations are wrong or have errors. It is a playing back process that serves as respondent validation of informant feedback.
The next technique often employed is peer review. Peer review has many names. Some think of it as auditing, expert review, or independent scientific review (Wicherts, 2016). Regardless, it works similar like the member check in that, someone else reviews the information for reliability. Except the work is subjected to others’ scrutiny. Those others being experts within the same field, before work can be published as a book, in a journal, or in conference proceedings (Wicherts, 2016).
The final technique up for discussion is a pilot test. Pilot studies allow for researchers to find potential barriers and issues concerning recruitment of potential participants and observing things from a phenomenological perspective. Other benefits gained from a pilot study that increase dependability is a chance for reflection of the significance of the epoche process and the struggle in performing phenomenological inquiry, along with potential modification of interview questions (Kallio, Pietilä, Johnson, & Kangasniemi, 2016). Pilot studies serve as an extra layers of reflection and assessment that could lead to thoughtful interpretation of qualitative data all while improving the dependability of the accuracy of said data.
Validity
a. Much like dependability, credibility often means employing certain techniques to see what may or may not work in regards to data collection and creation. To establish credibility member checking can again be employed. Member checking allows researchers to establish what level of expertise and education someone that has written something has (Varpio, Ajjawi, Monrouxe, O'Brien, & Rees, 2016). It is an important tool is establishing a valid interpretation of information via the validity of the person who wrote and collected the data.
Another important technique used to establish credibility is triangulation. This is because it can be used both qualitatively and quantitatively. Triangulation can indeed improve credibility of scientific information by refining both internal consistency as well as generalizability via combination of both quantitative and qualitative methods within the same study (Northrup & Shumway, 2014). Due to the dual nature, data can be evaluated in various ways allowing for a better idea of the kind of interpretation ability of the researcher.
The last one is Participant transcript review is another tool that helps establish credibility. Participant transcript review allows researchers to establish whether participants fully and correctly answered the questions. It also gives a researcher and opportunity to refine the interview guide should it need it. “Analysis begins after reviewing the first interview to examine whether participants are responding to the research question related to your area of interest in diabetes, or whether your interview guide needs refining” (Stuckey, 2014, p. 6).
b. Transferability is another important aspect to consider regarding qualitative research. To establish transferability, one must provide readers with evidence concerning applicability of findings to other times, situations, populations, and contexts. That evidence must come from other sources and not the researcher (Sacks, 2015). For example, thick transcription is an approach that allows the qualitative researcher to offer a detailed and robust account of their experiences while collecting data (Sacks, 2015). By making explicit connections to social and cultural contexts, this is a great way to check for transferability.
Another way is ethnography. Enography means the study of people in their environment via methods like face-to-face interviewing and participant observation (Gentry, Milden, & Kelly, 2017). Such a method allows researchers to explain through their processes, information that can then be checked by others regarding aspects related to transferability. It is a common means of providing information without direct interpretation.
The next approach is case studies. Case studies allow for transferability check because of the possibility for empirical generalization. “Critical realism recognizes the role of case study research in empirical generalization, theoretical generalization, and theory testing” (Tsang, 2014, p. 174). Furthermore, it allows for theoretical generalization. Such actions can lead to easier means of transferability of data.
c. Confirmability means the degree to which a researcher’s findings in a study can be confirmed. Confirmability established that interpretations and data are derived from data and not figments of imagination (Korstjens & Moser, 2017). One technique to establish confirmability is audit trail. Audit trail entails transparently describing research steps taken from beginning to reporting and development of findings (El Hussein, Jakubec, & Osuji, 2016). Records of such a research path are maintained throughout the study.
The other technique is reflexivity. Reflexivity is the attitude the researcher adopts when gathering and evaluating data (Newman & Clare, 2016). Researchers must look at their own position and background to see how such aspects can influence the research process. Reflexivity acts like a person’s diary acting as a reflective process on the journey.
d. Data saturation is an emerging approach in qualitative reasoning. While there is no universal definition of what it is, it can be thought of as, “theoretical sampling—the idea that sampling is guided by ‘ecessary similarities and contrasts required by emerging theory’ and causes the researcher to ‘combine sampling, data collection and data analysis, rather than treating them as separate stages in a linear process” (Saunders et al., 2017, p. 1893). Data saturation involves the process of data collection and relates to the degree of data repetition expressed in previous data (Saunders et al., 2017). To ensure data saturation has taken place, it must be convincing to the reader.
It is hard to determine data saturation as there is no set value of what constitutes sufficient data. The general rule of thumb is ‘10+3’ formula or “there is enough information to replicate the study when the ability to obtain additional new information has been attained, and when further coding is no longer feasible” (Fusch & Ness, 2015, p. 1). Additionally, one can apply conceptual depth criteria as it provides an evaluative framework and tool for creation of a structured evidence base for the substantiation of choices made during a researcher’s theoretical sampling process (Nelson, 2016).
References
Birt, L., Scott, S., Cavers, D., Campbell, C., & Walter, F. (2016). Member checking a tool to enhance trustworthiness or merely a nod to validation? Qualitative Health Research, 26(13), 1802-1811. doi:10.1177/1049732316654870
El Hussein, M. T., Jakubec, S. L., & Osuji, J. (2016). The facts: A Mnemonic for the rapid assessment of rigor in qualitative research studies. Journal of Nursing Education, 55(1), 60-60. doi:10.3928/01484834-20151214-15
Fusch, P. L., & Ness, L. R. (2015). Are we there yet? Data saturation in qualitative research. TQR, 20(9), 1.
Gentry, S., Milden, L., & Kelly, M. (2017). How can researchers generate meaningful public health policy impact? A meta-ethnography of case studies. The Lancet, 390, S39. doi:10.1016/s0140-6736(17)32974-4
Kallio, H., Pietilä, A., Johnson, M., & Kangasniemi, M. (2016). Systematic methodological review: developing a framework for a qualitative semi-structured interview guide. Journal of Advanced Nursing, 72(12), 2954-2965. doi:10.1111/jan.13031
Korstjens, I., & Moser, A. (2017). Series: Practical guidance to qualitative research. Part 4: Trustworthiness and publishing. European Journal of General Practice, 24(1), 120-124. doi:10.1080/13814788.2017.1375092
Nelson, J. (2016). Using conceptual depth criteria: addressing the challenge of reaching saturation in qualitative research. Qualitative Research, 17(5), 554-570. doi:10.1177/1468794116679873
Newman, D. S., & Clare, M. M. (2016). School Psychology as a Relational Enterprise: the role and process of Qualitative Methodology. Contemporary School Psychology, 20(4), 327-335. doi:10.1007/s40688-016-0090-1
Northrup, J. C., & Shumway, S. (2014). Gamer widow: A phenomenological study of spouses of online video game addicts. The American Journal of Family Therapy, 42(4), 269-281. doi:10.1080/01926187.2013.847705
Sacks, T. K. (2015). New pathways to analysis through thick description: Historical trauma and emerging qualitative research. Qualitative Social Work: Research and Practice, 14(6), 753-757. doi:10.1177/1473325015612189
Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., … Jinks, C. (2017). Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & Quantity, 52(4), 1893-1907. doi:10.1007/s11135-017-0574-8
Stuckey, H. (2014). The first step in Data Analysis: Transcribing and managing qualitative research data. Journal of Social Health and Diabetes, 2(1), 6. doi:10.4103/2321-0656.120254
Tsang, E. W. (2014). Case studies and generalization in information systems research: A critical realist perspective. The Journal of Strategic Information Systems, 23(2), 174-186. doi:10.1016/j.jsis.2013.09.002
Varpio, L., Ajjawi, R., Monrouxe, L. V., O'Brien, B. C., & Rees, C. E. (2016). Shedding the cobra effect: problematising thematic emergence, triangulation, saturation and member checking. Medical Education, 51(1), 40-50. doi:10.1111/medu.13124
Wicherts, J. M. (2016). Peer review quality and transparency of the Peer-Review process in open access and subscription journals. PLOS ONE, 11(1), e0147913. doi:10.1371/journal.pone.0147913