Comparative analysis of field experimental and survey research methods in social science

¶ … Experimental, and Survey Research

Social researchers have a wide array of qualitative and quantitative research methodologies available to them, including field, experimental and survey research. Each of these research methodologies has some strengths and weaknesses that make them better suited for some applications than others. The purpose of this paper is to provide a review of the relevant literature concerning field, experiment and survey research to identify their respective strengths and weaknesses and to determine what types of research are most appropriate for these approaches. A summary of the research concerning these issues and important findings are presented in the conclusion.

Review and Analysis

Field Research

The purpose of field research is to provide researchers with first-hand observations and interactions with the subjects who are of interest. Although there is rarely an actual field involved in this type of research, researchers are in fact required to leave the comfort of their offices to spend time observing people in their natural environment which can be their own home, place of employment, public space or other settings. For instance, according to Neuman (2003), field research is "a type of qualitative research in which a researcher directly observes the people being studied in a natural setting for an extended period. Often, the researcher combines intense observing with participation in the people's social activities" (p. 535). It is this interaction between the researcher and the people being studied that characterizes much field research. In this regard, Burgess (1999) reports that, "Doing field research is not merely the use of a set of uniform techniques but depends on a complex interaction between the research problem, the researcher and those who are researched" (p. 6).

Field researchers must be flexible in how they go about performing their studies because different situations demand different research tactics. This aspect of field research distinguishes it from experimental or survey research since the latter two are essentially procedurally inflexible. As Burgess points out, "It is on this basis that the researcher is an active decision-maker who decides on the most appropriate conceptual and methodological tools that can be used to collect and analyze data" (p. 6). In fact, field researchers may need to alter their methodologies "on the fly" as their studies progress or even use a number of different research strategies simultaneously (Burgess, 1999). For instance, Sanger (1999) reports that, "The difficulty for qualitative research is that the researcher is not trying to solve highly focused problems. Rather, the field tends to be a diffuse, implicated set of interrelated issues which include the researcher, the impact of the research and a process of continual contemporaneous change" (p. 96). Consequently, field researchers may be forced to rely on a research design that does not have proven reliability or validity (Burgess, 1999).

Notwithstanding this constraint, Veeck (2001) emphasizes that, "Field research serves as a vital check against the unguarded surety of theoretical abstraction. Frankly, the inductive / deductive cycle is seldom possible without field-based research that allows both the collection of data and the verification and analysis of existing explanatory models" (p. 35). Likewise, Hubbell (2003) cites the advantages of being able to ask questions and probe for follow-up answers in field research, but cautions that field research can be especially difficult in cross-cultural settings. Despite the cross-cultural constraints that may be involved, field researchers typically employ an open-ended interview approach that allows for a great deal of latitude in direction. In this regard, Hubbell emphasizes that, "Unlike forced choice questionnaires, the open-ended interview is an exchange of information and a joint construction of meaning. It is more structured and focused than a mere conversation, but allows for the surfacing of more serendipitous and potentially interesting information than a questionnaire" (p. 197). In addition, the findings that emerge from field research can be adversely affected by researcher bias that may be difficult to discern and overcome (Sanger, 1999). Therefore, field research can be regarded as a challenging but potentially valuable and effective research methodology that requires a significant investment of time and effort. Because field researchers do not introduce any interventions in their research design, this approach is distinguished from experimental research which is discussed further below.

Experimental Research

The purpose of experimental research is to test the effects of some type of intervention on a group of people. In sharp contrast to field research where researchers observe others but do not necessarily introduce any changes to their routine, experimental research "is research in which one intervenes or does something to one group of people but not to another and then compares results for the two groups" (Neuman, 2003, p. 534). The essential elements of experimental research, then, relate to manipulation and control. In this regard, Singleton and Straits (1999) define the experimental research method thusly: "The key features of the experimental approach are manipulation and control. To test hypotheses, the experimenter deliberately introduces changes into the environment…and observes or measures the effects of the changes" (p. 179). One of the major strengths of the experimental research method is its ability to provide replicable data that is known to be valid. For example, Jonassen (2004) report that, "The experimenter's interest in the effect of environmental change, referred to as 'treatments,' demanded designs using standardized procedures to hold all conditions constant except the independent (experimental) variable. This standardization ensured high internal validity (experimental control) in comparing the experimental group to the control group on the dependent or 'outcome' variable" (p. 1021). In other words, to the extent that internal validity of experimental research is high is the extent to which researchers can attribute the effects of the outcome to the intervention rather than a confounding variable (Jonassen, 2004).

Experimental research provides a number of important controls during the research process as well as valuable outcomes that make this approach the gold standard for researchers seeking to confirm or refute various hypotheses. According to Singleton and Straights, "Because greater control is exercised over the conditions of observation than in any other research strategy, experiments more effectively eliminate the possibility of extraneous variables offering alternative interpretations of research findings. For this reason, experimental studies have long been regarded as the optimal way to test causal hypotheses" (p. 179).

Researchers who employ experimental research generally assume that a causal relationship can be presumed if the following three conditions are satisfied:

1. Changes in the independent variable must be associated (correlated) with changes in the dependent variable;

2. Changes in the independent variable must occur before changes in the dependent variable; and,

3. No other extraneous variables can be offered as rival explanations for the observed relationship (DeMarrais & Lapan, 2004, p. 333).

The satisfaction of these three conditions is required in order to ensure that any changes identified in the experimental group are actually the result of the experiment rather than an alternative explanation. According to DeMarrais and Lapan, "These three conditions preclude drawing a causal inference between two variables simply because there is a strong relationship (correlation) between them. In simple language, correlation does not imply causality" (2004, p. 334). In order to be regarded as valid, though, experimental researchers must take some preparatory steps in designing their study to ensure that the subjects of interest are grouped and tested randomly. In this regard, DeMarrais and Lapan note that, "The only surefire way to ensure that experimental samples are comparable at the outset is to randomly assign the experimental units to treatment conditions. In educational research, the experimental units usually consist of students, but they could consist of classes, schools, teachers, or some other entity" (p. 334). Therefore, when groups are randomly assigned, the members of each group will only differ by various chance amounts on any variable that is under investigation (DeMarrais & Lapan, 2004). By increasing the size of the sample, experimental researchers can further reduce the chance amounts that do exist between the experimental and control groups (DeMarrais & Lapan, 2004). In this regard, DeMarrais and Lapan note that, "The larger the samples are, the smaller these chance amounts will be. These chance differences can be reduced further if participants are first matched on one or more extraneous variables, and then the experimental groups are formed by randomly assigning match mates to the treatment conditions" (p. 335). Therefore, when properly designed, administered and analyzed, experimental research can provide researchers with on-point findings to help confirm or refute their hypotheses (DeMarrais & Lapan, 2004). Field research and experimental research are distinguished from survey research which is discussed further below.

Survey Research

The purpose of survey research is to gather timely information concerning the opinions and views of potentially large numbers of people. In this regard, Leedy and Ormrod (2005) provide a useful definition of this common research methodology: "Survey research involves acquiring information about one or more groups of people ? perhaps about their characteristics, opinions, attitudes, or previous experiences ? By asking them questions and tabulating their answers. The ultimate goal is to learn about a large population by surveying a sample of that population" (p. 183).

Although field research also provides researchers with the opportunity to ask questions, the types of questions that are used in survey research typically differ from the open-ended questions used and rely on closed-end questions, the responses to which can be coded analyzed for thematic content and patterns (Leedy & Ormrod, 2005). It is important to note, though, that just as field research can employ a wide range of methods, survey research can likewise use different approaches to gathering information (Mason, Chang & Griffin, 2005). This point is also made by De Vaus (1996) who emphasizes, "A survey is not just a particular technique of collecting information: questionnaires are widely used but other techniques, such as structured and in-depth interviews, observation, content analysis and so forth, can also be used in survey research. The distinguishing features of surveys are the form of the data and the method of analysis" (p. 3). In most cases, the form of the data developed thorough survey research will assume either statistical data that is easily analyzed or textual data that must be coded in order to be analyzed (Neuman, 2003).

Although there are several strengths associated survey research, including the ability to gather large amounts of data in a relatively short amount of time in a cost-effective fashion, researchers using surveys must attempt to ensure that their results are generalizable beyond their immediate sample. In this regard, Grinnell and Unrau (2005) point out that, "Surveys can be designed to achieve a variety of ends, but they all seek to collect data from many individuals in order to understand something about them as a whole. It is essential, therefore, that survey research procedures produce data that is accurate, reliable, and representative so that findings can be generalized from a sample to the larger population or to different research situations" (p. 272). Although developing accurate and reliable survey data is not impossible, it can be difficult and this issue is cited as one of the main weaknesses of survey research. For instance, De Vaus (2005) also notes that, "Many criticisms of surveys are based on misunderstandings of what surveys can be and are based on examples of poor surveys and the inappropriate use of survey research" (p. 3).

Moreover, it is entirely possible for survey researchers to develop a valid survey instrument and still fail to achieve their specific research goals. In this regard, Proctor and Vu (2005) note that, "A poor survey will result in measurement error when the researchers inadvertently obtain data that do not accurately assess the sample on the intended dependent measure" (p. 310). Survey research has also gained in popularity in recent years as a result of the ability of researchers to post their surveys online where they can be accessed by large numbers of respondents from all over the world. According to Wood, Nosko, Desmarais, Ross and Irvine (2006), "For decades, paper-and-pencil surveys have served as the primary means for gathering survey information primarily because paper-and-pencil surveys provided a more economical and efficient means for collecting large amounts of data relative to other available data collection techniques, such as interviewing" (p. 147). These attributes have been further amplified by the Internet where survey services such as Zoomerang and SurveyMonkey providers researchers with templates for survey design and automated data analysis.