How and When to Use a Meta-Analysis

Nuclear Power A meta-analysis is defined by Cochrane as "the use of statistical techniques in a systematic review to integrate the results of included studies." What this means is that the scholar will gather a number of studies on the same subject, and then seek to find commonalities in the results, and differences in the results. The objective of gathering together the different studies is to highlight patterns that hold across the different studies, and identify those that do not.

Particularly in medical research, the studies will often be specific to certain demographics, particularly geographies. So a meta-analysis will pull together a study from Australia, one of Germany, another from Japan, a few from the U.S., and then add in others that focus on different age or ethnic groups, or genders. Individually, none of these studies can be extrapolated to the public at large, but any trends that are affirmed by the meta-analysis likely can be.

Likewise, trends that do not hold widely can be investigated for issues that might be specific to the one demographic in question. Something that holds across the UK, Australia and Canada might show that people of British descent are more prone to something -- these are the sorts of research threads that can emerge from a meta-analysis, which would have identified the sub-trend. One of the challenges associated with meta-analysis is that each research study contained within might have different metrics that are being used.

This is where meta-analysis differentiates itself from similar qualitative techniques. Meta-analysis is inherently a quantitative technique, as it seeks to define common metrics that can be used to compare the different studies. Where these are developed, the common metric can then be used to identify valuable insights about the multiple studies. Where it is impossible to develop a common metric, meta-analysis will not be nearly as useful, and may not work out at all. Meta-analysis has become particularly popular in medicine, because of the way medical research is conducted.

There are two key traits to medical research that allows for meta-analysis to be used. The first is that it features relatively common metrics. For any given condition, there are measures relating to the demographic, social and environmental factors, and there are commonalities in the measures used in outcomes. This makes it much easier for the development of a common metric by which the different studies can be compared.

The second factor is that medical studies are often conducted on the basis of local data that is easy for the researcher to acquire. In many instances, it is possible for a researcher to acquire a statistically-significant data set locally, which means that for a given condition, there will be a lot of studies. Researchers will do studies to test different things, but they will also test in different geographies because there are differences in things like environmental factors, health care quality and demographics in a region.

A study from Los Angeles cannot necessarily be extrapolated to Mumbai, so researchers in Mumbai will conduct their own study. The result of this is that there are many studies on the same or similar topic. Thus, meta-analysis is popular in medical research because there are enough similar studies on a given subject matter to actually perform the necessary statistical tests. In the field of medicine, meta-analysis is particularly useful for two things. The first is to test for commonalities, the second is to identify differences in outcomes.

This is necessary because conditions are influenced by a number of different factors -- environmental, social and genetic among them. By identifying the common factors, it is easy to learn about which factors influencing a condition are common to everybody. For example, it can be determined that smoking in a major cause of lung cancer, because this trend has been identified across all groups studied. If every group studied shows that smoking has a correlation with lung cancer, that can point researchers to determine the specific causal effect.

This is particularly useful in other situations, when the cause is not something that is purely discretionary. So for example, if there is a difference in lung cancer rates between urban and rural dwellers that runs across all other factors, that can lead researchers to start to think about what the differences are between those environments, and which of those differences might be contributing to the higher lung cancer rates. The second benefit of the meta-analysis is medicine is to guide the researchers to the exemptions.

For example, suppose that all women have a 20% chance of developing osteoporosis if they live past 80, except Chinese women, who have only a 2% chance. This might not be immediately apparent from a series of individual studies. In any given pair of studies, there could be four or five differences that might explain the difference in results.

But a meta-analysis can allow the researcher to control for all other factors, and this is the only way that it can be conclusively determined that across all studies Chinese women simply get osteoporosis less often. Only a meta-analysis can yield this information across all studies, but the information could be valuable. Researchers would then be able to look at the genetics of Chinese women to find out what is different between them and all other women, which can pinpoint how a condition forms, and guide researchers towards effective treatments.

Though this example is fictional, this is how meta-analysis can be used to identify outliers and exceptions, and then apply that knowledge to further studies that aid in our understanding of different conditions and ultimately how we can prevent or cure them. How to Conduct a Meta-Analysis The first step in conducting a meta-analysis is to identify the research. This is important for a couple of reasons.

All research should cover some sort of new ground -- it should add to the existing body of research -- as a starting point. From there, however, a meta-analysis can only be conducted where there are enough similar studies to perform a statistical analysis. So the research question needs to relate to a topic for which a meta-analysis can be conducted and begin the process of researching this topic.

It may be too early, at least initially, to focus on finding specific studies -- it is best to let the search inform the researcher as to what questions might be answered via meta-analysis. The second step is to determine the eligibility of the research. This is an important step because this is where the studies that will be used in the meta-analysis are identified.

First, understanding the questions in each study is necessary, so that the researcher knows what studies answer what questions, and whether there are enough similar studies on a topic to work with. Then, the studies have to be vetted for their fit for the project. Some studies might be too old, for example, or they might not have a study design that lends itself to the meta-analysis at hand.

The purpose of this step is ultimately to find enough similar studies for which a meta-analysis can be conducted, and add value to the current scholarship in the field. The third step is to extract data from the studies. Each study will present the data that was used to find its conclusions. This can be recorded by the researcher in a way that allows the data from multiple studies to be evaluated together.

Thus, the researcher has to have a model for statistical analysis in place before the data can be extracted. The data from the different studies will be tested in a number of ways, but the first step is to get the data out of the paper, which can mean the raw data or the output.