Factor analysis and cluster analysis in statistical research

¶ … transforms correlations among a set of observed variables into smaller number of factors, containing essential information about the linear inter-relationships among the original test scores (Kriegel, Kroger, Sander, & Zimek, 2011). Factor analysis is used in behavioral sciences, social sciences, and other applied sciences dealing with large quantities of data. In addition, factor analysis is subdivided into two major groups; exploratory and confirmatory. Cluster analysis on the other hand entails dividing a set of variables into groups; clusters with data in the same cluster being similar to each other as opposed to those in other clusters. Cluster analysis is mainly used in statistical analyses such as information retrieval, and bioinformatics. Just like factor analysis, cluster analysis has two subgroups; exploratory and confirmatory.

Q-sort vs. R-sort Factor and Cluster Analyses

In both factor and cluster analysis, Q-sort entails ranking sets of statements based on how strongly subjects agree or disagree with them (Storch & Fischel). In line with this, Q-sort is a reflection of the respondent's view about the topic and its analysis helps in identifying specific respondent groups. Despite being complex forms of regression, both Q-sort and R-sort are vital in revealing data similarities and check for possible similarities as well.

The major difference between Q. And R-sort is that, R-sort reduces data about individuals into traits while Q-sort identifies common patterns of traits (Ankerst, Breunig, Kriegel, & Sander, 1999). Additionally, while R-sort reduces data into traits, such as ethnic background, Q-sort identifies patterns of self-referenced traits and enables respondents to model their viewpoints on a matter of subjective importance.

Q-sort is used to analyze collected data while R-sort involves finding correlations between variables across a sample of subjects. Therefore, statisticians refer to Q-sort as R-sort with the data table turned sideways.

Exploratory and Confirmatory Factor and Cluster Analysis

Both exploratory and confirmatory factor analyses are vital in understanding shared differences of measured variables attributable to a given factor. Despite this similarity, in cocept, both the EFA and CFA present different analyses.

In EFA, the researcher cannot hypotheses the number of issues that will arise and what items the factors will comprise (Mead & Legg, 1994). However, the existence of these hypotheses is ignored and is not incorporated into the outcome of the analyses. In contrast, CFA analyses the researcher has to have an advance hypothesis, the number of factors, and the measures that best reflect which factors. Therefore, as opposed to EFA where loadings are free to vary, CFA restricts certain loadings to be minimal.

Exploratory Factor Analysis is often used in uncovering the basic structure of many variables when the researcher does not have a-priori hypothesis about factors of measured variables. Confirmatory analysis on the other hand is useful in confirming the consistency a construct with the understanding of the researcher. As such, the objective of CFA is to test the consistency between the data and the hypothesized measurement method.

Just like factor analysis, cluster analysis does not distinguish between dependent and independent variables. In line with this, cluster analysis is seen as the reverse of factor analysis. Therefore, the difference between factor analysis and cluster analysis is that factor limits the quantity of variables by combining them into a smaller set while cluster on the other hand limits the quantity of cases by combining them into a smaller cluster set.

In Exploratory Cluster Analysis (ECA), the number of clusters is unknown thus; the number of clusters has to be estimated by the researcher during the analysis. In Confirmatory Cluster Analysis (CCA), the clusters number is known therefore requiring no further estimation.

Next, for ECA, the characteristics of clusters remains unknown before the research requiring interpretation yet, finding a substantive interpretation remains a challenge. CCA on the other hand outlines partially known characteristic of the clusters which have substantive interpretations (Thompson, 2004). Finally, in ECA, the fit to data is maximized while for CCA, the fit to data may be poor.

Uses and Misuses of Factor and Cluster Analysis

Uses of Factor Analysis

Factor analysis is used in applications relevant to various scientific and policy concerns. These include data reduction, confirming hypotheses, and scaling among others.

The first application is data reduction. Factor analysis is used to reduce large number of variables into smaller and manageable number of factors. In line with this, the analysis helps researchers reduce a large number of variables into a set of few factors that account better for the underlying variance in the measured phenomenon.

The next use of factor analysis is confirming hypothesis of factor structure. In this step, factor analysis is used in testing whether a set of items designed to measure a certain variables reveal the hypothesized factor structure. In line with this, characteristics related to particular dimensions are easily postulated in advance and statistical tests of significance are applied to the factor analysis results.

Uses of Cluster Analysis

Cluster analysis is commonly used in market research when working with multivariate data from surveys and test panels. This analysis is used by market researchers in partitioning the general population of consumers into market segments and better understanding relationships between different groups of potential customers. In addition cluster analysis is useful in market segmentation, product positioning, and selecting test markets by the researchers.

The other use of cluster analysis is formulating of hypotheses concerning the origin of various samples, such as in evolution studies. Moreover, cluster analysis is useful in predicting the future behavior of population types by assisting researchers in modeling economic prospects for different industry sectors.

Importance of Factor and Cluster Analysis

Factor analysis is important to researchers as it outlines the number of factors necessary in explaining the relations among a set of indicators and with estimation of factor loadings. In addition, it also assists in theory development during dissertations researches. Moreover, factor analysis helps in determining factors that conform to what is expected on the basis of pre-established theory.

Cluster analysis helps researchers organize observed data into meaningful groups and clusters while maximizing similarity of cases within each cluster and dissimilarity between groups that are initially unknown (Kriegel, Kroger, Sander, & Zimek, 2011). In addition, cluster analysis is vital in finding substantive interpretation for clusters during research.

Limitations of Factor and Cluster Analysis

The main limitation of factor analysis is that when researchers apply factor analysis to similar and identical sets of measures, some individuals may come up with 3 factors; others will come up with 6 while some will comes up with 10 (Thompson, 2004). This lack of agreement has tended to discredit all uses of factor analysis.

Both cluster and factor analysis, do not differentiate dependent and independent variables. Cluster analysis appears to be the counter factor analysis.

Methods for confirmatory cluster analysis are not available in standard software. SPSS offers only a rudimentary confirmatory analysis. All starting values have to be fixed or freed for estimation. Linear or non-linear restrictions or fixing some parameters is not possible. The validity of the conclusions drawn from cluster analysis techniques is sometimes questioned since very different clusters can be formed from the same data depending on how the analysis is performed. This apparent ambiguity results from the decisions made by researchers prior to the actual analysis.