Data Mining
Predictive analytics help companies to understand the behavior of consumers. The way that predictive analytics works is that data from the past is used to help refine predictions about the future (CGI, 2013). Companies basically analyzed demand in terms of a wide range of variables in order to arrive at a better estimate for future outcomes than otherwise would have been found. It is basically the same principle as predicting that a colder, snowier winter will help Wal-Mart sell more snowblowers, but with hard data, sophisticated algorithms and reliable outputs -- such as x number of snow days will equal y number of snow blowers sold.
One of the interesting elements of predictive analytics is with associations, and this has been used fairly extensively in retail. Associations discovery is where correlations between things are noted that might not have been apparent. So that link between snow blowers and snow days is an obvious association. Associations discovery might show links between snow blowers and unrelated products. Maybe sales of hot chocolate go up, because people want a hot chocolate after they've been out with the snow blower. The associations are not necessarily intuitive at first, but they say a lot about consumer behavior. Amazon does this when it gives you the "people who bought X also bought Y" prompt. Sometimes the Y is rather obvious -- an album by the same band -- but other times it is not obvious and that is the value of associations.
In recent years, companies have used the Internet as a major source of data gathering. Companies gather information from their customers, process this information for valuable associations and then use those associations to increase sales. The basic gathering of data is called mining, and then processing this data to derive useful information is known as business intelligence. Companies with a high level of access to information are the ones that can win a competitive advantage over their competitors, which...
Clustering is defined as the "unsupervised classification of patterns into groups" (Jain, Murty & Flynn, 1999). Clustering allows for trends to be better identified. Typically the clustering process is automated, which allows for the clusters to be drawn without the interference of human classification. In marketing, this is particularly important. Businesses have frequently clustered their customers -- by demographic, psychographic and otherwise -- and typically this has occurred on the basis of intuition to derive a hypothesis and then test it. An automated clustering process essentially tests thousands of hypotheses at once, confirming back several different clusters that pass statistical significance.
2.
To determine the "reliability" of data mining algorithms some clarification might be needed. The statistical reliability of any given analysis of a data set can be determined in the course of the calculation -- any decent stats program will provide reliability figures for any given analysis on any given data set. It just depends on the sample size, which in most cases is sufficiently large. Where there are errors, they are usually related to issues with the data. For instance, the data set might not be large enough -- no statistical analysis works well on a small data set. The data set also might have false positives -- for example a data set might conclude that people who buy Big Macs prefer Coke and people who buy Quarter Pounders prefer Sprite. While the sample size might be large enough, it might not be dispersed enough -- if all the data was collected in New York, it might not hold across the rest of the country. Or it might hold in most of the country, but not in Miami. Errors…
