Data mining processes and organizational information overload management

¶ … Data Miners From Overwhelming the Organization

Data mining is not new, but the ways in which it is used are changing. There are more and more companies today that are interested in getting the 'hidden' information from their databases, but they can become confused about what they are really looking for and whether that information can tell them anything that they actually need to know. There is, naturally, a lot of interest for some people in finding out whether there are important pieces of information locked away in the data that they have already painstakingly collected. The patterns that might be there but are not now seen could tell them a lot of interesting things about their customers or their potential customers. It could also serve to tell them why some people will not be their customers and want only to shop at other places instead.

Where survey work is concerned there can also be important issues to address in the field of data mining. If there is a lot of information collected for a study, it is possible that data holds the key to other interesting concepts that will help that study or another one that is on a similar topic. Naturally, there are no guarantees that this will be the case, but it is generally thought to be worth a try. Extracting that data is not always easy, however, and one has to know how to search. It can be risky in the sense that it can easily cost a lot of time and money to pull this data from a large database. There is also the chance that there might be sensitive information found within that database, and that would be problematic for people who did not want that information uncovered or who did not intentionally give that information to someone for the purpose of extracting it and using it elsewhere.

Overwhelmed with Data Mining

The process of data mining itself is relatively simple. An algorithm is written that will pull information that otherwise would not be easily noticeable. This information has to be in the database, of course, but it is often 'hidden.' This means that it would not come up under a regular search, since it is not set in any of the parameters that the regular search entails. This does not mean that the information is classified or sensitive in a way that could be a problem - although this is possible. Mostly it means that it gives companies another way to study market trends and demographics that they might not have otherwise been able to study or that they might have otherwise had to pay a lot of money to collect through surveys or other means. The problem with data mining, though, is that it can quickly become overwhelming.

There is so much information hidden in there, and how to get it out is something that fascinates a lot of people. Once one gets it out, what does one do with it? That is another question that can be very difficult to answer, and it can require some visualization that not a lot of people are ready for or comfortable with. This is one of the truest examples of the idea of 'thinking outside the box.' People who are 'in the box' thinkers will likely not have a lot of use for data mining because they feel as though there is not anything there that could help them out. They are strictly logical, and they do not want to mess around with data mining and get a lot of information that they do not think would tell them anything important. For people who do not always want to think inside of the box in their businesses, though, data mining can be both a blessing and a curse.

It is a blessing because it can give them a lot of fresh new insight into relationships between variables that they might not have otherwise ever noticed. It is a curse because it is easy to overwhelm a person or a business with all the data that can potentially be mined. So, how does one keep data miners from overwhelming his or her organization? By understanding what data mining can and cannot do, and by setting limits and rules as to how much data mining will play into the organizational strategy of the company. All companies should have strategies that they live by, and if they do not have these yet, they should be working on them in order to ensure that they are in place. Data mining can certainly have a place in any organization, but only if the people who are operating that organization understand both the value of it and the limitations that it has.

Conclusion

There is no question that data mining is important, but when it comes to how to keep the data miners from overwhelming the organization, one thing is clear: data mining must be only one tool that the company uses. It is not necessary, nor is it wise, to put all one's eggs in one basket, but that is what many companies do with their data mining. Especially when they start to struggle just a little bit with some aspect of their marketing or sales, they tend to panic and assume that they have to do something drastically different. In reality, they might only need to make some small changes, and there are many ways to take a look at the organization and see if there is something different that they should be doing based on their target market's demographics. Data mining is one of the ways that a company can check into this consideration, but it is far from the only way. Companies that over-utilize it as a means to solve all of their problems are usually the companies that fail or struggle because they start making bad business choices.

These can be corrected but it takes time, and companies that do not learn what they did wrong will often continue data mining in an effort to discover where they went wrong. This can compound the problem and allow data miners to truly overwhelm the organization with facts, figures, calculations, and ideas for change. Some of these will certainly be good ideas, but they will get lost in the shuffle, and if a business implements one good idea and three bad ideas, that good idea will probably not be recognized because sales will continue to fall. Companies that are serious about growth and development must understand that there is much more to running their business and making changes to keep up with customers than data mining. When data miners are used as an important part of a balanced operation they can be very helpful and ensure that the company stays on the right track without in any way being overwhelming.