Data management concepts and applications

¶ … Management

The 21st century has brought with it a number of challenges in relation to data management. Translating data into high quality and understandable information is the key to competitive advantage. Determination can be unconstructively influenced when required information can neither be retrieved nor offered comprehensibly. At the same time as various large businesses have made it investments nearing 50% of their yearly principal expenses (Teresko, 1996), and nearly all companies have been plagued with data, there is still an unrelenting "information gap." In addition, there are human expenses. Executives managing database surroundings use a great deal of time with not only internal but also external resources, filtering data to acquire required information. Therefore, these huge demands on not only financial resources but also human resources have left executives with information overload (Ye, 2003).

Key to these 21ST century challenges has been the problem of acquiring dependable and assimilated outlook from various information courses. Previously, companies developed and applied "executive information systems" (EIS) along with "decision support systems" (DSS) to give assimilated presentations from broken and incoherent data that quite often got sorted from operational computer structures. Despite the fact that information given from an EIS or DSS had often been assimilated, significant, and practical, executives started to recognize that excavating huge quantities of data from a data warehouse might outcome in more precise information outlines and enhanced accurateness in market evaluation (Phillips, 1997).

Concentration on knowledge management in companies has enhanced the view of imparting and utilizing data to gain competitive advantage (Zorn et al., 1999). Attention in data mining is enhancing as companies look towards a more comprehensive understanding of their business, in order to not only satisfy their customers, but also enhance their efficiency. Data mining is a method that allows withdrawal of concealed predictive information from huge databases. It is a prevailing method with immense potential to assist businesses in concentrating on the most significant information accessible in their databases. It offers various kinds of tools to forecast future tendencies and actions to let executives make practical, knowledge-oriented decisions (Ye, 2003).

These new mechanized and innovative analyses surpass and exceed the analyses of preceding actions given by exhibitive tools distinctive of EIS and DSS. By means of data mining abilities, several business problems that had previously been too protracted to solve can be effortlessly tackled by exploring databases for concealed patterns and discovering prognostic information. Acquiring such advantages, on the other hand, is dependent on data quality in warehouses and efficient utilization of data mining tools (Ye, 2003).

Both these aspects (data quality in warehouses and efficient utilization of data mining tools) are rarely understood and seldom executed in the correct way. The purpose of this research is to explain the functions of data mining in favor of business decisions and to define some of the obstacles to its effectual deployment in companies and give recommendations.

Literature Review

For along time, business executives had complained that they are sinking in data generated by computer-founded structures. Nonetheless, practical and useful information is still in short supply. Companies have been amassing huge quantities of data. The range of the business world's data is predicted to double every year and a half (Frawley et al., 1991). The typical Fortune 500 business deals with more than a terabyte of digital information - that is amid 20 million to 500 million text reports and pages - every day, with 57% yearly increase (Kempster, 1998; Zorn et al., 1999). For case in point, the databases of at&T might comprise phone call patterns of its 100 million clients who produce more than 200 million phone calls every day.

On the other hand, most of the data has not been successfully used because of lack of highly developed systems to examine this significant and complexly developed data (Rosen, 1996). This difficulty has been further complicated by the company's rising utilization of Web tools (for instance, e-commerce, extranets and intranets). These tools allow companies to collect additional data from their digital dealings, which, consecutively, need far-reaching analyses to satisfy their clients. Problem of surplus data and very little information is the core issue at present (Ye, 2003).

Ye (2003) writes, "Advanced technologies have enabled us to collect large amounts of data on a continuous or periodic basis in many fields. On one hand, these data present the potential for us to discover useful information and knowledge that we could not see before. On the other hand, we are limited in our ability to manually process large amounts of data to discover useful information and knowledge. This limitation demands automatic tools for data mining to mine useful information and knowledge from large amounts of data. Data mining has become an active area of research and development (Ye, 2003)."

Data mining" can be defined as a procedure for making breakthroughs by excavating unidentified and actable information from huge and secure databases in favor of making practical business decision. Data mining evokes algorithms, which itemize designs from, or equip models to, data (Fayyad, 1997). The excavated information can be utilized to shape a forecast or categorization model, recognize relations amid database accounts, or offer an outline of the databases being excavated. Those designs or systems can be utilized to direct strategic decisions and predict the outcome of those decisions (Kempster, 1998; Saarenvirta, 1999). For that reason, data mining discovers and reveals previously unknown or overlooked information for executives which, consecutively, outcomes in more practical business decisions.

Kelly (2005) writes, "The term 'data mining' has come to refer to a set of techniques that originated in statistics, computer science, and related areas that are typically used in the context of large datasets. The purpose of data mining is to reveal previously hidden associations between variables that are potentially relevant for managerial decision making. The exploratory and modeling techniques used in data mining are familiar to many statisticians and include exploratory tools such as histograms, scatterplots, boxplots, and analytic tools such as regression, neural nets, and decision trees (Kelly, 2005)."

The procedure of data mining comprises the recognition of the data mining aims, recognition of the data, which is to be utilized, training and renovation of the data, along with data mining. May be the most effective access to data for analysis is to excavate the data from a warehouse. This, on the other hand, does not mean that a data warehouse ought to exist so as to extract and mine data. Data mining applications can, in addition, function outside warehouses. In this instance, nonetheless, data mining needs additional steps for mining, importing, along with evaluating data (Ye, 2003).

In operation, data mining needs the execution of two major steps. The first is the assortment and alteration of data into a reliable design for the mining processes (this is where warehousing comes into the picture), and the second is the utilization of analytical methods to assess data and recognize designs and forecasts for making decision (Ye, 2003).

The warehousing process

This measure might use from 50% to 80% of the entire data mining endeavor. In view of the fact that data is frequently stocked in various systems and designs, data from varied sources has got to be combined to generate a data warehouse, which is able to assist mining of data and making of business decisions. The warehousing procedure utilizes electronic and digital methods to combine data from various resources, redesign it, and thereafter arrange it in a manner to assist in drawing a macro outlook of information for managers and more comprehensive information for functional clients (Ye, 2003).

Data mining process

This process comprises the automatic withdrawal of embedded, formerly unidentified, and potentially helpful information from data in huge warehouses or databases. It comprises methods to mechanically discover designs in a warehouse with no previous production of a theory. It utilizes software designs to forecast future circumstances, categorize clients or conditions into known groups, discover developing operational difficulties, or assist in making concurrent business decisions (Ye, 2003).

Barriers to effective data mining

Several barriers exist in the implementation of effective data mining. Firstly, lack of vision towards data mining by organizational leaders is a common phenomenon. Secondly, producing and sustaining data and utilizing unacceptable resources and designs systems. Thirdly, ignoring corporate responsibility inside the data management workforce, which leads to inferior data quality. Fourthly, managers often do not set the standards for measuring data and fail to provide workable definitions of quality. Fifthly, selection of data mining software is done while disregarding company vision towards data mining. And lastly, failure to set up monitoring processes to maintain data quality at common stages of data formation, mining, and execution (Ye, 2003).

It is important to note that even the most complicated data mining applications cannot create successful databases and warehouses if the data is wrong. For case in point, a production firm had to fragment an almost $20 million database venture because of disconnectedly defined merchandise data along with inferior quality. A publishing and broadcasting firm used four years for cracking down client data that had been acquired from a dozen different archives (English, 1996).

Information that crawls into the databases or warehouse might be utilized for reasons beyond those initially planned by those filling and amassing the data (Phillips, 1997). Data that might be pleasing in quality for functional databases can be not viable in warehouses that confirm practical strategic business decisions. For case in point, the correctness of the analysis code in an insurance firm's functional database had been immaterial when disbursing insurance claims but might trigger a risk analysis to stop working.

From the above mentioned facts it is clear that research needs to be carried out on data mining so as to clarify and recommend solutions to the problems being faced by business executives in their data mining efforts. The purpose of this research is to explain the functions of data mining in favor of business decisions and to define some of the obstacles to its effectual deployment in companies and give recommendations.

Hypothesis

It is more likely for companies which have a strong vision towards data mining, which produce and sustain data and utilize acceptable resources and designs systems, which assign corporate responsibility inside the data management workforce, which have measurable standards for measuring data and provide workable definitions of quality, which select data mining software this is in line with the data mining vision of the company, which set up monitoring processes to maintain data quality at common stages of data formation, mining, and execution to successfully make use of data mining than companies which do not adhere to the these aspects.

Methodology have chosen case study as the design for this research. This is because I believe that case studies are a worthy tool for research even though it has been said that case studies lack the scientific rigor of other research tools. This is because they cannot rely more on the quality of the matter that the researcher has collected and interpretation of data. This study can be categorized as exploratory research because it will explore, in depth, this particular case and not generalize it.

This case study is different from historical research predominantly by its time-frame and importance. While historical studies characteristically include a period of quite a few decades, this case study centers focus on a reduced time range. In terms of importance, the intention of historical studies are typically to outline cause-and-effect relations as they grow throughout the years, while the intention of this case study is to explain the functions of data mining in favor of business decisions and to define some of the obstacles to its effectual deployment in companies and give recommendations.

This type of research also puts an emphasis on observations. As data is collected from an extensive set of research studies, articles, official publications and statements, I think that a case study would be most suited for my research, giving me an in depth study of the situation. As a researcher I will neither be making assumptions nor will I generalize my results when interpreting the data.

Research Design

Collection of Data

The tactic involved in the process of collection of relevant data will include gathering of concise, and yet, comprehensive information related to the topic (explaining the functions of data mining in favor of business decisions and defining some of the obstacles to its effectual deployment in companies and giving recommendations) from articles published in various scientific journals and magazines by individual researchers, as well as, the government, research institutions and companies using data mining tools. Both, online resources and offline resources will be used to compile the data.

Data analysis and Search tactics

The data analysis and search tactic will depend on manifold means so as to guarantee the most advantageous totality of facts and statistics available. At the outset a comprehensive literature exploration will be performed by means of internet, as well as, university and public library, as mentioned above. In this manner the bulk of published information relating to the topic (explaining the functions of data mining in favor of business decisions and defining some of the obstacles to its effectual deployment in companies and giving recommendations) will be distinguished, initially, and compiled, subsequently.

The analytical strategy that will be employed in this paper will firstly identify the gravity of the situation at hand relating to functions of data mining. Thereafter, the paper illustrates some of the obstacles to its effectual deployment in companies and lastly gives recommendations based in empirical research. Appropriate theories and facts will be given to prove not only the seriousness of the situation but also the validity of the arguments. Also, a brief overview will be given of the past and present situation by evaluating the strengths, weaknesses of the past and present trends and the emerging opportunities and threats. Furthermore, the most pertinent and possible factors relating to the topic (explaining the functions of data mining in favor of business decisions and defining some of the obstacles to its effectual deployment in companies and giving recommendations) will be analyzed and thereafter a thorough examination of possible variables and alternates will be conducted.

As mentioned above, factors influencing data mining are several, but only the most relevant ones will be discussed in this thesis so that the relevancy of the argument can be sustained throughout the paper. Furthermore, the use of quotations, throughout the thesis, will denote the diverse opinions of the most expert personnel in the pertinent field. The observations and views expressed by leading analysts' will offer support to the various assertions that will be presented in this thesis.