Text Mining: Techniques, Processes, and Applications

Introduction to Text Mining

The concept of text mining originates from the idea that a relationship exists between the terms used in an unstructured text message or file. That relationship may extend to other similar files, and once established, it can provide information to businesses and researchers across many areas — changing the way they operate or enhancing collective knowledge.

The definition of text mining is broad. In simple terms, "text mining" refers to the process by which information is retrieved in text form. At a deeper level, however, it involves establishing patterns within textual data — not only locating the appropriate text but also developing a theory for making that information useful. Many definitions of text mining assume the primary goal of extracting high-value information from a database or other unstructured text field and using it to arrive at some conclusion. There is also a connection between text analysis and the concept of data mining. The distinction is that data mining draws from structured databases, whereas the principal challenge in text mining is achieving the same result with unstructured data (Trujillo, 2010).

Text is the most widely used medium and data type. It is the primary means by which people exchange information, and the media involved include email, chat, digital libraries, reports, and books available on the internet and other communication channels. Beyond user-generated content, there are also vast volumes of journals, research materials, and valuable reports such as statistical documents and government publications. These databases grow at astronomical rates and are distributed on a global scale (Mitra & Acharya, 2003). Text mining has therefore become an important set of tools across many operations on the information spectrum.

Technical Details of Text Mining

The method of text mining is complex and involves many steps and determinants to be fully successful. Text mining begins as an algorithm that extracts facts from a textual source and converts them into a form that can be used to create "hypotheses that are further explored by traditional data mining and data analysis methods" (Maimon & Rokach, 2005). In text parsing, problems are encountered with hyponyms — that is, generalizations of information. A contributor such as "Human" and associated positions such as "corporate executive" may constitute casual information in one context, yet vital information in another. Information of a general nature is often ignored because the span and token of the program do not account for it, even though it may be critical when viewed from a different perspective (Srivastava & Sahami, 2009).

To address this, the major operation in text mining is tagging. A text mining program can tag documents using statistical tagging or semantic tagging, and this forms the basis for arriving at new information. There are requirements for managers to find information from new angles, and this is often found in unstructured customer responses. This need is addressed through a task-oriented preprocessing approach that creates structured documents from unstructured ones. Another method, called "Text Mining and Information Extraction," is used to summarize documents. In any case, text mining operations form the basis of tagging and thus create entities and relationships (Maimon & Rokach, 2005). Ongoing research continues to develop better algorithms, with one study demonstrating the possibilities of "implementation of information extraction and categorization in the text mining" (Mustafa, Akbar, & Sultan, 2009).

The aim of text mining is to provide a method for knowledge management, analysis, and decision-making. The numerous functions involved in text matter parsing combine to create a text mining algorithm. Mining activities include performing comprehensive searches that result in categorization, summarizing extracted datasets, and monitoring and answering questions based on specific needs. The fundamental objective of a text mining operation is to obtain an associative distribution for words and terms and to identify common significance that can be applied to research or business forecasting (Mustafa et al., 2009).

The most important part of the process is information extraction — identifying words or feature terms within a textual file and processing them through a layered model of the text mining application (Mustafa et al., 2009). Text mining and data mining share the same analytical functions but differ in their use of natural language (NL) and information retrieval (IR) techniques (Maimon & Rokach, 2005).