Desirable to Separate the Technical

¶ … desirable to separate the technical issues involved in creating and managing data warehousing from the political issues (i.e., the distribution of costs and benefits to different components)?"

To What Degree Is it Possible or Desirable to Separate the Technical Issues Involved in Creating and Managing Data Warehousing From the Political Issues?

The idea of separating the technical issues of data warehousing from the political ones is by no means new. The reason why it is still discussed, however, seems to be that no one has yet been able to do it properly. Data warehousing is very important for almost any company, as individuals that work in information technology (it) and other areas where the ability to access large amounts of data is growing very rapidly. Almost all companies now need to be able to get to most or all of their data at virtually all times, and in order to do that there are two things in play: the way that the data warehouse is designed and operated (including back-up plans), and the kind of data that is stored in it (including how that data is linked and what kind of data can be found in relationship to other data).

From it departments, to banks, to schools, to medical clinics, there are countless areas of life where the need for data warehousing is growing. Not all data warehouses are well-designed, although most of them operate along the same basic principles. The idea of having access to this data ties in with high availability, which companies see as the opportunity to access large amounts of needed data very rapidly, regardless of what might be taking place at the time. This is used more often than people realize in everyday life, and it becomes increasingly important when disasters strike. Disasters are not the focus here, however, as it is more significant at this time to look at data warehousing and whether the technical issues of it (how it is created) can be separated from the political issues of it (who it benefits and how cost is distributed). This can be confusing, because all of the departments will use the data warehousing to find information. Some will use it more than others, but the cost of creating it and operating it remains the same.

Data Warehousing - the Technical Issues

From a technical standpoint, data warehousing can be very difficult to create. These warehouses have to have a lot of infrastructure in them, and the sheer volume of data can make the thought of creating them strike fear into the heart of even the most seasoned it professional. Once they are finally created, they can also have their problems. It is important that they be tested periodically in order to ensure that they are working correctly, and this is not always an easy take, either. It can be costly and time consuming, and that turns off a lot of businesses that would otherwise not object to it. When data warehouses are not tested and not kept up-to-date, though, serious problems with their integrity can arise, especially when there is some problem with power or other systems. Whether the data is correctly backed up and stored properly so as not to be lost is important, but not all companies think about this and the implications that can come from it.

The main reason that there are so many technical problems with data warehousing, however, is that it is such a complex thing to do. Most engineers do not have a good understanding of just how difficult data warehousing is, which is why so many data warehousing project fail. Not everyone hears about them, because they are generally swept under the rug, but they do fail at a high rate. There are a lot of reasons for this, and on the technical side the complexity is only one of those reasons. Other problems include the components that are used to actually set up the system. It is very easy for these components to fail or to not work properly with one another. The wrong components can be purchased, or the system can be designed incorrectly from the beginning. It is not just a matter of linking computers to one another and letting everyone share. There is much more that must go into the volumes of data that are stored in a 'warehouse.'

When it comes to looking at everything from a technical standpoint the complications are certainly there but there is also a simplicity in that it is only technical. Once the political side of the issue is included it becomes more complex and it can also raise the aggravation level for those who are trying to create the system and for those who are trying to use it. Unfortunately for the people who are working to create the system, there is really no way to separate the technical issues from the political issues, as much as they would like to. Whether the technical and political issues should be separated is certainly something to consider, and there are many opinions on it, but if they really cannot be separated the question of whether they should be becomes largely moot.

Data Warehousing - the Political Issues

In addition to all of the technical issues that are addressed when data warehousing takes place, there are political issues as well, such as who is using the warehouse and who is paying for it. It would make sense that whoever used it the most should be the one paying the most, but it does not always take place that way. Instead, some of the departments have a lot more money allotted for them, and some of them have more people in them than others. In addition, there are a lot of companies where one small department uses the data warehouse a lot more than a few other, large departments. How much they can afford to pay for the data warehousing is not up to a department and how much they will use the warehouse, but is instead up to the company and the liquidity that it has or does not have.

Another political concern that data warehousing often comes up against is that, like children on a playground, a lot of people in companies with departments do not really want to share with other departments. It may seem petty and childish, but it is a fact of life and a fact of business. If one department is responsible for coming up with most of the data, that department might not want to just hand over all of its hard work to the rest of the company. It would certainly be for the good of the organization if the department would do this, but there is often animosity between departments. There are perceptions that one department works much harder than others, or that the people are nicer, or that the company benefits more from it than from others. This may or may not be true, but it is clear that the entire company will benefit from being able to access all of the data on all of their customers.

The departments will certainly turn over the data because that is part of their job, but there is another problem that falls into both the technical and political arenas, and it is one that shows how difficult it could be to separate them. This is the problem of how the different departments have collected their data and how they have stored it. As a company grows it will add different departments, add people to those departments, and add computers and other technology to those departments as well. By doing this, there might be a lot of people in the company who are working with different computers and different software.

Not all of their data collection methods and software will be the same, and they might not all work together with one another. They are not going to want to change over to a different computer and different software and a different method because of the hassle involved, and the other departments likely feel the same way. It can make it very difficult for those who are working on a system for a data warehouse to try to synchronize all the departments without making a lot of changes that may not be well received.