Progress in software engineering and software testing

Progress & Obstacles in Software Testing

Software testing is a process that some may overlook in a software development life cycle (SDLC). As how the processes of analysis and development are critical to an SDLC, it is important to consider that the success of the operation of a software relies on the accuracy of software testing. There are many instances when a system or software fails due to unanticipated cases. Such situations thus result to interruptions in activities that rely on the software. For instance, the continuous flow of a company's business operation that is involved in sales may be suspended for some time if the ordering system it uses fails, causing customers to stop ordering products. It can be a factor that a faulty or incomplete software testing may have caused such problem. Here, therefore, it is apparent how software testing is critical to the success of software. Fujiwara and Yamada (2002), in their study C0 Coverage-Measure and Testing-Domain Metrics Based on a Software Reliability Growth Model indicates the following as further implications that a faulty software testing my cause.

Therefore, once system failures due to defects or faults latent in the software system come to the surface, the computer system is entirely useless and many people incur great damage. Further, there are also defects or faults latent in the software system, which may cause serious and critical accidents to a human life.

In the history of software development life cycle, the process of software testing is improving continuously. Although there may be barriers to the process of testing, alternative measures that can substitute software testing problems are being taken by many development teams. The following pages will try to explore the progress and obstacles in software testing. As part of the discussion on software testing improvements, the different kinds of testing procedures and tools will be included. On the other hand, for the obstacles part, the different kinds of problems encountered by development teams in SDLC as well as the applied solutions will be explored.

Progress in Software Testing

The process of software testing in the early years of software development is perhaps the simplest method among the software testing methods that we have as information technology grows older. Before, to test a software would mean executing the program and checking if there are errors that will exist, either run-time error or data error. These days, on the other hand, software tests would include different instances that a software can possibly encounter.

The existence of computers for some decades ago has changed a lot of things in software development. There are different levels of programming nowadays. There are the low-level languages and high-level languages. Moreover, there are different styles of programming such as object-oriented and procedural styles. Such changes in software development cause the need for improvements in software testing.

One of the improvements in software testing is the use of various CASE tools. In the current market, there are many CASE tools that can be used in software testing. To list them all will require time. Frequently, software testing teams used different CASE tools in their projects. The advantage of which is that the result of testing does not depend to only one kind of testing process, thus, providing more probability that a software is thoroughly tested. Fujiwara and Yamada (2002) indicates how CASE tools are commonly used.

Generally, a software development manager is performing the testing-progress management by using various CASE tools. These CASE tools support testing activities of Windows applications or embedded software systems. Then, by using various testing coverage-measures, the testing-progress management can be recognized visually.

It is normal for a software developer to develop different modules based on requirements and specifications. The process of setting a goal for software developers does not end in the completion of program coding and development. Instead, the process of software testing is considered by many as the most crucial part because it is in software testing where the success of coded programs depends. According to Yamada and Fujiwara (2001), software testing affects the different part of a coded program. They indicate that software developer has to test to verify the implemented functions based on its requirement specification. We use many various test-cases for testing. Then, there is a set of the modules and functions in the software system to be influenced by the executed test-cases.

The set of different modules is termed by Yamada and Fujiwara (2001) in their study as the testing-domain. In their study, one progress in software testing has been indicated. That is, the use of SRGM or software reliability growth model where the reliability of a software is based on the number of errors that occurred based on a given time.

Finally, the one improvement in software testing that can be considered as the most important is the process of testing not just the software but the other related elements to the software as well.

This includes the people and environment around the software. Because of the new innovations that are born in information technology, softwares do not live on its own. There are other elements in the different networks that can affect a software, even if the software itself do not need integration with other components. Hence, this process of analyzing and ensuring the security of softwares is nowadays included in software testing. According to Kunal Panchal (1999), successful testing program tests people, process and technology, despite several vendors offering to sell you "silver bullets" that simply don't exist in the software security-testing world.

Another improvement in software testing is the use of automated testing from object models. A research and study, conducted by Robert Poston (1999), made use of object modeling technique (OMT) for the process as a comparison to automated testing. The object modeling technique was basically used for object-oriented programs where testing must be built for each module of a program. To do such testing, the specifications of each module are used as basis for the test cases. In his study, Poston (1999) found the usefulness of automating tests in object modules. According to Poston (1999)

The OMT life cycle was shortened significantly through the use of automated testing tools such as a test case generator and a test execution tool. The OMT model and the automated testing tools were successfully used to build an automated price-calculating system called the Customer Service System.

The test generator is the tool that provides automatically generated test cases. Information must be provided to the test generator for it to generate test cases. After the creation of the test cases, they are ready for execution to an instance of situation in the software that is similar to the situation when the information for test cases are gathered. One advantage that automated testing presents to a development team is that it minimizes the cost of repeating the testing process, as well as the process of rebuilding and integrating modules due to confusions as to where errors occurred. Poston (1999) indicated that the availability of these already integrated tools enabled the team to measure the costs and benefits of re-using object models for testing without having to deal with the sometimes confusing cost and productivity factors associated with doing the integration, or building and maintaining tools.

Bugs and errors in software can sometimes cause dismay to users and may affect the market value of a software product. Michelle Lang (2004), in her study, however discussed a solution to this that companies like Siemens Energy and Automation (SEA) and UGS PLM Solution Software implements to their software products for testing before they release the final version. The latest progress in software testing is the use of alpha and beta testing. According to Lang, alpha and beta testing can benefit both the users and the developers. The process of alpha and beta testing goes like bringing out sample softwares to users for them to assess and evaluate the product. After which, they will send feedbacks for any potential changes that users need or errors that they encountered. Lang (2004) explains this further, stating that,

These alpha and beta testers, both new and experienced, provide the developers with raw feedback on potential changes and improvements. And while the software industry obviously benefits from the immediate customer feedback that beta testing provides, the incentive for potential beta testers may seem less apparent.

Problems in Software Testing

If the process of software testing earned different improvements in the history of software development life cycle, so is the increase of obstacles that hinder the progress of other software testing methods. Several researches and studies found that finding the right methods that would be perfectly suitable to testing a software product is one of the most common obstacles to make software testing a 100% success. Because testing requires a thorough and meticulous examination of a software, to allow a seamless operation of the software product, many development teams aim to develop a first-rate testing methods. However, despite of the new testing methodologies and tools that are available, it is inevitable for a software to encounter unanticipated errors during its operation. Thus, providing development teams more problems during the testing process.

Lee J. White, in his study Domain Testing and Several Outstanding Research Problems in Program Testing indicates that in the area of program testing, there are several significant problems which need to be addressed.

White indicates some of these problems include the following.

The determination of a scientifically sound basis for the selection of test data

The development of program specifications which can be used to both generate test data and also ascertain the correctness of program output.

The development of relationships between program testing and formal verification.

Jorgensen and Erickson (1999), on the other hand, focused on software integration as a cause of problems in software testing. Such integration presents problems because new test relationships need to be established between the integrated modules. Unlike in a per module software testing, which limits a test on the specific functionalities of a module, testing integrated modules require an establishment and a collection of relationships of the different modules.

Moreover, Jorgensen and Erickson (1999) indicated that object-oriented programming has been a trend in coding for some years now, yet there are times when the process of software testing is done in a traditional manner. Their study examined the traditional styles of testing and they suggested several enhancements and modifications. According to Jorgensen and Erickson (1999), an object-oriented style of programming is based on improving the structure of codes and data resources. Object-oriented introduces a condensed yet efficient code structure. However, if a development team would use the same style of testing, as it is being done in a traditional manner of testing, the objective of achieving success for the software product may not happen. This is because softwares are seen by customers based on their behavior, how they function and work, and how they make things automated. Jorgensen and Erickson (1999) stated that Software testing is fundamentally concerned with behavior (what it does), and not structure (what it is). Customers understand software in terms of its behavior, not its structure.

Software testing involves the development of test cases that will must the structure of a software roduct to ensure that every functionalities and components are tested. Hence, it is important that the architecture and structure of a software is clear and well-organized. However, during the process of software development, there were times when the developers find it hard to solve some coding problems. To enable them to quickly solve difficulties, oftentimes, the developers code the solutions in any style as long as it allows the software to run without errors, even if the solution causes the structure of the software to become formless and destructed. Thus, such spoiled structure of software architecture sometimes makes it hard to design test cases that ensures the excellent reliability and performance of software products. Having this view on software testing, Richard McDonell (2003), indicates that Designing efficient test systems requires a modular software architecture and development tools optimized for test. To develop test systems faster and more cost effectively, it is critical that you evaluate your test software architecture to maximize code reuse. Understanding the importance of modular test-software architectures and how to develop your tests as modules rather than building stand-alone applications will significantly improve test-software reuse.