Future impact of Linux as a viable server

¶ … future impact of Linux as a viable server?

Linux is on its way to becoming the dominant server platform. It currently has a large installed base, which is growing faster than any other server operating system. Linux is being deployed on inexpensive commodity hardware to replace proprietary UNIX systems and Microsoft Windows. Linux is also starting to be used on larger, more specialized server hardware; Linux 2.4 supports 16 CPUs on x86-based servers, with up to 16 gigabytes of memory. Linux 2.6 expands that support to 64 CPUs and 64 gigabytes. The technical improvements in Linux 2.6 allow Linux-based products and non-commercial software compilations to match or exceed the performance of proprietary server operating systems. Non-commercial compilations, such as Debian and Gentoo are especially noteworthy, as their use results in a fundamental shift in how software is obtained and managed. Linux 2.6 offers a number of significant improvements over 2.4. More filesystems are supported, including XFS, which is notable for its performance and scalability. Larger file and filesystem sizes, up to 16 terabytes are supported. The default process scheduler now uses an algorithm that is unaffected by the number of running processes, giving a performance boost under high load. Preemptibility has been added to the kernel, optionally allowing the process scheduler to run and interrupt a process during a system call, reducing latency. Linux 2.6 adds support for POSIX threading, with greatly improved performance over LinuxThreads. The number of devices that can be represented has increased by several orders of magnitude, which is important for servers connected to large storage arrays. Support for IPSEC is built-in, making it easy to create secure, encrypted networks that any protocol can run on top of. Support for additional network filesystems has been added, including CIFS, which will improve integration with Windows networks.

Traditionally, operating systems for servers are supplied by the server manufacturer or purchased from a third-party software company. These methods are the most common way Linux is acquired for servers, but a third method is gaining in popularity; non-commercial compilations. (Wilson) Debian and Gentoo are two popular non-commercial Linux distributions that are slowing the growth of the server market as they increase in usage. Non-commercial distributions generally differ technically from commercial operating systems. In general, they tend to provide the system administrator with more control, for which they find favor among experts. The most common differences are the variety of packages provided with the distribution, and the ease of customizing the installation. Generally, non-commercial distributions come with far more native packages than commercial ones do; Debian currently includes almost 16000 packages. (Wilson) the reason for this is simple; commercial vendors want to keep their systems as simple as possible for customers in order to reduce support requests, and make the system usable by inexperienced system administrators. Commercial operating systems usually install a base system suitable for the type of use the vendor anticipates. In most cases, the system is usable with only a minor amount of adjustment. In some cases, the system is completely inappropriate for the intended use and must be rebuilt. Non-commercial distributions usually make the process easy by offering the option to install a simple base system, with nothing unnecessary included. (Wilson) Third-party commercial support is available for major non-commercial Linux distributions, making them a more viable option for business use.

Native support for a single filesystem, SGI's XFS has been added. XFS, along with IBM's JFS, which was added to late versions of the 2.4 series as well as 2.6 offer better scalability and performance for servers than previous Linux filesystems. ReiserFS, generally the preferred choice for performance on 2.4 performs well for small filesystems filled with small files, but cannot keep pace with XFS and JFS for large filesystems or files. (Meredith 26) the maximum size for a filesystem has increased from 2 terabytes to 16. Support for large file and filesystem sizes means Linux can more easily be used for large databases and storage systems. The performance increase offered by XFS and JFS translates directly in to higher load capacity for most types of servers, as IO latency is often the worst bottleneck on a server. For general purpose use, XFS is probably the best performing filesystem available. (Wilson)

Another addition to the filesystems in Linux 2.6 is extended attributes for all major filesystems. Extended attributes can store a variety of types of metadata for a given file. (Meredith 29) a common use of extended attributes is access control lists. The traditional Unix and Linux means of access control is to specify read, write and execute permissions for a file's owner user, owner group and everyone else. This simple permissions model is a rudimentary access control list that handles basic access control fairly well, but makes it difficult to express more complex permissions, and nearly impossible without root access to create groups. (Wilson) POSIX ACLs allow a user to make a list of exactly who is allowed to do exactly what to a given file. (Wilson) One common example would be to give a large group of users permission to read a file and a smaller group permission to modify it. Modern servers need to be able to enforce any combination of user-based access controls imaginable, and POSIX ACLs provide a way to do that.

Sometimes simply controlling which user accounts have permission to access a file in a given way does not adequately address all security concerns. Access controls based on the program being executed as well as the user executing it limit the damage a security flaw in a program can cause. Linux 2.6 supports type enforcement, which allows access controls to be enforced on programs as well as users. Type enforcement allows the administrator to specify which combinations of user and process may access files in a given way. (Wilson) Type enforcement mitigates the problem of an all-powerful superuser account, which certain processes must be run from in order to accomplish their required tasks. An example of how type enforcement might be useful is controlling the actions of a script that starts the SMTP server. The SMTP startup script must be run as root, but should not be able to modify the password file. Type enforcement can restrict the startup script to only accessing the files it requires, even though it is running as root, which would contain the security breach in the event that the script was compromised.

The process scheduler in Linux 2.4 has two major performance issues. Every time a task finishes, the scheduler runs and recalculates the priority and next timeslice for each process. The time this process takes is directly proportional to the number of processes in the queue; what computer scientists call an O (n) algorithm. (Marshall) the scheduler can only be run by one processor at a time, which can leave other processors sitting idle while there are waiting tasks in the queue. (Marshall) the problem grows geometrically worse as the number of processors grows. This is clearly a suboptimal way to use expensive multiprocessor hardware. 2.6 solves these problems with a completely new scheduler. The new scheduler maintains separate runqueue for active and expired processes. When a process uses up its timeslice, its priority and next timeslice are calculated and it is moved to the expired queue. When the active queue is empty, the queues are switched. (Marshall) the new scheduler requires the same amount of time to run regardless of the number of processes running; an O (1) algorithm. The O (1) scheduler leaves more CPU time available for getting real work done under heavy load with many processes. The new scheduler also has separate runqueues per CPU, eliminating the problem of each CPU locking the runqueue while it runs the scheduler, greatly improving performance on large multiprocessor servers.

Most operating system kernels do not allow the scheduler to run during a system call, even if the process making the system call has exhausted its timeslice. A lengthy system call executed near the end of a timeslice can result in a process using far more CPU time than the scheduler has allocated it. A race condition during a system call can hang the entire system. Linux 2.6 allows the scheduler to run at preset preempt points during system calls. The result is lower latency and improved stability. Preemptibility is intended more for desktop and embedded systems than for servers, but reducing latency and increasing stability is potentially beneficial for most server tasks as well. Kernel preemptibility is still new, and may have undesired results, so it is offered as an option.

The threading system used by Linux 2.4, called LinuxThreads has a number of problems. It is not POSIX compliant, which makes porting software between proprietary Unix and Linux more difficult. It scales poorly and it does not work well on multiprocessor machines. The new Native POSIX Thread Library offers an eight fold increase in performance. (Deshpande) the new thread library is better, but may not be as significant an improvement as it seems. Threading is not as popular or useful on Linux and other Unix-like operating systems as it is on other systems. Threads became popular on operating systems that have high overhead for starting new processes. Starting a new process on Linux has fairly low overhead, so use of multiple cooperating processes is usually a simpler approach. (Raymond Chapter 7) Threaded applications are generally more complex and perform worse than those than use multiple cooperating processes to split up tasks. Having more options is never a bad thing, however, and some Linux programs do use threads to split up tasks and gain improved performance on multiprocessor systems. The new threading model should provide a significant performance boost for these types of applications, especially on multiprocessor servers, provided the applications are compatible with NPTL; it is not backwards compatible with LinuxThreads. The performance improvement is a strong incentive for authors of threaded applications to rewrite them to take advantage of the new threading model.

Linux 2.4 cannot address enough devices for certain applications involving large multiple device storage arrays. It can only support 256 major device types with 256 devices each, for a total of 65536 devices. Devices being split in to types, this means that Linux 2.4 can only connect to 256 SCSI disk drives at a time. A larger number of disks may be required for a server managing a large storage array. Linux 2.6 supports 4096 major devices with 1048576 minor devices each, for a total of 232, or over four billion devices. (Meredith19) it is unlikely that anyone will want to connect a single server to more than 1048576 SCSI disk drives at a time in the foreseeable future. The IO model is also improved with regard to large systems. Previously, Linux could only handle IO requests to block devices such as disk drives by breaking them up in to 512 byte sectors. 2.6 allows larger IO requests and larger sector sizes. The result is improved performance for applications involving large IO requests, such as databases. (Meredith 20) the maximum size of a block device has increased to 8 exabytes, though filesystems are limited to 16 terabytes, and nobody manufacturers an 8 exabyte disk drive.

IPSEC provides encryption for all network traffic over IP, regardless of application protocol. This makes any network effectively private. Modern servers need to be able to communicate over public networks securely. Previous attempts to secure communications involved adding support for encryption to individual applications. IPSEC provides a standard, uniform means of securing network traffic. Applications no longer need to support or even be aware of encryption to communicate securely. It is reasonably common that an application required by a user does not support encryption, but does transmit information that should not be accessible to anyone other than the intended recipient. IPSEC solves the problem, providing security for all network communication. Third-party support for IPSEC existed in Linux 2.4, but could not be assumed to exist on any Linux-based system. Linux 2.6 has native support for IPSEC. An important application on a server might be allowing remote access to a file share on a networked filesystem that does not have support for encryption.