The Relevance of Chaos Theory to Organizations Today

Abstract

Today, organizations of all sizes and types rely on different types of projects to achieve their objectives. Indeed, project management has become a discipline unto itself in recent years and a growing body of scholarship has been developed in response to this trend. Moreover, there are also international organizations such as the Project Management Institute which are dedicated to helping business practitioners improve their project management skills. Although there remain some mixed views concerning optimal project management strategies, there is a growing consensus that successful project outcomes depend in large part on how well a specific project is organized from the outset, including most especially the initial conditions under which it will be pursued. In addition, the initial conditions of a given project also include the respective design and structure of an organization, which are the primary focus of this study. In sum, the likelihood of successful project management initiatives can be improved or diminished depending on the starting conditions under which a project is launched. To gain some new insights into these issues, the overarching purpose of this study is to identify what initial factors are most salient in promoting successful project management outcomes. The basic design of the study is exploratory, drawing on chaos theory which maintains that even minor changes in the initial conditions of a project can have a profound effect on the success of the project’s outcome. While additional research in this area is required, the findings that emerged from this study underscore the need for thoughtful project design and implementation in order to optimize the effectiveness of modern project management strategies. Finally, the study also identified a significant trend towards assigning effective project management a high priority in organizations of all sizes and types, making studies of this nature especially timely and relevant at present.

APPLYING CHAOS THEORY IN ORGANIZATIONAL DESIGN AND STRUCTURE TO IMPROVE PROJECT MANAGEMENT OUTCOMES

Today, businesses of all sizes and types are faced with an increasingly complex and globalized marketplace that demands effective strategies to achieve and sustain a competitive advantage. Indeed, the vast majority of project-driven companies have encountered countless novel challenges to achieving their goals in recent years due in large part to the growing complexity of the environment in which they compete. To identify some viable strategies that can help in this regard, the purpose of this study was to determine what initial factors are most salient in promoting successful project management outcomes. Drawing on the basic principles of chaos theory which maintain that even minor changes in the initial conditions of projects can have major effects on their outcomes, this study used an exploratory research strategy that focused on three main areas: the origins and fundamental tenets of chaos theory, current applications of chaos theory to organizational design and structure and the implications of chaos theory for business practitioners. Finally, a summary of the research and important findings as well as recommendations for business practitioners and suggestions for further study are presented in the study’s conclusion.

Findings

Background and overview of chaos theory

During the early 1960s, a Massachusetts Institute of Technology professor of meteorology, Edward Lorenz, became interested in determining why weather was so difficult to forecast (Glenn, 1996). In fact, accurately predicting any weather patterns beyond a day or two seemed beyond the ability of meteorologists at the time. Based on Lorenz’s original thinking about these constraints and his subsequent experimental inquiries, a veritable scientific revolution which has been termed “chaos theory” resulted which has transformed the manner in which business practitioners conceptualize the problems that are arrayed against their organizations (Oestreicher, 2007). In essence, a chaos theory view of the natural environment in which organizations operate holds that “the world is a nonlinear, complicated and unpredictable system [and] refers to systems which while displaying disorder contain a kind of order hidden inside them, and present disordered, nonlinear, unpredictable behavior in systems” (Namaki, 2018, p. 41). In other words, even when humans are able to discern orderly patterns, there are invariably underlying factors involved that will have unexpected effects on the outcome of a given event.

Other authorities also cite the decades immediately following Lorenz’s original conceptualizations of chaos theory as being particularly formative in shaping its application to modern organizations. For example, according to Millerd (2020):

In the 1970’s and 80’s a new field of research began to emerge called chaos theory. Scientists were looking at complex dynamic systems and trying to understand how they emerge and evolve. They drew inspiration from the natural world, looking at phenomena like how organisms grow in the wild, and how weather evolves. Eventually, they began applying the lessons to fields such as finance, biology, economics and eventually, organizations. (para. 5)

While this incarnation of chaos theory is relatively recent in its introduction to the scientific community, its basic precepts actually date to antiquity. For instance, according to Moshiri (2002), “Chaos theory is rather new in science, but it is rooted in ancients' perception of the world” (p. 29).

Indeed, ancient theorists seemed to intuitively understand that the same survival activities, such as hunting strategies, frequently produced mixed results which made absolute predictability all but impossible. In this regard, Moshiri notes that, “The main idea [of chaos theory] is that although a complex system, such as the world, seems to be generated by a random, and therefore, unpredictable process, it may run by a nonlinear deterministic process” (p. 29). In some ways, like fuzzy logic, chaos theory suffers from a branding problem since its title connotes scientific and technological complexity, but in reality, the fundamental principles of chaos theory are in fact fairly intuitive and readily understandable by ancient and modern humans alike. In sum, chaos theory is a branch of mathematics which focuses on developing a better understanding of different types of complex systems in order to gain some type of analytical insights (Adewumi, Kagamba & Alochukwu, 2016).

This primary focus is based on the inevitability of unpredictable – and unexpected -- behaviors in complex systems that are caused by even miniscule changes in the systems’ initial starting conditions. In this regard, one expert on chaos theory advises that, “Chaos theory is the science of surprises, and not always pleasant surprises” (Schmarzo, 2017, para. 4). It is reasonable to posit that practitioners in most disciplines would actively seek out ways to avoid unpleasant surprises that could derail their efforts, and it is little wonder that chaos theory has been used for a wide array of applications. For instance, Schmarzo (2017) adds that:

While traditional science deals with predictable phenomena like gravity, electricity, or chemical reactions, chaos theory deals with nonlinear things that are mostly impossible to predict, calculate, or control, like turbulence, a bar brawl, the stock market futures, debris flying out of the bed of a truck, or a child darting onto the street. (para. 4)

As noted above, the basic principles of chaos theory are fairly straightforward and include the following:

· The so-called “butterfly effect”: This is probably the most common principle; that a butterfly flapping its wings in one part of the world can eventually cause a hurricane in another part of the world. Here is a more realistic way to describe the Butterfly Effect: small changes in the initial conditions can lead to drastic changes in the results. People’s lives are an ongoing demonstration of this principle.

· Unpredictability: Because it is impossible to fully know all the initial conditions of a complex system in sufficient detail, it is also impossible to predict the ultimate fate of a complex system. Even slight errors in measuring the initial state of a system could be amplified dramatically, rendering any prediction useless or even wrong.

· Mixing: Turbulence describes how two adjacent points in a complex system could eventually end up in very different positions after time has elapsed. Examples: Two neighboring water molecules may end up in different parts of the ocean or even in different oceans. Or a group of helium balloons that launch together eventually landing in drastically different locations.

· Feedback: Systems often become chaotic when there is feedback. A good example is the behavior of the stock market. As the value of a stock rises or falls, people are inclined to buy or sell that stock. This in turn further affects the price of the stock, causing chaotic, unpredictable stock price movements (adapted from Schmarzo, 2017, para. 6).

The latter principle is referenced time and again in the management literature concerning the application of chaos theory to organizations but the other three elements are also relevant depending on the specific circumstances. Nonetheless, the feedback loop that is an inherent component of chaos theory is especially significant since management initiatives routinely receive ongoing feedback from project managers and other stakeholders that can have a substantial impact on the project’s outcome. This consideration also makes it clear that project management is not a static enterprise but rather requires ongoing oversight and thoughtful responses to changes in the operating environment.

One of the more noteworthy findings that emerged from the research concerned the fact that while even miniscule changes can have major effects down the road, major changes tend to have less effect on the outcome when the organizational design and structure is appropriate for the purpose. For example, according to Millerd (2020), The individual behaviors and reactions of people within a complex system are unpredictable, but they are linked to one another. The feedback from each of those unpredictable actions will give feedback to others in the organizations and influence their subsequent decisions and reactions” (para. 7). In other words, the extent that “everyone is on board” with respect to a given project’s successful outcome will likely be the extent to which the feedback loop is responsible for contributing to this success (Millerd, 2020).

Taken together, it is clear that chaos theory represents a valuable tool for modern business practitioners that are routinely confronted with seemingly inexplicable variables that ultimately have an effect on their ability to achieve organizational goals and these issues are discussed further below.

Application of chaos theory to organizational design and structure to improve project management outcomes

Since its formal introduction in the early 1960s, chaos theory has experienced some important changes and expansions on its original tenets which can be summed up, for comparison purposes, as follows:

· Complex systems are unpredictable and disorderly;

· It is an essential process for natural systems to renew and revitalize;

· Small changes in initial conditions create enormous consequences; and,

· Similar patterns take place across layers (i.e., fractal geometry) (adapted from Englund, 2009, p. 2).

Over the past several decades, though, the increased application of chaos theory to various organizational settings has resulted in a new perspective which is called “complexity science” (Englund, 2009). In an organizational context, complexity science also includes the following concepts which have special importance for project management applications:

· Information is the primary organizing force — share widely;

· Develop diverse relationships;

· Embrace vision as an invisible field;

· People have similar needs and corresponding responses;

· Working together is a source of meaning and purpose;

· Establish a shared sense of purpose (Englund, 2009, p. 3).

Although the items in the foregoing list of additional chaos theory-based concepts may appear unrelated, they all share the feature of representing an initial starting point of some sort for any project management initiative. It is therefore essential to ensure that organizational design and structure are aligned with these concepts in order to optimize the likelihood of successful project management outcomes. For example, according to Englund (2009), “Each of these points provides guidance for organizational behavior” (para. 6). Consequently, implementing the appropriate organizational design and structure which takes these points into account represents an essential element of effective project management.

While these considerations are perhaps most relevant for larger companies given the larger number and presumed additional complexity of their projects, the same considerations are also highly relevant for small- and medium-sized enterprises since project success is invariably associated with organizational performance, productivity and profitability. In other words, it is possible to plan for success by optimizing the initial conditions for any project to enhance its likelihood of success (Djavanshir & Khorramshahgol, 2006). As Englund concludes, “You need to create conditions for people to make connections, because those initial conditions provide the idea or practice that could lead to resolving a major issue or inventing a new product or service” (2009, para. 7). Both of these types of outcomes are critically important for growing a business as well as securing and sustaining a competitive advantage in today’s globalized marketplace (Doherty & Delener, 2015). These findings also mean that an informed application of chaos theory to organizational settings can provide a number of important benefits and these issues are discussed further below.

Implications of chaos theory for business practitioners

The implications of the foregoing findings for business practitioners who are searching for ways to improve their project management strategies in order to achieve their organizational goals are multiple and significant. In fact, in recent years, a number of so-called “management fads” have come and gone while business practitioners continue to seek a viable analytical tool that can help them better understand the dynamics of the business environment in which they compete. For example, Levy (2007) points out that, “One of the enduring problems facing the field of strategic management is the lack of theoretical tools available to describe and predict the behavior of firms and industries” (p. 168).

The challenges that are associated with this inability to predict the behavior of firms and industries are further exacerbated by the sheer numbers of actors that are involved in most organizational settings, especially those with far-flung international operations where cultural and time differences can limit effective communications (Levy, 2007). In addition, the unpredictability and volatility of a given competitive environment depends on the type of industry that is involved. In some cases, organizations enjoy length periods of stability with little competition. This type of situation may cause complacency on the part of business practitioners who are rudely awakened to the harsh realities of their predicament when unexpected innovations or new competitors enter the marketplace. In this regard, Levy (2007) emphasizes that, “Even if we know that oligopolistic industries are likely to experience periods of stability alternating with periods of intense competition, we do not know when they will occur or what will be the outcome. Similarly, it is almost impossible to predict the impact of the advent of a new competitor or technology in an industry” (p. 169).

These realities do not mean that business practitioners are at the mercy of the whims of the marketplace, but they do underscore the need for continuous vigilance to identify potential threats to market share or organizational performance. Furthermore, these realities also highlight the fact that many conventional analytical tools lack the robustness that is needed to capture all of the variables that may have an effect on project management initiatives or an organization’s ability to achieve its goals. Moreover, conventional analytical tools and theoretical models tend to assume that the relationships between the multiple actors that are involved in the evolution of industries, including labor, consumers, governmental agencies, financial institutions and so forth are all linear in nature (Levy, 2007).

In this context, an organization’s structure plays a key role in shaping its ability to respond to new challenges in its environment. For instance, Levy (2007) notes that, “Not only does industry structure influence firm behavior, but firm behavior in turn can alter the structure of an industry and the contours of competition” (p. 170). This consideration means that conventional analytical tools and theoretical models such as Bartlett's classification of international strategies or the Boston Consulting Group matrix for resource allocation which rely on a linear assumption about relationships and corresponding behaviors will not generate the level of predictability that is required in the dynamic environment in which organizations operate. As Levy (2007) concludes, “Although these models are based on recurrent patterns that we recognize in the real world, there are usually far too many exceptions for the models to have much predictive value” (p. 170),

Although it remains impossible to determine the precise starting points of many natural and anthropogenic phenomena, by recognizing that systems are chaotic in nature, it is possible to gather as much relevant information about these issues in ways that help overcome the inevitable chaotic behaviors that are involved. For example, according to Glenn (1996), “One consequence of understanding the results of chaos theory is that if we are confident that a system can behave chaotically, then we know that it must have all the properties of chaos. Some of these properties are hard to prove, but we ‘get them for free’ if we know the system is chaotic” (p. 46). This guidance means that business practitioners must recognize that “their best laid plans will often go awry” even when this concession suggests they lack magical predictive abilities everyone wants but no one possesses.

Although it remains impossible to completely mitigate the risks that are associated with the unpredictability of complex systems, it is possible to apply chaos theory in ways that reduce these risks and maximize the opportunities for successful outcomes (Schmarzo, 2017) including the following:

· Build a plan for the continuous testing and refinement of the analytics. The performance of analytic models decays over time; they have a half-life because the world changes such as public sentiments driven by widening wealth gaps, corporate acts of malfeasance, government scandals, rising burden of student debt, growing ranks of the under-employed, wage stagnation, mounting populace divide on social media, legalization of marijuana, rising media sensationalism, climate change, terrorism, Cubs win the World Series, etc.

· Create a dashboard to monitor the performance of analytic models. A dashboard (see example at Figure 1 below) is the perfect monitoring device to ensure that business managers and the data science team are proactively monitoring the performance and state of the analytic models. In addition, consider integrating some data science into the dashboard to predict when a model is starting to fail (i.e., predictive maintenance); and,

· Refine and modify as required. Evaluate and fine-tune the analytic model because these require constant review (adapted from Schmarzo, 2017, para. 10).

Figure 1. Example dashboard for delivery operations manager

Source: Levy, 2007 at https://infocus.delltechnologies.com/wp-content/uploads/2012/01/ Package-Delivery-Operations-Dashboard3.jpg

Further, an informed application of chaos theory can also provide business practitioners with a useful theoretical framework that can be used to better understand the numerous dynamics that are involved among various industry actors, including organizational stakeholders, supply chain partners and competitors (Levy, 2007). In this regard, Stapleton, Hanna and Ross (2006) report that:

When applied correctly, chaos theory shows potential to be a tool that can be instrumental in helping explain why unpredictability occurs within nonlinear systems. A better understanding of this phenomenon may help researchers to develop better, more accurate models to assist managers in making better supply chain management decisions, benefiting organizations and customers by simultaneously enhancing cost?effectiveness and improving customer service levels. (p. 2)

The vast majority of these significant and valuable benefits for organizations using chaos theory are attributable in large part to the fact that “the business climate is a lot like the weather,” meaning that it is possible to predict the near-term future with some degree of accuracy but chaos tends to emerge and disrupt even the most carefully laid plans over time (Safian, 2012, p. 37). Even if project managers do not attribute these disruptions specifically to the tenets of chaos theory, it is reasonable to suggest that they do recognize that complex projects involve a wide range of factors, and changes to any of these factors can have a profound impact on the project’s outcome.

These constraints to developing a better understanding of the business climate, though, can be overcome to some extent by implementing and administering appropriate project management teams that are comprised of different disciplines in order to gain the various perspectives these stakeholders have on the process and outcome. For instance, van de Vliet (2020) points out that the chaos theory analytical strategy is “not only multi-disciplinary but cross-disciplinary: biologists might draw on insights from engineering and physics, for example, while the study of stock-market prices might be related to weather records and other such apparently random phenomena” (para. 7).

This guidance concerning the need for a multidisciplinary and cross-disciplinary approach to effective project management is also congruent with the empirical observations that were made in a study by Yudin (2008) who notes, “In its raw form, [chaos theory] is only reliable for making extremely short-term forecasts. One way to solve this problem is to apply additional analysis methods that have a wider field of view [by] amplifying the original signal” (p. 39). Applying a wider field of view to any organizational initiative therefore represents one of the more important aspects of chaos theory since organizations not only resemble weather in their behaviors, organizations also tend to exhibit regular patterns that can be discerned and used to guide these behaviors towards a common goal. In this regard, Englund (2009) concludes that:

Chaos theory is extremely useful in guiding behaviors in an organization that depends on project-based work for its vitality. The theory informs us that small initial conditions can have a huge impact on project outcomes; however, what actually happens is unpredictable. Nature, while chaotic, follows regular patterns, as does human behavior in organizations. (p. 3)

Although human beings appear to be hard-wired to recognize patterns in nature as well as organizational settings, some patterns may be elusive or otherwise especially difficult to discern in a dynamic business environment where numerous significant and relevant changes are taking place simultaneously, any one or all of which could have an effect on an organizational initiative’s outcome. Fortunately, there are also some steps that business practitioners can take to help them identify meaningful patterns in these situations, including the following:

· Focus on relationships;

· Keep in mind that people respond to energy and enthusiasm;

· Ask questions that engage others;

· Use compelling evidence and vivid language to describe goals;

· Get explicit commitments;

· Remember that people know what is expected from them at work;

· Provide more value to others by giving feedback, observing currencies of exchange and providing learning opportunities;

· Develop common cultural values and leverage differences; and,

· Focus on trust, authenticity, and integrity (adapted from Englund, 2009, p. 5).

All of this is not to say, of course, that long-term planning is inappropriate or a waste of time and scarce organizational resources. Indeed, few projects could even get off the drawing board without some type of long-term plan in place to help guide the process. These observations, though, do mean that business practitioners in general and project managers in particular must remain vigilant concerning any changes in their operating environment that may have an impact on an organizational initiative to ensure that steps are taken to address these changes before they can delay a project’s completion or even doom it to outright failure.

While every organizational initiative is unique in some fashion, it is possible to apply some fairly standardized chaos theory-based strategies to maximize the chances for successful outcomes and minimize the adverse effects of unforeseen events, including most especially the following:

· Reevaluate environmental factors that impact project success;

· Overcome confusion by focusing efforts on purpose, vision, and mission; and,

· Apply specific tools and examples to harness chaos and achieve extraordinary results (Englund, 2009, p. 4).