For the past 200 years, our economy has been defined by industrialism. Companies have strived to maximize scalability and efficiency. They have engaged in top-down planning and implemented hierarchical reporting chains. They have thought in terms of product-centric value chains, and sought to maximize value production through centralized control.
Now, though, our society is transitioning from industrialism to post-industrialism. The product economy is giving way to the service economy. Siloed experiences give way to digitally infused ones. We have entered the age of accelerating disruption, where incumbent businesses are ever more quickly harried by new competitors who compete by changing the conversation.
These economic shifts are manifesting themselves as organizational shifts from complicated to complex systems. A complicated system may have many parts; the relationships between those parts, however, are hierarchical, fixed, understandable, and predictable. A car is a perfect example of a complicated system. Few of us know how to fix one, but we all understand that a flat tire is unlikely to cause the carburetor to fail. If you are driving, and you turn the steering wheel to the left, the entire car will go left. Parts of it will not wander off in different directions according to their own whims.
A complex system, on the other hand, consists of many independent agents, all of which have dynamic, shifting relationships with many other agents. Complex systems appear in forms as diverse as a flock of birds or an economy. Complex systems tend to be sloppy and failure-prone, yet resilient. If a flock of birds encounters a windmill, it will split into pieces and fly around the windmill, then regroup on the other side. If an airplane tried to do the same thing, it would fall to the ground and crash.
Cloud computing is a major driver of the transition from complicated to complex business systems. The cloud confronts customers with ecosystems of cooperating yet independent service vendors. A car with a built-in Internet radio service, for example, may be part Honda, part Pandora. A website may mash up content from Amazon, Google, and Doubleclick. A SaaS invoicing service may run on top of a PaaS platform, which in turn runs on top of an IaaS cloud. In order to understand where your data is, and whether it’s safe, you may need to understand the entire complex system of vendor relationships.
Cloud computing also generates complexity within companies. So-called “social business”, using collaboration tools such as Yammer, enables employee relationships that cross, and sometimes disregard, reporting chains. Facebook and Twitter force companies to provide customer support in public, using forums they don’t control. They extend complex relationships beyond the previously well-understood, controlled boundaries of an individual company.
Complex systems present very different management challenges from complicated systems. They generate counter-intuitive behaviors such as emergence and cascading failures. These behaviors make comprehensive modeling, prediction, or control infeasible. Instead, it is necessary to manage them in ways that maximize their resilience. Fail-safe strategies give way to safe-to-fail strategies. All activities become experiments. Decisions about whether to continue an experiment depend on feedback. Feedback requires permeable boundaries. Complexity redefines the very nature of a company.
The post-industrial age of disruption challenges businesses to shift from being change-averse to embracing change and innovation as core competencies. Companies are adopting a variety of adaptive practices, such as Lean Startup, Agile, DevOps, Design Thinking, and Cynefin, among others. These practices are the spiritual inheritors of Cybernetics, developed in the 1940’s by Norbert Wiener and others. Wiener was investigating problems related to control. He observed that, in many cases, improved control required improved communication. Whether you were trying to control the temperature in your house, or the aim of a turret gun trying to shoot down an enemy plane, or the movement of your foot while you were walking, you needed continuous external feedback in order to determine the accuracy of your actions.
Feedback loops enable efficient adaptation to changing market conditions. For them to work, however, they require the ability to listen, learn, and adjust. If the enemy plane changes its evasive strategy, you need to change your targeting strategy. Bolting an innovation department onto an otherwise industrial-style control mechanism is likely to fail. The entire organization needs to adopt an attitude of continuous learning. Employees need the ability, not just to execute a given practice, but to explore, question, and change it. In environments characterized by complexity, best practices are always provisional. They are only “best” for a limited and changing set of circumstances.
Enterprises can’t achieve pervasive agility without a new, post-industrial approach to IT. The 21st-century economy shifts IT’s essential mission from providing stable, efficient information infrastructure to creating a cybernetic substrate for the entire business. Cloud computing, continuous delivery, and DevOps all contribute to creating this substrate. Just as with the rest of the business, however, IT needs to value continuous learning over best practices. It needs to focus, not on executing a chosen methodology, but on continually optimizing its ability to help the business learn and adapt.
Scrum, for example, is an Agile software development methodology that includes regular retrospectives. Retrospectives are intended to provide feedback about how well the process is working, along with an opportunity to make adjustments based on that feedback. Unfortunately, many Scrum practices limit themselves to superficial, “we need more donuts on Monday mornings”-level feedback. They congratulate themselves for holding retrospectives, without examining their effectiveness.
The simple fact of executing “the right” methodologies, or using “the right” tools, does not guarantee a successful 21st-century IT organization. IT must transform its mental model, and learn to think beyond methodologies. Instead of asking itself, “Are we following the practice?”, it must begin to ask itself, “Are we adapting more quickly and accurately?”, and more importantly, “Are we helping the business adapt more quickly and accurately?” In the age of complexity and continuous disruption, the ability to process feedback trumps the ability to execute a plan.
