Can we establish a self-adaptive “Matrix” in reality?

The Matrix went on-screen in 1999. It was a huge success and a milestone in film history. In the movie, the virtual world made by the software is known as the Matrix. It is so enormous that it contains countless programs for various functions. The programs can find the bugs in the system, delete obsolete components, and help the system self-update. In short, the Matrix can adapt and react to changes by itself without help from any programmer. However, if you had asked any software engineer in 1999, they would have considered such self-adaptive functionalities for software as a childish fantasy. Now, in 2018, things may have changed. Can we build a self-adaptive software like the Matrix today?

Historical development:

Self-adaptiveness is a kind of artificial intelligence (AI). It detects, decides, acts, and evolves. Self-adaptive software has been pursued by software engineers for a long time. The primary reason to pursue self-adaptiveness is the need to minimize human supervision, configuration, troubleshooting, and maintenance because they take on huge costs. Especially in recent years, software structures have become more complex than ever, as we try to use them to solve more problems in real life. This complexity results in unexpected system vulnerability. Well-known recent examples include Intel’s Spectre and Facebook’s account takeover. They showed that even for the big companies with big teams, engineers may not find major problems before pushing products onto the market.

Self-adaptive software may be the solution, as it is expected to improve the system by itself with minimal human maintenance. The concept of self-adaptive software can be traced back at least a few decades. Peyman Oreizy, in 1999, proposed the famous concept of self-adaptive software as a computer program that adapts to changes in its operating environment by changing its self-behavior when a better behavior is available to improve performance. Since that proposal, many efforts have been made towards the development of self-adaptive software. As can be seen from Figure 1, the number of publications related to this concept increases steadily over the years. In 2017, the number of publications increased by 1300%.

To achieve the self-adaptiveness, researchers first established the theory. Horn proposed that the software needs to achieve self-configuring, self-healing, self-optimization, and self-protection. Hinchey and Sterritt, and Parashar and Hariri further pointed out that all four of these achievements should be established on self-awareness. These “self” properties now serve as standards in the field. After the theory development, researchers moved on to consider the software structure. Among many researchers, those at IBM contributed significantly at this stage. The basic structure of a self-adaptive software program is now recognized as a feedback loop to monitor operating environment changes, detecting symptoms, making decisions, and taking action.

Many challenges still exist in realizing the software structure. One major challenge is perhaps to take action at run time, which requires a totally different architecture design from the most current software on a fundamental level. To understand the difficulty, you may think about how the software upgrade process, regardless of whether Windows, iOS, or Android, including the operaturation system, needs to be stopped before making any functional changes. Many engineers choose to face the challenge by adding an independent component responsible for the the adaption, but there are still engineering difficulties. Other challenges include monitoring and detecting requirements for initiating changes, testing and evaluating adaptations, and deciding the level of human interaction. Of course, the advances in self-learning based on neural networks and other deep learning algorithms achieved in the recent years will assist in solving these challenges in the future.

Current applications:

Self-adaptive software provides a practical way to cut costs and increase system reliability. After years of development, we are now starting to see many self-adaptive software applications. The applications spread across many fields, including software engineering, artificial intelligence, control theory, and network and distributed computing. Table 1 lists some of the latest examples.

Table 1. Examples of self-adaptive software.

Most of the applications are still closely related to IT industry, but we believe we will see more applications in more traditional industries, such as the car industry, the oil and gas industry, and the aerospace industry, in the near future. In fact, an Israel-based company called Aurora Labs recently released a vehicle software solution utilizing self-adaptation. Nowadays, as vehicles become more digitized and dependent on computers, software becomes more critical to ensure normal operation and safety. The software from Aurora Labs claims to proactively search for software defects, fix them before any problem occurs, and ensure a smooth driving experience. We believe this example, as well as the others in the table, represents a trend in software development. Self-adaptation will gradually become a regular component embedded in any software.

To summarize, we still may not be able to build software as self-adaptive as the Matrix described in the movie, but we are getting there.

If you have any questions or would like to know if we can help your business with its innovation challenges, please contact us here or email us at solutions@prescouter.com.