Managing software complexity through intent-based programming

Uri Sarid
We simply demand too much from the current approaches.

Software is losing its magic. We simply demand too much from the current approaches. As a result, software developers are losing the battle with complexity, often without realizing it. More often than not, little failures pile on top of other little failures, and life for the consumer, as well as the business, becomes more frustrating rather than easier.

For example, Apple's products have become buggy, travel is still a nightmare and call center experiences make us doubt both artificial and human intelligence.

To bring the magic back into software, developers will need to stop walking systems through each step to ensure the desired outcomes. Instead, as systems grow more voluminous and complex by the minute, developers will need to make software itself more autonomous through the use of layers, intent-oriented algorithms and artificial intelligence (AI).

If we take a step back, it’s not a huge surprise some of the magic has been lost. We’ve greatly raised our expectations of software, while simultaneously broadening our expectation of who should be able to command it. More and more is expected to “just work” automatically, and more of us expect to be able to control the automation of our digital life and work.

It’s hard enough to meet these expectations when the problems the software is addressing are static. But this automation is also expected to address real-time needs -- ones in which the parameters change rapidly, even while the automation is executing.

Car navigation from point A to point B is hard enough in the face of traffic, weather and construction. But what about optimizing for passengers’ phone meetings during the commute, or for physical and digital commerce along the way? How about doing that for millions of cars simultaneously sharing the same roads? How about doing that for trips that combine cars, trains, planes, hotels, restaurants and more.

These factors are starting to mandate a different programming model: a declarative one. In this alternative model, we declare an intent — a desired goal or end state — and the software systems autonomously figure out how to simply "make it so." Humans set the boundaries and constraints, but it’s too much to expect humans to always figure out the solution to get there. So, computers step in and do it themselves.

The new programming model will leverage computers to solve their own issues of complexity.

An illustrative analogy is well understood in the corporate world: management by objectives (MBO). A strong MBO approach has employees being told which objectives they'll be measured by versus how precisely to achieve them. The objectives might be centered around sales results, customer engagements or product adoption. It's then up to the employees to figure out the path to get there. It’ll often require adapting, as conditions change unexpectedly, and learning in the process so it gets easier over time. So, in a sense, this alternative programming model is one that manages software by objectives: MBO for machines.

Examples of this need are everywhere. One of the hottest areas right now is bots, or interfaces that accept voice or text commands. While today's bots are often command-oriented (e.g. find Jane Doe on LinkedIn), they will need to become intent-oriented (e.g. find me a great job candidate).

Imagine you need to hire a new sales representative, engineer or CIO. Rather than sitting at your computer and scouring the web to source talent, you interact instead with an intelligent chatbot that does the nitty-gritty work for you. Behind the scenes, the chatbot is linked to an API that orchestrates candidates from LinkedIn and Glassdoor, enriches their information using GitHub and Meetup and reaches out to them to gauge interest and fit. Once a suitable candidate is located, the chatbot connects you two to get the ball rolling. Over time, the chatbot learns which candidates work out, and gets better at sourcing them. While it sounds futuristic, this hiring method is possible today with the right orchestration of existing software.

We can look to the way our own built-in computers (our brains) process images to gain lessons in how software can solve complex scenarios at scale:

  • In the first layer, light is absorbed by photoreceptor cells, which do minimal processing and pass signals to the second and third layers of the retina.

  • In the second and third layers, neurons and ganglion cells work together to meet the goal of detecting edges or shadows, and send findings into the brain through the optic nerve.

  • In the visual cortex, there are more layers: one layer calculates where objects are in space; another layer detects and processes edges to put together shapes; an additional layer turns these shapes into recognizable things, such as faces and objects. Each layer learns over time and improves its performance.

  • Eventually, the last layer actually matches these faces or objects against the person’s stored memory bank, and he or she recognizes or doesn’t recognize the people and objects.

This approach, where each layer is responsible for just one goal and where the goals become increasingly more sophisticated at each higher level of abstraction, allows software to be intent-based and solve complex scenarios at scale. In the machine world, the layers come in the form of APIs freeing up critical data; composite services orchestrating data from disparate systems; and artificial intelligence making smart decisions at every layer.

This is the future of software, and that future has already started in modern, massively distributed cloud computing systems such as Google’s Kubernetes and its rich ecosystem; in autonomous vehicles, both terrestrial and aerial; and, of course, in artificial intelligence and machine learning, which are seeping into every layer of our increasingly digital world.

A paradigm shift is inevitable, because the mounting complexity brought on by an explosion of co-dependent systems, dynamic data and rising expectations demands to be solved at its core. The new programming model will leverage computers to solve their own issues of complexity and allow humans to do what they do best: choreographing the outcomes.