Future-proofing your software
For those of you working in software development, especially enterprise software, how many times have you been told you need to “future-proof” what you are building?
Or perhaps you have been the one making the request to future-proof something? Maybe you have been frustrated by how often the team says “For that feature, it’s going to take us a sprint to go back and extend some APIs to support it”. Or maybe the team starts talking about the process to migrate a database, and how much effort needs to go into doing it correctly.
For those who are in the second group, here is a dirty little secret known well by those in the first group: there is no such thing as future-proofing your software.
Let me re-state that: you cannot future-proof software.
But why does this statement rarely seem to be made? Why does the engineering manager enthusiastically nod his head when the importance of future-proofing comes up, even though it’s not actually possible to plan for how the software will need to behave 5 years from now?
As with many things, the same phrase means different things to different people.
To the engineering manager in the example above, the APIs that the team had to go back and modify were in fact future-proofed! Had they not been future-proofed, it would have taken at least 3 months to make the necessary changes. Because the APIs were well-architected, extending them in ways that hadn’t been originally anticipated was fairly easy to do.
If your organization subscribes to agile software development, you should understand that trying to fully future-proof things is about as anti-agile as you get. I’m talking about actual agility here of course, as it relates to the 4th principle of the agile manifesto: “Responding to change over following a plan”. The manifesto acknowledges that your plan won’t be perfect and that you will need to be agile enough to adapt to necessary changes. If being agile means being ready to adapt while you are carrying out a plan, what hope is there of anticipating things that you don’t even know you need a plan for yet?!
So what does an executive or stakeholder mean when they ask for something to be future-proofed? Do they really expect the engineering manager to accurately guess the software needs of the business 5 years from now?
Chances are good (hopefully!) that they don’t really expect that, and they just want to minimize the amount of rework being done. Being told that a product needs to be reworked for any reason can be frustrating, and the request to “just future-proof everything” is born out of a desire to avoid all that rework.
Some work will always be needed to adapt existing software to support new, unanticipated features – but with care, you can prepare in a way that minimizes that work.
Why we can’t future-proof software
Let’s talk about why the utopia of fully future-proofed software can’t exist.
Firstly, and most obviously, nobody can actually predict the future – so the very idea should be laughable from the beginning. If you look back at your business a few years ago, could you have anticipated exactly where you would be today? No? Well, neither could anybody else – software development teams included. Business needs, customer tastes, and industry directions shift all the time.
Imagine building a house for your family of 4. You want to minimize waste, so you build exactly the house required. Layout, size, number of rooms, everything is designed and built to fulfill the exact needs of your family.
Of course, you can guess what happens next in our hypothetical situation: something changes. A grandparent needs to move in and now there aren’t enough bedrooms; the family falls in love with a dog that they decide to adopt so the back yard is now too small; a bigger vehicle is bought and the garage is no longer sized appropriately, etc.
What should have been done differently? You certainly could have designed and built a house that anticipated every single situation possible. It would have 10 bedrooms just in case, a giant lot with lots of grass, and a 6 car extended garage. Make sure to include an indoor basketball court – one of your kids might show a real proficiency for the sport and need it for practice. Oh, and there’s a chance that one of your kids could be an Olympic swimmer someday, so make sure to have a pool to support their training as well.
You have spent 100 times as much money as was the case with the house designed and built for exactly your use-case, but congratulations: your house is future-proofed! Except even then, new standards and trends and approaches come along all the time and no matter how hard you tried to anticipate every possibility, your house just isn’t going to be cutting-edge after a while.
It seems that the house built to exacting, inflexible specifications wasn’t ideal – but truly future-proofing a house appears unreasonable as well. Is there a happy medium?
As always, every situation is different – but some things a hypothetical future homeowner might think through could include:
- Could they plan an extra room or two that can be used as a small home gym/guest room, or as a dedicated bedroom if the family happens to grow?
- Would a few of the children be able to double up in bedrooms short-term if somebody needs to move in for a while?
- A concrete pad can be placed in the backyard, and portable basketball hoops can be had fairly inexpensively. And if your children never develop an interest in basketball, it can be a nice patio instead.
- If something truly unexpected happens, it’s likely easier to move into a better-suited house when the time comes. Most people move from house to house periodically anyway, why not have that contingency in mind as you plan so you don’t over-build? Save the extra money from not over-building, invest it somewhere, and if you need it you will be ready to support the larger house.
Homeowners aren’t usually trying to completely future-proof their build or their purchase; instead, they are trying to balance value, future vs. present needs, and the potential for life changes that might require a substantially different home arrangement.
When building software, a lot of the same philosophies apply. Decisions are made every day about how your software is built – and they will influence what can be done with that software for the rest of its existence. Here also you must balance future vs. present needs, value, and the risk of major changes that might be needed as you decide how to build your software.
Flexibility instead of future-proofing
Since we can’t truly “future-proof” software, let’s aim for “flexible” software instead. We won’t pretend to know the future, but we can acknowledge that there are ways to minimize how much rework we have to do when it is time to add new features.
Flexible software is built in such a way that it can adapt, without trying to make every part of the system infinitely flexible. There is an art to understanding what parts of the software are most likely to need to change, and then architecting solutions that support those different probabilities. If you operate in a trendy, customer-facing industry, the UI to your software will likely change frequently and your systems should be architected appropriately. A line-of-business application that supports an internal process that hasn’t changed in 20 years? Don’t waste time there trying to plan for UI changes.
Flexible software is also built in a way that the business can grow around it. If there is a reasonable chance that the business can grow 10X in the next few years, make sure that is considered. At the same time, if you are currently a $10M/yr business growing a respectable 10% annually, don’t build as if you are going to have the scale of Google or Facebook anytime soon. You must understand the kind of load the system is likely to experience, but over-complicating your system in order to anticipate scale that isn’t likely to ever arrive is a particularly common (and harmful) type of future-proofing.
Flexible software is also built by trying to anticipate coming changes. What types of customers are coming in the sales cycle? What general types of features are being requested by customers and stakeholders? How many new customers or users are expected across different systems? We aren’t trying to predict the future here, but as long as we are building something we might as well use the best information available to influence the design and production of the software.
Flexible software is kept as simple as possible. Simplicity is almost always harder to achieve than complexity is, but simplicity is a hallmark of well-designed systems. Complexity breeds complexity, and the more elaborate a system is the harder it becomes to adapt and extend that system.
Simplicity implies a lot of things, but it really comes down to focusing on the absolute core of what is needed and eliminating any extra cruft in what needs to be built. This applies not only to the features of the software but their implementation details as well.
Responsibility for flexible software
Building flexible software is the responsibility of the entire company leadership team. It isn’t completely up to the CTO, or the scrum master, or the VP of engineering – it is a responsibility fully shared by every leader at a company.
There are certainly various responsibilities that are held directly by the software team, and those will be covered in a post later. The VP of engineering is unlikely to involve the VP of marketing when making a technical decision about the choice of which database to use, or a new language and framework. While those choices are important, they are only part of the big picture.
The single most important element to flexible software is giving everybody the best view possible into what the future is expected to look like and then making decisions based on that view. In a best-case scenario, the sales team is influencing the software and the software team is equally influencing sales.
As the saying goes, the easiest way to predict the future is to create it.
