Optimizing the Software Velocity vs. Safety Tradeoff
Jason Bloomberg of Intellyx challenges the assumption that all code must be tested before deploying it to production.
Optimizing the Software Velocity vs. Safety Tradeoff
“If everything seems under control, you're not going fast enough.” ― Mario AndrettiWhen we drive our cars, safety is paramount. We moderate our speed, use our mirrors, and drive defensively. If conditions require us to slow down, then we slow down.
Unlike legendary racecar driver Mario Andretti, our goal is to get to our destination as safely as possible.
For Andretti, however, the goal is to win the race. Safety is but a means to an end, as crashing is a surefire way to lose.
This contrast between the two extremes of automobile driving has a close analog in software development.
For software, safety refers to reliable, bug-free code. Sometimes safety is paramount, like with bank transaction processing or satellite software. In such situations, delivering code that is optimally reliable is the main goal, and if it takes more time to deliver it, then so be it.
In other situations, software velocity is a top priority, for example, with web-based companies or digital offerings in general. These organizations’ competitiveness – and thus, their survival – depends upon delivering changes to code quickly.
Both perspectives are valid, as they both focus on managing the risks inherent in software development – the risks of delivering broken code vs. the risk of delivering code too slowly to meet the competitive requirements of the business.
What, then, is the best way of trading off velocity and safety? Once we answer that question, then another question becomes paramount: how can we improve both velocity and safety at once? Understanding the tradeoff is one thing, but we really want both at the same time.
After all, that’s how Mario Andretti won his races – and lived to race another day.
Shifting the Velocity/Safety Balance Point
The only way we’ll avoid the pitfalls inherent in trading off velocity and safety is to manage software development risk across the board.
At some point, the development organization reaches the optimal tradeoff. Conventional wisdom says that this tradeoff is the best that development teams can achieve. After all, that’s what we mean by ‘optimal.’For modern development organizations, however, settling for this optimal tradeoff simply isn’t good enough. They want both better safety and higher velocity – at the same time.
The only way to shift the optimal velocity/safety balance point is to change the underlying assumptions that lead to the conclusion that this tradeoff is the best a team can achieve.
Specifically, the assumption that must change is the assumption that the development team should test all code before deploying it into production.
In other words, we’ve always assumed that testing in production was unsafe. Now we’re saying that under the right conditions, it’s safe enough.
Given today’s emphasis on software velocity, we must reconsider whether it makes sense to test everything in every iteration, thus slowing down the process – or to forego testing in some situations and deploy untested code into production.
Deploying such code requires that developers carefully consider which code they should deploy without testing and how to manage the risks inherent in such a decision. There are many variables to consider: existing CI/CD processes that typically include automated testing, as well as code reviews, varied environments, and other considerations.
Once again, the challenge becomes a balancing act. How should developers prioritize which code to test vs. which code to deploy without testing it first? Given untested code is more likely to cause errors, how should developers find and fix those errors?
Rethinking Quality Assurance
To answer these questions, developers and their managers require insight into their quality assurance activities. With a tool like Faros AI, developers can gain critical insights into testing effectiveness, impact, and performance metrics that indicate how well quality assurance can impact the business while also pointing out areas of improvement.
Engineering managers can then assess various quality metrics for their teams’ applications and repositories. Working with their teams, managers can help make testing more effective.
Instead of erring on either side – running too few tests thus leading to too many errors vs. running too many tests thus slowing down the development process – the right data provide the necessary insights so the development team can focus on the testing activities they should perform to maintain the optimal tradeoff between velocity and safety.
Code coverage is an important source of data for this optimization, as some code will remain untested at various times. If errors do crop up, they are more likely to come from untested than tested code.
It’s important, therefore, for developers to leverage code coverage to understand which code has been partially or fully covered to avoid the same or similar errors from cropping up in the future.
Only via such careful, proactive management of untested code can development organizations shift the optimal tipping point between velocity and safety, thus improving both velocity and safety over time.
The Intellyx Take
Organizations not only tolerate issues in production, they expect them – and leverage them to deliver even greater software velocity. Today’s developers are indeed following in Mario Andretti’s footsteps, giving up some safety in exchange for greater velocity.
However, it is important for developers to remember Andretti’s hidden message: give up too much control, and you crash and burn. Avoiding issues that adversely impact users of the software can undermine whatever competitive advantage software velocity promised.
The result is a reconsideration of the nature and importance of software quality. In the past, balancing velocity and safety has been an exercise in compromise. With insights from tools like Faros AI, development teams can rest assured they can optimize this tradeoff – without slowing themselves down.
Copyright © Intellyx BV. Faros AI is an Intellyx customer. Intellyx retains final editorial control of this paper. No AI was used to write this paper.
Jason Bloomberg, Intellyx (Guest)
Read the report to uncover what’s holding teams back—and how to fix it fast.
What to measure and why it matters.
And the 5 critical practices that turn data into impact.
More articles for you
Engineering Leadership Framework: Vision, Strategy & Execution Guide
See what Faros AI can do for you!
Global enterprises trust Faros AI to accelerate their engineering operations. Give us 30 minutes of your time and see it for yourself.