Single Points of Failure: The Biggest Risk Not on Your Radar

If you’re not actively rooting out single points of failure in your organization, you’re playing Russian Roulette with your productivity, uptime, and profitability. Yes, that’s a bold claim. But I’ve worked with a lot of organizations on a variety of problems, and one thing that work has made clear is that single points of failure are the biggest unrecognized risk most organizations have.

Here, I’ll explain what single points of failure are, how they happen, and how organizations can eliminate them to shore up their long-term viability.

What Are Single Points of Failure? (And Why Are They So Expensive?)

A single point of failure is what it sounds like: any single person whose absence would prevent an entire system (e.g., your company) from functioning. If your payroll person has to be on call during all their vacations because nobody else knows how to get paychecks out, you’ve got a single point of failure.

Another common example: only one person knows how to update the code that underlies some critical part of your infrastructure. If that person retired or quit or got seriously ill, you’d be in major trouble.

Don’t be fooled, though. Even before single points of failure actually fail, they’re expensive. Here’s why:

• They create bottlenecks. When only one person can perform a mission-critical task, the entire organization depends on that person having time in their schedule. This can create major bottlenecks and generally lead to inefficiency.
• They create perverse incentives. Think about it: there’s no better way to create job security than by making sure you’re a single point of failure. But that’s horrible for an organization as a whole.
• They put excess pressure where you can least afford it. If it’s hard for a single point of failure person to take time off, they risk burnout, which may push them to leave your organization. Or if it’s impossible for them to advance in their career while staying at your organization, they may be motivated to look for that advancement elsewhere.
• They prevent innovation. Innovation happens when every member of a team understands how things are done and is therefore empowered to conceptualize and suggest better ways of doing things. With single points of failure, this can’t happen, which means organizations can’t keep up with (never mind get ahead of) their innovative competitors.

In short, single points of failure are detrimental to both short-term and long-term viability. I’ll explain how to eliminate them in a moment; first, though, let’s take a look at how organizations end up with these single points of failure.

How Single Points of Failure Happen

The bad news is that it’s very easy for single points of failure to form. In fact, it’s the default mode for most companies. Unless you have specific guidelines in place to prevent single points of failure, you likely have them.

That’s because every time someone solves a problem, they create new knowledge. If you don’t have embedded processes in place for sharing that new information on an ongoing basis, you’ll build up at least some single points of failure.

The realities of the world we live and work in, too, make it easy for single points of failure to form:

• Constantly changing values and best practices: This is a reality in nearly every field but is most visible in IT. For example, current training programs may focus exclusively on the newest coding languages, which means any organization with infrastructure in older languages becomes ever more reliant on people who know that language. Without updates to infrastructure, these organizations become more and more reliant on fewer and fewer people.
• A lack of operating principles: When deadlines loom, it’s easy to focus on the “what” of a project rather than the “how.” But as soon as two or more people are working on a project, you have to be agreed-upon rules of engagement. Without such rules, you end up with either single points of failure or a scenario where communicating about what’s been done and how it was handled eats up more and more time.
• A desire to create job security: As I mentioned above, single points of failure have excellent job security. Poor communication, secrecy around processes, complexity of processes – the more difficult a person can make their job, the harder it is to let them go, which means they become a single point of failure.
• High short-term costs: There’s never going to be a “good time” to invest in knowledge transfer. That is, there’s never going to be a time when nothing mission-critical is happening so you have spare time to spend on it. But even more so, there’s never going to be a good time for a single point of failure person to leave. And that departure will inevitably be a much bigger long-term cost for the organization.

The important takeaway here is that it’s easy for single points of failure to form. Now let’s take a look at how organizations can eliminate them to set themselves up for long-term success.

How to Eliminate Single Points of Failure

To eliminate single points of failure, organizations have to take proactive steps. These four strategies are a good starting point.

1: Aim for “extreme ownership”

Many managers think that team structures with single points of failure (e.g., a single contact person) are more efficient. In reality, the opposite is true. Instead of top-down hierarchies, aim for “extreme ownership,” which becomes a reality when everyone knows what’s required of both themselves and the entire team for success to happen. (For more on this, see “Teamwork Makes the Technology Dream Work.”)

In practice, this means that teams should approach tasks by asking questions and simplifying requirements. Nobody should start working until everyone has a baseline of understanding. This process is complete when every team member could complete the entire task on their own, if necessary.

When this happens, everyone is accountable for the team’s success, which makes single points of failure unlikely to form.

2: Accept inversion of control

Extreme ownership is the opposite of top-down hierarchies. If your organization relies on such hierarchies now, know that you’ll have to embrace a new way of doing things to eliminate single points of failure. That’s hard – but working with an experienced partner can help.

3: Establish operating principles

When everyone is clear about the rules of engagement – and those rules are designed to create transparency – it’s much harder for single points of failure to form.

In IT contexts, I like to use test-driven development (TDD) and SOLID Principles, which provide a framework for how an entire team approaches projects and ensures that a codebase is continually revised and refined as new things are added. This also means that any new team member can easily understand existing code and how to add to the codebase. It’s a way of working that eliminates the potential for single points of failure to form.

4: Empower everyone to contribute and make a difference

When everyone in an organization understands how things are done, everyone is empowered to find ways of doing them better. This is something every organization today should strive for, as every industry is at risk of disruption from new ways of doing things made possible by new technologies.

Not every idea is good, and not every good idea will lead to measurable business gains, but creating an environment where new ideas are welcomed helps keep organizations vital and competitive as their industry’s landscape evolves.

Eliminate Single Points of Failure to Make Innovation Possible

If your organization doesn’t yet have a process in place to make sure single points of failure form or to weed out those that have formed, you’re setting yourself up for an unpleasant surprise at some future date – likely when you can least afford it.

If the prospect of eliminating single points of failure is overwhelming, don’t worry: Apexon can help. Get in touch today, and we’ll chat about getting to a place where anyone on your team could walk out tomorrow (for vacation or anything else) and your organization would hum along just fine.