Home » People & Culture » Fault Tolerance

Fault Tolerance

It’s been a long week, already.  I need a little break, and this piece from 3 years ago is still very relevant.  

You pay the insurance bills every month.  Car, home, life – they’re all about the same, a bet against yourself that you actually hope is money wasted.  But when things go wrong, like a drunk driver smacking into you one sunny day, it’ll be there when you need it.  If you listen to the commercials, what you get for your money is peace of mind.  It should help you sleep at night without anxiety.

Insurance is just the ultimate form of taking care of when things go bad.  Building fault tolerance into a system so that it never gets that far is a far more complicated and thoughtful process.  Anyone who designs a system of some kind – a physical thing or a process that involves checks and balances – is probably going to be proud enough of their achievement to not want to think about when things go horrible wrong and the whole thing breaks.  But that’s exactly what needs to happen for it to be truly robust.  It’s also something that a culture or society has to think about ultimately, painful as it may be.

Cracks usually appear at the boundary.

Cracks usually appear at the boundary.

A recent post in Barataria on Boundary Failures generated more side conversations via email than just about anything else I’ve written. It struck a nerve with many of you.  What surprised me more than anything was how these conversations came in private, as if failures in the systems of our government, society, and life in general are something to be ashamed of.  More likely it’s a topic that most of you didn’t think you had anything to add in public even though you’ve had it in the back of your mind for some time.

It’s OK, we’re family here.  We can talk.  I put out a lot of partially baked ideas, you know.

It’s easier to imagine this situation with physical things, such as cars.  It seems a bit counter-intuitive but the best way to design a safe car is to make it so that when you do get smacked the thing more or less flies apart.  It absorbs the hit so that the people inside are safe.  Engineers are often trained to think this way and some take on the task gleefully.  It’s fun to wreck stuff and see what happens.

Social systems are not as much fun to wreck.  To some extent, the ongoing assault on public employee unions in several states is a reflection of this same problem.  Systems were built up over the years through collective bargaining that were supposed to handle all the bad times in one holistic method.  Right now, they’re taking a huge hit – and some people want the unions to simply fly apart and take the blow.  Is that really the right way to handle it?  Probably not, but a system that is inflexible will tend to be more brittle than one that has fault tolerance built into it.  The opinion that these unions are inflexible has led to the opinion that they have to be destroyed for everything else to carry on, at least among some people.  How it plays out through these tough times is still an open question.

Things often get knotty.

Things often get knotty.

It’s far from the only example.  A global economy has very little tolerance for unrest in nations that have strategic resources like oil.  The system that was rather deliberately put in place encouraged dictatorships because they appeared to put a lid on social upheaval and manufactured a sense of stability.  In the short term it appeared to work, but it should have been obvious that this is not a fault tolerant system – when unrest does strike hard everything has to shut down for weeks or even months, making things worse.  Democracy is going to be inherently more fault tolerant than any strong arm.

The financial world also had remarkably little fault tolerance built into it as well.  The assumption that risk had been eliminated actually increased the risk because it encouraged bad behavior that tested the limits.

How do we as a society build fault tolerance into our world?  It has to start by accepting the fact that any system, physical or social, is only as good as when things go horribly wrong.  That means facing our own destruction, our own death, at a time when we’re full of life and energy.  It’s not something humans do naturally.

strong half-step back is often the first place to start.  Our interdependent world requires us to be even more aware of what can possibly go wrong especially as the systems of our life become so complicated that few people understand them.  Participatory democracy requires that we all stay plugged in, especially when it looks like there’s no reason to worry about a thing.

But please, let me know what you think now that we’ve seen a lot of things start to break down in ways we never even tried to anticipate.  We can talk.

3 thoughts on “Fault Tolerance

  1. My gut says that the cost of an economic system that tolerates shocks and screwups is way more than most people are willing to pay. Plus the people who game the system are the real problem and they are the ones who should pay.

    • That’s where regulation is so important. But a system set up that is hard to game could have FDIC-like systems all through it, and the insurance charges to make that possible should pay their own way.

Like this Post? Hate it? Tell us!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s