Expected By A Government Agency — Is Your Business Smarter?

I’m sure by now you’ve heard all the news from LAX.  Our “front line” of border defense, the US ICE (Immigration and Customs Enforcement agency) took our tax dollars and screwed up again.

A U.S. Customs computer outage that stranded more than 17,000 passengers at LAX was blamed Sunday on faulty hardware and an insufficient backup system that left frustrated travelers sitting on planes or standing in long lines.
Saturday night’s delays in screening people arriving on international flights were unprecedented, said Kevin Weeks, director of Los Angeles field operations for the U.S. Customs and Border Protection agency.
The computer malfunction, which began at 2 p.m. Saturday and lasted about 10 hours, came on a peak summer travel day, when nearly 25,000 international passengers arrived at the airport… read more here:

This is inexcusable.  Yes, that’s what I say.  I’m sick of seeing so-called computer and communications professionals abdicate their responsibilities and hide behind the fiction that computer outages are acceptable.

Computers and communication networks absolutely will fail.  They are designed by man(and thus subject to human error) and they are subject to acts of God which happen with no notice.  So since systems will fail from time to time the conclusion is that we, the American traveling public (paying millions in extra taxes and fees to fund the fumblers in the DHS … ICE’s parent agency) must accept outages such as this one.  And Joe’s corporation, doing business and feeding the families of its employees must put up with failures like this also.

The conclusion that failures of components of a system have to lead to outages of the total system is both specious logic and economic claptrap!  Forget about the inconvenience caused by outages.  We’re humans, we all have to put up with adversity at some point in our lives.  Do the math on delays such as this.

• The lost productivity of all the passengers held captive … missed work hours, cascading schedule disruptions due to rescheduled events, etc.  Not to mention the thousands of government employees forced to sit on their asses and do nothing except collect their pay.
• The cost to the airlines in missed flights/profits.  The average passenger may feel, “great it serves the SOB’s right” but someone has to pay for the loss and guess who that will be?
• The perpetuation of the myth that outages are acceptable.  More than anything else this is an educational issue and not a technical one.  The sooner we start education the faster the country will learn.

Especially for an organization such as ICE the solution to avoid this problem happening and likely ever happening again is virtually free … if they wanted to avoid the situation.  Since they have multiple computer systems at multiple geographically separated locations, all within their security perimeter and all owned/leased by the same agency it would be almost trivially easy to back up each airport’s operation on at least one other airport’s system.  The disk drives are there, the processors are there, the connectivity is there and the knowledge is there.

What is not there is the will to stand up and say, “we, as part of America’s front-line of defense are not content to look like failures who depend upon the whim of an electro-mechanical system.”.  Now that commodity I don’t quite know where to buy.

Popularity: 1% [?]

Matt Makes Excuses, Do You?

Here’s a snippet from a recent post on Matt Cutts’ blog.

I had to wait for PayPal to come back up to send some moolah. Every website has downtime now and then; (emphasis added) it’s just a bother when you want to send some money that second…

Sorry Matt but I emphatically disagree.  The entire Internet experience (and perhaps many business’ profitability) would be enhanced if the movers and shakers of the ‘net would just get off this “last century” attitude of accepting failure.  the technology is available to see to it that this does not happen, but as long as we are going to operate in the “failure is inevitable” mode we will get we we expect.

Especially for a huge money handling site like PayPal or a site that serves millions of users, like BlogSpot, perhaps), these unplanned outages should be a thing of the past.  real-time server mirroring combined with geographic diversity of the servers (to avoid single points of failure) are “nickel-dime” in relationship to the money at stake … but we accept excuses instead of progress.

Popularity: 1% [?]