PLEASE NOTE: “As an Amazon Associate, [Need Coffee] earns from qualifying purchases." You know we make money from Amazon links,
and I know you know this, but they make us say it anyway. More info, click here.

#AmazonFail: Welcome to the Lesson of 21st Century Damage Control

Failure Types: a chart

So I’m not going to write directly about the running debacle that is #AmazonFail yet (get your primers here and here). For two reasons. One, because I submitted a request to Amazon for a response and I said I would give them until end of day tomorrow before I did a write-up. And I think we’ll get a better answer than “glitch” for this mess. Or we’d better. And two because somebody’s put forward a very interesting theory that “glitch” could be a euphemism for.

Right now, I want to talk about something else. I want to talk about what every company in the world needs to learn from #AmazonFail, whatever the reason turns out to be. I’m sure this will be a topic of great interest in business blogs, but you don’t have to be a business expert or strategist or whatnot to know what happened here. The guy with the English degree can tell you. That’s how plain it is.

[ad#longpost]You have to have someone minding the store. Because I watched #AmazonFail go from a few mentions to a shitstorm in a matter of hours. From something that could have been handled to the type of PR nightmare that will take forever to get past, if you can ever get past it. And also it became something that nobody in business wants: an opening for your competitors.

Ken and I were talking about this: I think you could have contained this a few hours ago. I know everybody’s pissed (I’m pissed for multiple reasons) but I think the shitstorm could have been contained if you had come out with a response that could be believed and was transparent. If you as Amazon simply acknowledged the problem and said that it was a problem and would be corrected and the root cause determined and addressed ASAP, then that would at least be something. Not a note stating this was policy and then another note saying it was a glitch. Nobody is going to buy that. But saying it was wrong and we’re fixing it–a lot of people will still be pissed but that’s a lot better than trying to come up with stories to cover up something that’s way past the cover-up stage.

Ken was right when he said that if you do business online then are open for business twenty-four-seven. You can’t close. You have to be ready to respond to something like this before it gets out of control. I would take that a step further. I would say if you’re a major corporation, even one that’s not known for having an online presence–like Burger King, just to pull an example out of the air, which has websites but is known for real world hamburgers–you have to have somebody standing watch in case #burgerkingfail kicks off. And they can sound the alarm.

The other lesson is this: lame ass excuses aren’t going to cut it. Once the alarm is sounded, acknowledge the problem, find the problem, and then state what the problem was. The problem is not that there are problems–people get pissed off but at the end of the day I think most reasonable people know that shit happens. The problem comes when there are problems and nobody acts or reacts accordingly. The fact that the CTO of Amazon is on Twitter and has given no sign that there’s an issue–not good. Just adds fuel to the fire.

It’s a lot easier to prevent a fire than to put one out that’s out of control. So to all the companies out there: put somebody on standby. Hell, if you can automate it so that any variation of your name rising to the top of any social media system’s most chatted about topics will ping somebody who can check on things, that might be good. But personally I think somebody somewhere has to be minding the store and with processes in place that allow them to respond. Even if that response is to wake somebody up who has the authority to actually respond directly to the crisis, that’s something. Because you don’t want to have your company be the one attached to the word “fail” a few hundred times a minute, whether it’s your fault at the end of the day or not.

Image. Found on a blog post that was what came up when I image searched for Amazon and Fail. But talks about technical issues on Amazon Web Services.


  • They are open 24-7. There are people there, right now, taking phone calls from angry people like me.

    ‘Course, that’s just their call center slaves. Anyone with a higher rank than that can’t be bothered to work on a holiday, after all.

  • Hunter: Perhaps I should have been more specific. *Functionally* open. If this is happening to your company, you might as well be closed. So. :-)

  • So how would this relate to problems on YOUR website, Widge?? If I enable scripting (which I USUALLY have enabled, but don’t at the moment, because I’m here) I get a very strange repeated string of “#”s and (some sequence number, the same one) and “more” repeated over and over across the top of my browser window, and my IE window freezes up hard enough that even the Microsoft error message asking for permission to “phone home” about a fatal error gets stuck…
    But if I turn off scripting, all that goes away and I can get on with what I laughingly call a life, more or less amusedly.

    Do YOU have anything doing some kind of “quality watch” on your product performance? Would you notice if/when your rotating ad-servers suddenly start throwing up the occasional “trojan’d” response by anti-viral products?? I’ve actually had that happen on at least one other site which REFUSED to even consider the possibility, and threatened legal action if I kept “spreading rumors” while attempting to warn others about it…

    Don’t recall having any similar problem with your site in the past (well, other than one of your server sites that I finally put into my HOSTS file because its response time was soooo slow).

  • flared0ne, first of all, I’m a bit surprised at your rant there. I didn’t expect a kind of Spanish Inquisition.

    Anytime someone reports a problem with the website I move to address it as quickly as I can and as best as I can, considering this is just me, and unlike Amazon or Burger King or any of the companies I had in mind, I am just me and don’t have staff standing by to take complaints that come up. This whether it’s somebody trying to give Scott shit for writing about mime ministries or me pulling an all nighter to keep the server up when it’s having an issue. And I do all of this myself (apart from the content which is not attributed to me and even that I have to edit) and I make very little money off of this website, frankly. I’m surprised at the number of people who think we’re some big company and make truckloads off of all this.

    As for your problems with IE if you have reported that before, then I apologize I haven’t addressed it. However, I will say that I don’t recall that issue being brought up before and I don’t experience the same problem on IE that you do. And I would recommend you move to a more secure browser like Firefox if you are indeed having issues.

    I’m sorry if that isn’t a good enough “quality watch” for you, amigo. But this is a free website. And if that’s still too much for you to pay to come here, then…well…sorry we couldn’t live up to your expectations. Hope you find some place that does.

    I apologize if this response is not what you were intending to invoke. But when I get an accusatory comment like this, this is how I respond. So. If this wasn’t what you had in mind, then perhaps less capital letters and keep the question marks singular next time. Just a thought.

  • I only recently started using Twitter, and I wasn’t entirely sure about its purpose, but now I know. #amazonfail has been impressive to watch, and I like the fire analogy. Calling it a glitch after earlier calling it policy really was like throwing gasoline onto the fire and I have no idea what they will try next to put out the fire.

  • Email I sent to Amazon:
    I would very much like to know why books with no adult content, such as “Heather has Two Mommies” – non-fiction books such as “The Fenway Guide to Gay, Lesbian and Bisexual Health”, and classic works of fiction like “The Well of Loneliness” (which, if any of your staff had read this book, would know that there is no sexual activity described within) have had their sales listings removed? Until this situation is rectified, I will not spend one red cent with Please note that your customer base is not ignorant, and it is downright insulting when your corporate communications director tells the LA Times that this was a “glitch”, but your staff is telling impacted authors it is because their work has been deemed “adult”. Either your communications director is lying…or your staff members are. Which is it?

    T***** F******

    Response I was given by Amazon:

    Thanks for contacting us. We recently discovered a glitch in our systems and it’s being fixed.

    Thanks again for contacting us. We hope to see you again soon.

    Please let us know if this e-mail resolved your question:

    If yes, click here:
    If not, click here:

    Please note: this e-mail was sent from an address that cannot accept incoming e-mail.

    To contact us about an unrelated issue, please visit the Help section of our web site.

    Best regards,

    U**** S
    We’re Building Earth’s Most Customer-Centric Company

  • I should note that after writing the above email, I got more information about the meta data on some of the books impacted. While that may explain a computer glitch, it does not address the fact that Amazon is maintaining radio silence. If it was bad filtering, then the Corporate Communications Director would have had a response ready to the LA Time’s question.

  • Great overview, and well said… I agree, even if this is a benign mistake on Amazon’s part (Which I largely doubt, since someone had to take that metadata and exclude “gay and lesbian”, from an assessment by another blogger), but they have been criminally sluggish dealing with it. Very, very stupid.

    I expect the response to kick in around 9 or so, or 6 on the west coast.

  • Hey, no inquisition intended — I will admit that I had just a little earlier opened ten or fifteen windows onto various of your contests, not expecting them to all go “belly-up” and lock up my system for half an hour or so… Which stepped all over my “regularly scheduled routine” and etc and etc.

    So when I came back looking for a venue to pass the info along to you, finding an ongoing discussion about “mechanisms for monitoring quality-of-service” etc seemed like a natural “lay it out here” spot. Granted, it came out a little more energetically than sober consideration might warrant – sorry. Some of that was also just “in memorium” of the hassles I had with that website I mentioned, where “spreading the word about a problem” drew an awesome amount of denial and threats of litigation — which still makes me bristle.

    Oddly enough, most websites never seem to have “in case of serious degradation of performance, click here” emergency-services feedback-entry points. And the real alternative of having someone sit there mindlessly exercising different functional paths: that’s why software exists, to avoid brain-burn and still be able to flag “weird response in testing loop XYZ, not matching expected signature” to some responsible adult on nominal guard duty…

    “Responsible adult in-the-loop”: the only way to handle the totally unpredictable.

    That link to their planning presentation was quite informative – kinda established a mark to shoot for. Ooops, missed. Okay, try again. Have to wait and see how much credibility they burned through… And “this too shall pass”.

  • flared0ne: No worries…I think the problem with Holy Shit the World is Ending contact lists is that they’re either on the site that you can’t access or they are, and thus open to being abused. In a small operation like mine, the only thing that I find amazing is that somebody *couldn’t* get a hold of me. Between Twitter, e-mail, Facebook, MySpace, any number of places where I frequent, I’m kind of easy to find. And I have been found, no doubt. But yes, you have to have a plan and responsible adult supervision. And if One Tusk ever becomes a mega-corp, I assure you we’ll be covered. :)

  • This sort of thing is nothing new, really. I mean the PR debacle caused by leadership that has gradually come unclued.

    As an example in somewhat recent history, two words: “unintended acceleration.”