L'Ombre de l'Olivier

The Shadow of the Olive Tree

being the maunderings of an Englishman on the Côte d'Azur

13 April 2009 Blog Home : April 2009 : Permalink

#amazonfail is a Symptom

A regular twitstorm occured this Easter weekend concerning Amazon's failure to rank certain sorts book causing them not to show up in bestseller lists etc.

In consideration of our entire customer base, we exclude "adult" material from appearing in some searches and best seller lists. Since these lists are generated using sales ranks, adult materials must also be excluded from that feature.

Amazon blames the whole thing on a "glitch", which isn't an explanation so much as an admission. As numerous people noted it wasn't "adult" material as a whole which was being excluded so much as non-heterosexual material and this speculative post suggests that the reason for this is some kind of organized bansturbation by the "religious right".

It's obvious Amazon has some sort of automatic mechanism that marks a book as "adult" after too many people have complained about it. It's also obvious that there aren't too many people using this feature, as indicated by the easy availability (and search ranking) of pornography and sex toys and other seemingly "objectionable" materials, otherwise almost all of those items would have been flagged by this point. So somebody is going around and very deliberately flagging only LGBT(QQI)/feminist/survivor content on Amazon until it is unranked and becomes much more difficult to find. To the outside world, this looks like deliberate censorship on the part of Amazon, since Amazon operates the web application in question. To me, this looks like one of two things:

  1. Some "Family"-type organization astroturfing Amazon in an attempt to rid the world of EVIL PRO-HOMOSEXUAL FILTH!!
  2. Bantown
I find that to be highly plausible and, as I say, symtomatic of Amazon as it stands today. There have been other examples such as the TCS Books issue that I commented on yesterday and the automatic banning of certain users. Basically Amazon has hit a scalability problem and a monopolist problem.

Scalability first. Amazon's business secret has been that it allows its customers and partners to help it provide content and advice to other customers and that it heavily relies on computer automation to take this 3rd party provided content and turn it into the stuff that its visitors see. The problem is that with a bit of effort, once the algorithm has been figured out, humans can game the system. This, I believe, is what happened here: a bunch of people figured out that if enough people complained then they could get books marked as "adult" and removed from the general ranking. The catch for amazon is that the problem almost certainly highlights a fundamental data-structure assumption that now needs to be corrected and thus I predict that a bunch of Amazon programmer serfs are now going to be pulling all nighters to correctly implement "safe search" as opposed to the current solution because the current method as explained in the top quote is clearly not acceptable.

I suspect that the TCS issue is partly the same in that TCS books gets extra Amazon brownie points/recognition etc. for the breadth of its inventory. This means that TCS has a large incentive to claim that it has titles that it doesn't have and it also knows that Amazon has no way of detecting these fakes unless a visitor reports them. And making this sort of a report can be tricky as it isn't always clear what the right channel is.

This leads us onto the banning users issue. One of Amazon's problems is the difficulty one now faces of talking to a real person if one has a problem that can't be resolved by a call to their low level customer service folks. If you are a publisher you have better access but even so you can't get to a real human being much of the time and a regular customer, particularly one who has been "banned" finds it next to impossible. I suspect the reason for this is a deliberate choice by Amazon because otherwise they would have to employ a lot more human beings in various support roles. This adds costs to Amazon's business model without directly increasing turnover and thus it isn't going to happen. Unfortunately Amazon is now so big and so influential that its every move is analysed by the great unwashed.

Furthermore because Amazon is so big and influential - it is the major sales channel of many publishers - there is a huge incentive for people to game their system. It doesn't matter whether the people gaming the system are (as it seems in this case) the religious right or whether they are the loony left, other evil corporations or someone else. The problem is that because Amazon is such a large storefront if a group can game the system to remove things they disagree with or inflate the rank of things they do agree with it can seriously affect the sales of the products in question.

This is the start of its monopolist problem. Amazon, like Google, has tended to be seen as mostly benign but events like this give it the PR black eyes that erode that "mostly benign" perception into "just another nasty monopoly". Since Amazon, particularly with regards to the Kindle, is indeed acting like a nasty monopoly (see Kindlepid.py) this should not be surprising but it does leave amazon with a potential problem. The problem is twofold.

Firstly because it is so influential it needs to invest heavily in humans to oversee the computers and react when things go wrong. These humans need to be reachable when needed and be empowered to speak for the company without legal obfuscation or other buck-passing. They also need to be able to implement policy to make changes to resolve the problems identified. This, as I noted above, will add significantly to Amazon's cost base without directly increasing sales - in fact the business case is more that these people are needed to stop a decline in sales which is always a hard sell inside a company.

The second part of the problem is that "nasty monopolists" are treated differently by their customers than nice companies are. People tend to behave well towards companies that they perceive as treating them well and fairly and they tend to abuse companies that they perceive as ripping them off. If Amazon becomes generally perceived as a nasty monopoly then I predict it is going to have a lot more people putting them in column B. Not only will this likely reduce sales as people look for alternative suppliers it also seems likely to lead to customers feeling that it is acceptable to scam Amazon and take advantage of offers of "free shipping" and "free returns" to borrow stuff for a short while. Amazon will no doubt detect these scammers and start banning them but this, in turn, will lead to more negative publicity and the vicious circle will continue.

Update; More on the underlying architectural problem here