Kevin's Weblog

One of our partners, who shall remain nameless, recently reported that the error rate on their queries to us had gone up by a factor of about 10. After a day or two of our tier 2 staff looking into it without identifying the problem, I was pulled in last night. The first thing I asked was, “What do they mean when they say ‘error’?”.

This morning I reviewed what had been done so far with our tier 2 guys, concluded that they had done everything I would have thought of to do and that the problem didn’t appear to be on our end. I then emailed the partner to say that we suspected that the cause was elsewhere, but that we would be surer if they could give us a histogram of the types of errors they were seeing. About an hour later, they came back and told us that it turned out they had accidentally launched some code they shouldn’t have and that the errors were all 404s as they sporadically queried the wrong URL.

Sigh...