One Failure, One Success

The customer should not have been the alarm

There is something particularly painful about a checkout failure. Someone has already done the work of finding your product, understanding what it does, and deciding it is worth paying for. Then you put a dead end in front of them.

It would have been completely reasonable for this person to close the tab and move on. They did not owe me a bug report, a screenshot, or an explanation of why I wasn’t getting their business. They were trying to buy something, not volunteer for quality assurance.

Instead, they took the time to submit feedback. I really appreciated that, and I gave them 50% off as a thank-you. They gave me an opportunity to fix a problem I otherwise might not have known about.

That is also the uncomfortable part. I know about the person who told me. I don’t know whether anyone else encountered the same problem and left without saying anything. A failed checkout can look an awful lot like a person who just decided not to buy, unless you are actually catching the failure.

Mine wasn’t being caught. Whatever confidence I had in my tests and monitoring, this was a fairly unambiguous demonstration that they had missed something important.

The email I needed to notice

My feedback form sends me a brief email notification telling me there is feedback to review. It is a little blurb, not the full feedback itself. To find out what someone has said, I need to notice that email and go check.

I had overlooked it. But that notification showed up in my Premail morning briefing, and I immediately saw that there was feedback I needed to act on. I went to read it, learned about the checkout problem, and got to work.

This is a distinction worth making: Premail did not detect the 404. It did not inspect my checkout, diagnose the bug, or somehow replace the monitoring that should have caught it. The customer found the problem. Premail surfaced the notification that prompted me to go read what they had reported.

That was exactly the sort of thing I wanted briefings to do, and it happened while I was still getting used to using them myself.

Using the thing I just built

I added briefings to Premail only a few days ago. I’m trying to train myself to stop defaulting to my email client and instead use my briefings as a summary of what needs my attention. That is a habit change for me, not just a feature release.

Opening an inbox and figuring out what matters are two different activities. I can look at email without noticing the particular message that should change what I do next. In this case, a small notification about feedback needed to become an actual task: go read this and respond.

The morning briefing made that connection for me. I saw something I had overlooked, understood that I needed to act, and acted immediately. There wasn’t much ambiguity about whether it had been useful.

It is one thing to build a feature because you believe it will help people. It is another to have it pull something important out of your own email and change the course of your morning. I would have preferred a less embarrassing example, but this was a pretty convincing one.

Fixing more than the checkout

The checkout issue is fixed now. I also fixed the alerts, infrastructure alarms, and tests that failed to catch it. Getting the page working was necessary, but stopping there would have left me relying on the next helpful customer to tell me when something went wrong.

I’m not going to declare that nothing like this can ever happen again. I put effort into preventing it the first time, and it still happened. The goal of these changes is to prevent a repeat and, if it does happen, make me aware of it before a customer has to report it or a potential customer quietly gives up.

The lesson is certainly not that a morning briefing is an acceptable substitute for testing and monitoring. “Wait for someone to complain, then summarize the notification” would be a terrible reliability strategy. I needed to fix the systems that let the problem reach a customer in the first place.

But I also got to see Premail do something genuinely useful after those systems failed. It brought an overlooked message to my attention, and that message led directly to a fix.

The checkout failure doesn’t become less embarrassing because my own app helped me find out about it. Both things are true: I let a customer hit a broken checkout, and a feature I had just built helped me respond. I’m grateful the customer bothered to tell me, and glad my morning briefing made sure I noticed.