Beta Testing Considered Harmful

Joel Spolsky warns about the dangers of large-scale software beta testing. He calls it "shipping early and often":

Does ship-early-and-often really work for a huge company doing massive PR pushes that's going to get millions of people checking out their early release?

I don't think it does. This is a classic example of what I've always called the Marimba Phenomenon. The Marimba Phenomenon is what happens when you spend more on PR and marketing than on development. “Result: everybody checks out your code, and it's not good yet. These people will be permanently convinced that your code is simple and inadequate, even if you improve it drastically later.”

As usual, Joel singles out Microsoft when he could just as easily be talking about any large software company. He also adds a plug for small companies (like, for instance, his small company). And he doesn't mention big company Beta success stories. For example, the Google Mail Beta has generated lots of positive buzz for Google.

However, by and large, I agree with Joel. Software companys would do well to heed his warning.