tfosorciM.org

Achieving Effective Beta Programs

The issue of how to achieve effective participation in beta programs has come up on a couple of beta programs that I am currently participating in.

For NDA reasons I won't be sharing what the companies are or what products are involved. Interestingly, however, the issues that arise about how to achieve an effective and productive beta program are not dissimilar in the two situations.

It seems to me that a fundamental problem is that software companies are software company-centric rather than customer-centric or, specifically in this context, beta tester-centric.

Underlying that are questions about why beta testers beta test. What is the spectrum of motivating factors for beta participation? And how do those motivating factors interact with the factors that mitigate against actually spending time beta testing and, the only measurable factor, reporting bugs, design flaws, crashes, confusing interface and so on?

One key issue is making things that can be made easy for the beta tester just that i.e. easy.

Time wasted on installation, perhaps because of poor installation instructions, is time that cannot be used for productive beta testing.

In addition, wasted installation and configuration time is a potentially significant demotivating factor. The thought pattern might be something like this, "If [Company X] cannot be bothered providing a beta that is easy to install or provide good instructions about installation why should I bother spending my time with this heap of [select epithet here]?".

It happens. And I believe that software companies need to address those issues. Not that it is easy. The sofware developers are busy people under pressure too. But beta testing is crucial to achieving software quality.

Clumsy interfaces to report bugs are another time consuming, demotivating factor. Each unnecessary, time consuming step to report a bug will, I believe, reduce bug reporting rates. The result, very likely, is that some bugs will make it through to the marketed product and all users suffer.

How do software companies keep beta testers motivated? Remarkably, this seems to me not to have been seriously thought about. At least there is very little evidence of action to reflect previous serious thought about the issue.

There are only two basic approaches - carrot and stick. Stick doesn't seem applicable, so what carrots can software companies offer to improve beta test activity and results?

I suspect, and it may be heresy, that software companies will have to pay beta testers if they are serious about improving beta testing activity and quality.

After all, beta testing isn't a hobby. At least in my opinion it ought not to be a hobby.

Would better beta testing reduce product support costs? I don't know but it may. Would better beta testing improve customer satisfaction? Almost certainly.

A serious look at improving incentives to beta testers is, I suggest, a no-brainer. The detail of how best to improve those incentives needs more careful thought.