Elder Game: Who’s In Charge of Quality?

Who gets to say when an MMO patch is ‘done’? Does QA answer to the live producer? Can QA stop the launch of a patch or does the producer alone have that responsibility?

Every live MMO team I’ve been on has had this debate at one time or another, and with some of them it’s an ongoing power struggle that ripples out and affects development in a huge way.

A head QA honcho that I worked with once told me that my job as producer was to produce — to push the product as spec’d out the door on time. And his job as QA was to make sure that the product I pushed out was acceptable and didn’t reflect badly on the company. That meant that the final call on pushing a patch was his responsibility, he said. He could stop the patch if he felt it was unacceptable.

But while that model of control may work perfectly well for a pre-launch MMO or a one-shot boxed game, it doesn’t make any sense for a live MMO – especially not one with a constant (and regular) content update schedule.

The producer of a live MMO is different from the producer of a pre-launch product. We have different responsibilities and goals. We answer much more directly to the customers. Our bottom line rests less on development costs and more on customer retention.

While the live producer is still in charge of scope and schedule, both the scope and schedule of a live MMO are much more flexible in the short term than those of a pre-launch product. Whether you are patching once a month or every three months, there will be another patch. If you have to, you can slip content or push your patch a week without screwing up a multi-million dollar advertising budget.

As producer, I used QA as a resource (one of the my most valuable resources – no argument there!) to tell me the state of the patch. I used that information to decide what was ready to go and what needed to slip to the next patch. And yeah, I pushed without QA sign-off once or twice. After gathering all the information and balancing all the requirements, that was the action I decided was necessary.

The live producer needs to sit down with the people who are invested in the patch – QA, CS, etc. – to work out any difficulties. But giving one of those groups a flat out veto? Not useful. Not in the live MMO world.