Kotaku: So, how has it been, hearing some of the negative feedback from the Beta?
Patrick Bach: The only thing we can do is tell people that: "These are the things we have fixed since the beta." And also based on the feedback from the beta. But other than that… you need to let people play it. The problem with having a beta is that you hand out a product that is not done. Deliberately. You do it because you want to get feedback on not only what the players think, but also on how things are holding up, what can we read from the network traffic, how is everything holding up, how are all of the systems that are brand new holding up. You could argue that maybe we shouldn't do a beta."
It's a double-edged sword.
If we don't do [a beta], then we might have problems day one. So the only way to ensure that we have less problems day one is to have a beta. But if you have a beta (laughs) people will hate you, and think that you're stupid.
We get complaints like, "How can you guys miss obvious things like A, B, and C?" And we didn't miss it. We just weren't done with it. But we had to get it out so that we get results back, so we can fix it. The lead time when you do a beta is actually pretty long. You need to go through certification on consoles, and do a lot of things before you get it out. And since you want to get it out on all three platforms at the same time to avoid further whining… (chuckles)… it takes as long as the longest platform certification time.
"This was a real beta, and I don't think people are used to it. They get the product, and they think, "you have to entertain me," because it's an entertainment product."
How long is that?
Around a month and a half. Which means that when we're done with that beta, we say "Okay, this is it. Now let's go back and finish the game." When we got to the point [six weeks later] that we actually released the beta, the game was in completely different shape. We couldn't really tell people because then we'd have to go through the process again.
It sounds like certification hobbles the beta process somewhat.
Yes.
What's a way to make that better in the future?
One way of fixing the problem is… don't make betas. At least not open betas. Because often when you see betas, sometimes they are actually demos. But they call it a beta, to sound cool. This was a real beta, and I don't think people are used to it. They get the product, and they think, "you have to entertain me," because it's an entertainment product.
And it's like, "Well, it's a broken entertainment product, because we want your feedback." And they say, "Well, I understand that, but it's not polished enough!" And we say, "Well, that's exactly what we're talking about!"
So the challenge is: listen, but don't overreact. Understand that people are disappointed, but that you have to do it.
Or, don't do it. Don't make a beta. At least not an open one.
...