The hugely scientific console reliability poll

[DuckThor Evil]

Okay, so 1up reported 75% of their launch consoles to be broken. You think thats a statistically correct number? You think that worldwide, 75% of all X360's are broken? Its whats called an abnormality. If 75% was the case, Microsoft could just close up the X360 production, and instead, buy every gaming division Sony owns including the PS franchise, for the money they would have lost on repairs.

No, but if everything is abnormality then what is normal? We get these pieces of information from multiple sources saying the exact same thing, yet nothing can be considered as real evidence. Why don't you go ahead and go find a source that claims even remotely something like this.

If i where to poll 100 people left and right on different forums, i would sooner or later encounter a group of people with 100% working consoles.

The fact is that Shiftys poll is not the only source in the internet saying the same thing that echoes across the whole damn internet, the box has had horrible failure rate and almost everybody has realised that by now. BTW it's funny how at one hand you try to act like Mr. Spock with uber logic and then on the other hand you make weak assumptions like the supposed hardcore nature of Beyond3d posters and the effect it would have on the failure rate. There is no evidence that we play more than the average EA customer. And the X360 installed base contains at least 2 million early adopters even when using a very conservative definition of the word, so it's not like we are the only early adopters of the world.

Also many reports indicate that eventhough the problem is most evident in the early units, it is not completely eradicated meaning that the problem has been quite severe a design flaw, which is hard to fix completely.

I personally think that your arguments are nowhere near the level of your arrogance and I find your tone against Shifty to be very disrespectul. Shiftys posts have pretty much always been a pleasure to read and I think he deserves more honourable behaviour than what you are giving him, just imo of course.

 

[Cheezdoodles]

No, but if everything is abnormality then what is normal? We get these pieces of information from multiple sources saying the exact same thing, yet nothing can be considered as real evidence.

What would be normal is whatever the real average numbers would be. 75% is certainly not the average number.

The fact is that Shiftys poll is not the only source in the internet saying the same thing that echoes across the whole damn internet, the box has had horrible failure rate and almost everybody has realised that by now.

And im not arguing that. I never once said it wasn't high. What i was pointing out that the poll itself has no statistical value, nor should it be used to give any indication of global rates. We do not know these rates, but there is a lot of evidence to suspect them for being high.

Also many reports indicate that eventhough the problem is most evident in the early units, it is not completely eradicated meaning that the problem has been quite severe a design flaw, which is hard to fix completely.

But still, if it was most evident in the early units, a sample with a higher consistency of launch units would have a negative impact.

I personally think that your arguments are nowhere near the level of your arrogance and I find your tone against Shifty to be very disrespectul. Shiftys posts have pretty much always been a pleasure to read and I think he deserves more honourable behaviour than what you are giving him, just imo of course.

I dont give a flying **** about shifty's post history. Shifty is a good guy, but the way he is arguing here, is seriously below par.

He disregarded all statistical arguments, as not being relevant, because he doesnt understand it. (Until the recently turnaround, where suddenly the focus is not of it being an indicative worldwide, but for completely other reasons).

 

[Jesus2006]

What i was pointing out that the poll itself has no statistical value, nor should it be used to give any indication of global rates. We do not know these rates, but there is a lot of evidence to suspect them for being high. .

The poll is a statistic and has a statisitcal value as any other statistic. There is no difference in asking people on the streets from asking them on a forum in the internet. It's all about collecting opinions, and that's what it does.

The only thing that counts is the amount of people asked, the more the better.

And since we have the same results in different countries on many polls over the whole internet i think we can definately call this reliable.

 

[NovemberMike]

The poll is a statistic and has a statisitcal value as any other statistic. There is no difference in asking people on the streets from asking them on a forum in the internet. It's all about collecting opinions, and that's what it does.

The only thing that counts is the amount of people asked, the more the better.

And since we have the same results in different countries on many polls over the whole internet i think we can definately call this reliable.

Might want to read up on the story of how the Gallup polls started (hint: Gallup polled about 5k people, the other guys polled 10 million. Gallup got it right).

Numbers mean absolutely nothing unless you have a reliable sample (and even going to the internet messes it up).

I personally think that your arguments are nowhere near the level of your arrogance and I find your tone against Shifty to be very disrespectul. Shiftys posts have pretty much always been a pleasure to read and I think he deserves more honourable behaviour than what you are giving him, just imo of course.

Shifty is a great guy when it comes to conversations about videogames, but on statistics he is about as pleasurable to read as the twelve year old kids who think that Crysis is possible on the PS3 because it has 4 times as many processors as a computer does...

 

[BoardBonobo]

I don't quite get what all the fuss is about here.

Shifty's poll is statistically relevant to the forum members of B3D, not the global collective. So for the group that it represents, the slightly more tech savvy perhaps and in that maybe more early adopters, it is very relevant. It's a straw poll that asks how many people have experienced a particular thing. It's not at all relevant how many people suffered a dead 360 and now have working ones, that would show something completely different.

Out of 110 people, who lay claim to having posessed (and may still posess) an Xbox360, 43 people had theirs die on them or just over 47% of those 110 people. And including the dead 360's ~61% have suffered from problems.

Now I agree that these results can't be expanded to cover the globe, because more questions would need to be asked such as date of purchase, date it broke, average number of hours used per day etc.

Anyway you want to read it (the joy of stats!) it still shows a very worrying trend even if just restricted to the local sample. Maybe someone needs to get the results of other surveys (polls) completed along with the questions asked and do an analysis of the lot. Even a stochastic will root out the more out there results.

 

[Cheezdoodles]

The poll is a statistic and has a statisitcal value as any other statistic. There is no difference in asking people on the streets from asking them on a forum in the internet.

This is, to put bluntly, flat out wrong.

There is an enourmus difference between asking people on street or asking a forum. You would ask different people, with different values and different experience. Forum goers is a certain segment of people, people that go to b3d on a regular basis are not the same kind of people that go to say, gameFAQ's, the average person on the street is then again a completely different person, in terms of values and experience than an average forumgoer.

If you make a poll about which console your going to buy at the www.teamxbox.com forum, you would get a completely different result than asking at www.ps3forum.com (or whatever). Neither of them have any real statistical value, in order for a poll to be significant, it needs to have a proper sample, the sample size needs to be big enough to agree with certain scientifical standarts, and the sample population needs to be a representative of the average persons\items we are polling.

If your going to make a poll about car reliability, your not going to get accurate results if you ask only people who drives Lada's. You could ask every Lada owner in the world about reliability, but the number would not be a good indicative of anything except for the reliability of Lada's.

T
It's all about collecting opinions, and that's what it does.
The only thing that counts is the amount of people asked, the more the better.
And since we have the same results in different countries on many polls over the whole internet i think we can definately call this reliable.

This is also, flat out wrong.

I'm going to illustrate this with a simple example. You and me are both contracted to do a poll on gun control. You decide to go ask random people on the streets of New York, and Los Angeles. You being eager to prove you did a better job than me, asks every citizen in New York, and everybody in Los Angeles, you have recorded 30 000 000 opinions about gun control.

Meanwhile, i do some calculations and figure out how many people that live around in the varius american cities, how old the average american is, and so on.

I only call 100,000 people, but the people i call, will in total represent the average american, i got the same amount of people from Texas, that there is living in texas compared to the rest of the states, they have the correct age, corresponding to the different average etc...

My result would be a much better indicative of the american people's gun control opinions than yours, simply because my sample's are a better representative than yours. People living in LA and NY, have completely different views on gun control than a local hillbilly from Texas.

 

[patsu]

I could see the user names of people who voted in this poll, but now I can't. What gives ?

 

[Falkster]

I could see the user names of people who voted in this poll, but now I can't. What gives ?

They're still there as far as I can see:
http://forum.beyond3d.com/poll.php?do=showresults&pollid=704

 

[patsu]

Oh goodie you're right.

Ostepop, in what ways do you think people's distribution on this board change the number of 360 failed units ? (I was trying to see if there are sufficient new/unknown posters who vote for negative 360 responses).

 

[Bigus Dickus]

Oh goodie you're right.

Ostepop, in what ways do you think people's distribution on this board change the number of 360 failed units ? (I was trying to see if there are sufficient new/unknown posters who vote for negative 360 responses).

People here are internet savvy. To be at a site like B3D they are likely also graphics savvy. And it stands to reason that they are more enthusiastic gamers than the average consumer. And the purchase date for B3D forum goers is also likely to be much earlier than the average. And people who bother to register and post on tech forums tend to like to bitch and "share" more than the average consumer. And forum posters in general (much like the average person) like to complain about the bad more than praise the good. And there is a "MS bad, 360's fail" inertia on the web that reinforces the tendency to report every failed unit on every poll people can find. It has become something of a "fad" lately to see how many failed 360's there are in the world through internet polling.

Do any of those things make you think that a poll here, or on any other forum, is going to remotely represent the consumer population?

 

[deepbrown]

People here are internet savvy. To be at a site like B3D they are likely also graphics savvy. And it stands to reason that they are more enthusiastic gamers than the average consumer. And the purchase date for B3D forum goers is also likely to be much earlier than the average. And people who bother to register and post on tech forums tend to like to bitch and "share" more than the average consumer. And forum posters in general (much like the average person) like to complain about the bad more than praise the good. And there is a "MS bad, 360's fail" inertia on the web that reinforces the tendency to report every failed unit on every poll people can find. It has become something of a "fad" lately to see how many failed 360's there are in the world through internet polling.

Do any of those things make you think that a poll here, or on any other forum, is going to remotely represent the consumer population?

Anybody who says it does would be saying something quite absurd.

But this certainly shows a trend. I believe everyone has been honest, and if it was a few here or there then the complaints may be met...but it's quite staggerring the amount. The trend is there...even though the percentage would of course be much lower...perhaps half?

 

[patsu]

People here are internet savvy. To be at a site like B3D they are likely also graphics savvy. And it stands to reason that they are more enthusiastic gamers than the average consumer. And the purchase date for B3D forum goers is also likely to be much earlier than the average. And people who bother to register and post on tech forums tend to like to bitch and "share" more than the average consumer. And forum posters in general (much like the average person) like to complain about the bad more than praise the good. And there is a "MS bad, 360's fail" inertia on the web that reinforces the tendency to report every failed unit on every poll people can find. It has become something of a "fad" lately to see how many failed 360's there are in the world through internet polling.

Ok, so you're saying:
(1) B3D folks are more likely to complain if their consoles break and keep quiet if their consoles work ?
(2) If so, doesn't the same behavior apply to PS3 and Wii as well ?

You also implied launch 360s are more prone to failure. Doesn't the above numbers reflect this phenomenon vis-a-vis early PS3 and Wii ? So you're agreeing with them ? Or do you think the figures should be higher/lower... since some also argue that the longer one waits, the more likely their consoles will break ? Savvy 360 owners (like someone on this board) may have swapped their launch 360s out for newer model via store warranty too.

My current reading is:
This is just a snapshot. The reported 360 failure is noticeably/significantly higher than other consoles. That's all. We can't use it to infer any trends.

Do any of those things make you think that a poll here, or on any other forum, is going to remotely represent the consumer population?

That's what I'm asking. The 360 is a single-purpose machine. Does Xbox 360 break according to who bought it ? Does the consumer distribution somehow determine the fate of these 360s above... or does the manufacturing characteristics play a more major role ?

OTOH, PS3 is a multi-purpose box. So it may indeed fail earlier/later depending on how it's used. e.g., A small but significant number of PS3s are folding constantly. These people's PS3 may have worked longer (active) hours than the oldest working 360... so I'm not sure if we are being fair by just comparing the number of calendar days these console survived.

 

[Rolf N]

1. There's no correlation between frequenting forums and having your stuff fail on you, no matter how hard the hand-waving gets. The other way around perhaps, yes, to a certain degree. So just dismiss the "Hi, first post, my Xbox 360 just died!" posters. Easy enough, the poll is open.

2. There is not a single sample population that shows failure rates under 30%. Nowehere. None. Never happened. So it's not even cherry-picking the wrong forums and gaming the polls. There is no known species on this planet that has experienced acceptable XBox 360 failure rates.

3. Look at the NeoGAF tally (full list of names, tracks multiple failures, tracks near-deaths).

437 dead consoles, 14 on life support. 359 individual posters. Any incorrect info, let me know.

I can't believe arne is on the list! Just to remind, I don't include "my brother/friend/etc." on the list. This is for NeoGAFers only, so we can have some minor semblance of percentages. I also don't include HDD failures, that's a separate issue.

And it's hard to manipulate polls on NeoGAF. The registration process takes months to go through and trolling juniors are usually banned without warning.
Beyond3D is much easier to game, especially since everyone can vote in every poll, unlike some other fora I know of where a minimum of 50 posts and ten days registration has been instated after a major manipulation was discovered. Was very interesting who voted Wii60 somewhere at some point in time in an open poll, but I digress. GAF is tighter than this no matter how much you hate the results.


And the response?
*LaalaalaaaIcan'thearyouuuuu*

 

[Shifty Geezer]

And now please, take the time to respond the the prior post as well. Because your switching opinions here, first you try to argue with me, that i haven't provided any real answers as to why these numbers are flawed. And now suddenly, you have switched your opinion completely.

No I haven't. My argument has always been the same and if it appears to have changed, maybe I'm phrasing myself poorly? Perhaps you've missed my argument and think I'm talking about something I'm not? This discussion I have with you is all from this first comment in your second post -

I'm not even going to get into speculating the failure rate, im just going to explain to you why poll's on websites are largely useless for anything:

"Useless for anything." This I disagree with. I say polls are useful. Inaccurate, but useful. Is this poll useful for extrapolating whole world views? No. Never thought it was (when did I even say I was viewing these results as representative of the whole world?). Is it useful for other reasons? Yes. So I go on to argue that the things you say can swing a poll and thus make it useless can be discounted by reasoning the population, and so makes the poll data useful. Without saying the poll is showing us how the world is. That was never my case. I dunno how many times I've repeated that, but it's true! My argument has never been 'this poll is representative of the whole XB360 user base.' It's about getting some data to consider, and that the info in this poll is useful, in contradiction to your statement that poll on websites are largely useless.

So all my responses to you have been to argue that the given reasons why statisticians throw these numbers out the window and label useless can be considered and on occassion 'overruled', not using proper statistics to form opinions but using other reasoning. They are not useless!

Okay, reviewing the thread, I can see where I started talking representative number for the whole populace. I wasn't specifically arguing that from this poll we get 40% failure rate for all boxes. I was still arguing that the poll is useful, and talking about the points you raised. So when I said...

"Of course. However this is the best we can do, and unless there's good cause to think the average has that much variety, the small poll size should be indicative, if not accurate."

...I wasn't saying 'these results are statistically valid for drawing conclusions on all XB360s' but saying 'poll data can be useful for drawing conclusions if you consider other logical reasons.' I was saying that your assertion 'internet polls are largely useless for anything' is ignoring the value that they do provide even without being statistically accurate, and trying to explain how so.

 

[Cheezdoodles]

Ok, so you're saying:
(1) B3D folks are more likely to complain if their consoles break and keep quiet if their consoles work ?
(2) If so, doesn't the same behavior apply to PS3 and Wii as well ?

1. People with broken consoles are more likely to complain about it, yes.
2. Yes, but they are reliable as hell to begin with.

You also implied launch 360s are more prone to failure. Doesn't the above numbers capture this phenomenon vis-a-vis early PS3 and Wii also ?

The latest discussions aren't about PS3vsX360vsWii. The wii\ps3 could have excellent build quality and have a very small % of broken units no matter when you buy it.

Does Xbox 360 break according to who bought it ??

In some ways yes, launch consoles are more prone to be flawed than later ones. B3D has a higher consistency of early adapters than the average console buyers, therefor the data will give you result not representative of the average

Does the consumer distribution somehow determine the fate of these 360s above... or does the manufacturing characteristics play a more major role

Further not sure what exactly your asking here, im not sure if you understand what Bigus Dickus (gotta luw that nick) and I have pointed out here.

The main point is that the sample size, and the poll data at hand is not big enough (nor a good enough representative of the average console) to give you any numbers of statistical significance, and is therefore irrelevant to be used as an indicative, or as data for any real world analysis.

What you see here could be an abnormality, it could be a proper accurate number (purely by luck) , there is no way to tell, because the data is flawed.

Because the data at hand is not a representative of the consumer population, the question about consumer distribution and manufacturing characteristics is irrelevant, and something which cannot be answered tbh.

2. There is not a single sample population that shows failure rates under 30%. Nowehere. None. Never happened. So it's not even cherry-picking the wrong forums and gaming the polls. There is no known species on this planet that has experienced acceptable XBox 360 failure rates.

Im not saying below 30% is an acceptible number, but i can assure that there are sample populations that has such numbers. Simply by the rule of abnormality you could even see samples with 99% working units.

3. Look at the NeoGAF tally (full list of names, tracks multiple failures, tracks near-deaths).And it's hard to manipulate polls on NeoGAF. The registration process takes months to go through and trolling juniors are usually banned without warning.

But that is not a poll, its only takes into account how many consoles are dead. Nobody is posting "My X360 is working fine". Your only counting broken consoles, how is that a valueable number of anything? Its not a poll, its a "official bitch about x360 thread".

How many members does NeoGaf have? 20.000. How many have X360s? Unless you know those numbers, that data is worthless. How do you know what the % of failures are, you dont know the total population on neogaf that has x360's?

Who is most likely to be in the official bitch thread posting? The guy with the working X360, or the guy with the broken X360?

 

[Cheezdoodles]

delete this

 

[Shifty Geezer]

But that is not a poll, its only takes into account how many consoles are dead. Nobody is posting "My X360 is working fine". Your only counting broken consoles, how is that a valueable number of anything? Its not a poll, its a "official bitch about x360 thread".

Which is precisely why this poll was created! And why it's different. It may not manage to apply mathematically correct statistical principles to an analysis of failure rates, but it does provide data and logical relations to help get some useful figures to help determine if there really is a problem with the XB360s sold thus far, and how that problem relates to other systems.

 

[PARANOiA]

It would be nice to post a poll along the lines of "I owned a 360/PS3/Wii for six months and it's working/broken". 360's - with what appears to be a fairly high failure rate - can't really be compared to the other two machines that have been on the market for six months (less than two in Europe!)

Early PS1's worked well for about nine months before they turned to rubbish. Just some food for thought.

 

[deepbrown]

It would be nice to post a poll along the lines of "I owned a 360/PS3/Wii for six months and it's working/broken". 360's - with what appears to be a fairly high failure rate - can't really be compared to the other two machines that have been on the market for six months (less than two in Europe!)

Early PS1's worked well for about nine months before they turned to rubbish. Just some food for thought.

Really? My ps1 still works

 

[Shifty Geezer]

... can't really be compared to the other two machines that have been on the market for six months (less than two in Europe!)

Early PS1's worked well for about nine months before they turned to rubbish. Just some food for thought.

Yep. Which is why I served this little morsel up in the first post...

It doesn't take into account long term problems, so of course there's a possible issue there - PS3's could all die horribly in 13 months...