Predict: The Next Generation Console Tech - AUDIO

[hoho]

Because we have only two ears!

+ with speakers you can move your head relative to the sound source to quickly find out where it's coming from. With headphones you loose that ability.

 

[Shifty Geezer]

A proper binaural system for games would have head tracking. And 5.1 headphones are at the same disadvantage anyway, although you could add headtracking to those.

 

[function]

Using something like Kinect you could create a 3D model of each user's ears (during a guided setup process). That would allow relatively inexpensive stereo headphones with one large, high quality driver to deliver the best possible results. And it's all perfectly doable on current tech.

C'mon Xbox 3, make it standard feature of the device that's tied to a user profile so all games can use it! Allow for headphones/headsets to connect to the pad (or wirelessly) just as current Live headsets do, then you can guarantee no latency and that everyone can access the feature with no difficulty and little cost.

Perfect binaural audio, with headtracking, for every user of the system (even in multiplayer) for every game. I'd take that over any other possible advance in console audio.

 

[mrcorbo]

Because we have only two ears! 5.1 allows placement of the source sound, but you have to average sounds that come between the speaker positions, and you can't place sounds behind the speaker positions. To do this requires cheats. Well, if you're going to model sound placement, why not do it just for 2 speakers for the two audio sensors that the brain uses? 5.1 Audio is like trying to generate 3D images with 10 images and a lenticular lens instead of using two images, one over each eye. The ability for binaural audio to model spaces and especially upclose sounds (what could be more immersive than a jungle shooter with mozzies buzzing aggravatingly around your head?!) is pretty prefect, in the same way a stereoscopic headset is leaps and bounds a more immersive 3D experience than looking a screen some distance away.

OK, I see what you were getting at. But I don't really see that as a knock on the headsets themselves. They do what they were designed to do well and their function is dictated by the source material.

The problem I see is that people would be resistant to wearing headsets or even earbuds all the time in the same way that they have proven to be resistant to wearing glasses to watch 3D television. It's more fatiguing and it's also often impractical to completely isolate your hearing from the outside world for an extended period of time.

And as long as you have a fixed position display, head-tracking audio seems pointless to me. You're always going to need to be staring forward at the screen! It would have to be a package deal with a head-mounted stereoscopic display and that isn't likely to make it into the next gen.

 

[hoho]

And as long as you have a fixed position display, head-tracking audio seems pointless to me. You're always going to need to be staring forward at the screen!

I'm not sure about you but I can move my eyes and head independently

 

[Shifty Geezer]

OK, I see what you were getting at. But I don't really see that as a knock on the headsets themselves. They do what they were designed to do well and their function is dictated by the source material.

Sure they work. It's just time for new-and-improved (and cheaper, or better, with only needing two speakers).

The problem I see is that people would be resistant to wearing headsets or even earbuds all the time in the same way that they have proven to be resistant to wearing glasses to watch 3D television. It's more fatiguing and it's also often impractical to completely isolate your hearing from the outside world for an extended period of time.

More resistant, maybe, but not as bad as wearing glasses I'm sure. People listen to their iPods et al for hours at a time. They needn't be closed off to the outside world either. We don't currently game in silence, so there's no need to enforce silence for future games.

And as long as you have a fixed position display, head-tracking audio seems pointless to me. You're always going to need to be staring forward at the screen! It would have to be a package deal with a head-mounted stereoscopic display and that isn't likely to make it into the next gen.

Nah. That'd be the ultimate, ideal immersion, but you can quickly turn your head to listen in on a sound without really losing anything in the game.

Someone needs to create a demo at a public show and see what the response is.

 

[AlNom]

So...... what are devs likely to do next gen? Do we just continue to see the multiple audio options that we currently see in a number of PS3 titles?

I would like to see next generation boxes with audio processors capable of doing 96khz@24bit in real time and as standard.

I don't think we should have a problem finding a cpu core for handling audio. What resolution and sampling rate do PS3 games even use for LPCM?

 

[mrcorbo]

I'm not sure about you but I can move my eyes and head independently

I'm not sure about you, but I can hear sounds all around me without turning my head.

Sure they work. It's just time for new-and-improved (and cheaper, or better, with only needing two speakers).

Well, if you're going to narrow it down to headsets that use multiple drivers to provide surround sound instead of doing it in software using a HRTF-based algorithm, then I can agree that those are pretty clunky.

More resistant, maybe, but not as bad as wearing glasses I'm sure. People listen to their iPods et al for hours at a time. They needn't be closed off to the outside world either. We don't currently game in silence, so there's no need to enforce silence for future games.

It's also a question of volume and how much immersion you really want. That was the goal wasn't it? At how much of a lower volume would I have to listen to my game/movie on a headset than I would from speakers in order to still be able to reliably hear sounds around me, keeping in mind that in order to work at all your headset would be blocking both ears?

Nah. That'd be the ultimate, ideal immersion, but you can quickly turn your head to listen in on a sound without really losing anything in the game.

Someone needs to create a demo at a public show and see what the response is.

This is what video game immersion looks like and it never involves looking away from the screen. First and foremost we are visually-oriented creatures.

 

[hoho]

I'm not sure about you, but I can hear sounds all around me without turning my head.

So can I but with a small movement of my head I can quickly pinpoint the exact direction of the sound. With headphones (without tracking) it's impossible.

 

[mrcorbo]

So can I but with a small movement of my head I can quickly pinpoint the exact direction of the sound. With headphones (without tracking) it's impossible.

What's the utility? I see this as being as limited as head tracking has been for altering viewpoint in games. Without 1:1 mapping this isn't more immersive, it's just another hands-free controller input.

 

[Shifty Geezer]

So...... what are devs likely to do next gen? Do we just continue to see the multiple audio options that we currently see in a number of PS3 titles?

Same as this gen. There's no reason to change. CD/DVD quality audio in the mixing; MP3 audio sources as they are small and stream faster; various output options that are built around the same audio engine. I'm not even convinced a proper spacial audio engine (raytracing for sounds) would be bothered with even if we had the processor cycles to spare, as I doubt many would be in a position to use it and it wouldn't ever be a selling point. Sounds will just be placed with a basic occlusion engine to muffle them a bit.

I don't think we should have a problem finding a cpu core for handling audio. What resolution and sampling rate do PS3 games even use for LPCM?

I think this is kinda moot. Aren't most sounds loaded as MP3's anyhow? Although I'm curious if audio mixing is done in 16 bit ints or 16/32 bit floats. The latter would be better for quality and just as fast.

 

[Shifty Geezer]

I'm not sure about you, but I can hear sounds all around me without turning my head.

Yes, but when you want to pinpoint one, you turn your head.

It's also a question of volume and how much immersion you really want. That was the goal wasn't it? At how much of a lower volume would I have to listen to my game/movie on a headset than I would from speakers in order to still be able to reliably hear sounds around me, keeping in mind that in order to work at all your headset would be blocking both ears?

I mostly use large open-backed cans. Any in-the-ear phones I've used don't block out background audio either. I don't know what typical gamer use is, but in my sphere of experiences people play games similar to watching TV, so at a volume where they can comfortably hear the game but which doesn't block out background sounds. You can still hear people in the kitchen or the dishwasher or a conversation.

This is what video game immersion looks like and it never involves looking away from the screen. First and foremost we are visually-oriented creatures.

Except those players have no reason to look away from the screen as they aren't in a surround-sound environment! Now imagine them playing a horror game, walking through a black and leafy woods. They hear some noises behind them, but they're unsure. They stop moving in the game because that generates sounds. They stop still and listen to the muffled noises around them trying to ascertain if they are little animals or fearsome zombies. At that point, a typical response would be to turn one's head and shift focus from the visual sense to the audio sense. If I'm concentrating on a sound, I'm not paying any attention to what my eyes see (I may even close my eyes), and I'm sure you've experienced similar. I notice I tend to furrow my brow when focussing my hearing.

It'd definitely work. It wouldn't be as good as a proper head display, but it wouldn't break immersion or be unnatural. And it'd add considerably to the emotive experiences when you can hear shrapnel whizzing past your ear or bees buzzing around.

 

[rekator]

I think this is kinda moot. Aren't most sounds loaded as MP3's anyhow? Although I'm curious if audio mixing is done in 16 bit ints or 16/32 bit floats. The latter would be better for quality and just as fast.

If I'm remember well three years there already a post (made by dev) about this, all sounds are treat in mp3 due to size, bandwidth in actuals consoles, so all the formats below MP3 quality are useless.

For the sound I'm personally thinking, in first, is more a better "human" approach (test mix in different environments and on many speakers quality) than an increase in encode quality. The second without the first is a total garbage…
Many editors don't invest correctly in sound, and I'm don't speak about the specific "French Touch" in Sound Mix for the doubling…

On some games I'm really think that nobody listen the mix on a 5.1 speakers system in a living-room.

The need is also got speakers surround menu to position correctly speaker (like in your room), is simple to implement and made 5.1 mix correct.

 

[mrcorbo]

Yes, but when you want to pinpoint one, you turn your head.

I mostly use large open-backed cans. Any in-the-ear phones I've used don't block out background audio either. I don't know what typical gamer use is, but in my sphere of experiences people play games similar to watching TV, so at a volume where they can comfortably hear the game but which doesn't block out background sounds. You can still hear people in the kitchen or the dishwasher or a conversation.

Except those players have no reason to look away from the screen as they aren't in a surround-sound environment! Now imagine them playing a horror game, walking through a black and leafy woods. They hear some noises behind them, but they're unsure. They stop moving in the game because that generates sounds. They stop still and listen to the muffled noises around them trying to ascertain if they are little animals or fearsome zombies. At that point, a typical response would be to turn one's head and shift focus from the visual sense to the audio sense. If I'm concentrating on a sound, I'm not paying any attention to what my eyes see (I may even close my eyes), and I'm sure you've experienced similar. I notice I tend to furrow my brow when focussing my hearing.

It'd definitely work. It wouldn't be as good as a proper head display, but it wouldn't break immersion or be unnatural. And it'd add considerably to the emotive experiences when you can hear shrapnel whizzing past your ear or bees buzzing around.

Ultimately, after thinking this through for a bit, I still think the head-tracking audio idea is gimmicky. I have come around to the idea, though, that better positional audio would be very welcome in next gen consoles and would provide a bigger benefit than higher-resolution audio would.

The form I'd like to see it take would be for the platform holder to make a state-of-the-art positional and environmental audio an integral part of the SDK and make it relatively easy for developers to implement it in their titles. The system should also have different output modules that adapt the generated soundfield to the hardware being used to render it with those modules allowing for user configuration at the system level. So, if I wanted to game wearing a stereo headset I would run a configuration wizard that would set that device up and attach those settings to my profile. If I wanted to use a 5.1/7.1 HT setup, I would do the same for that. TV speakers, etc. And it should allow for setting up multiple audio devices and easily selecting between them. This would be great, IMO, and a major step up from what we have now.

 

[menmau]

As we move towards digital content, I already can already see some problems.
I believe someone mentioned that Virtua Fighther 5:FS had somewhat lower sample rates.
Probaly no one noticed, but Soul Calibur for XBLA shows the very same problem, low sample rate comparing to the Dreamcast version.

There might be others...

Seems that digital format sometimes limits the sound sample rate, for obvious reasons (space).
I wonder if newer titles that are being developed face the same destiny or have the audio part limited or not rich as producers would want, in case they had access to a physical format.

 

[bkilian]

The high frequencies get destroyed, it's quite easy to spot in a number of tracks, some instruments are either distorted or missing.
Triangles are a good example of instruments suffering from compression, AFAIR.

I wouldn't go through the trouble of ripping all my CD in FLAC if I didn't find it worth it...

The high frequencies get destroyed in a high end Tube Amp too, that's why the sound is described as "warmer". The better you reproduce high frequency sound, the colder and more clinical the sound feels, which turns a lot of people off.

That's also why the 64Kb (may have been 96, it was a long time ago I saw the study, and I have not been able to find it again) audio was picked as "best" by some, it's not that they couldn't tell the difference, it's that they preferred the distortions in the low samplerate audio.

There are some folks on AVSForum who _insist_ they can tell the difference between lossless, and losslessly compressed audio, despite the two files being bit-for-bit identical when played.

Personally, considering you often have hundreds of audio samples mixing, filtering, resampling, and playing at the same time in an average game, and you and I, as the players are _not_ generally sitting back eyes closed concentrating on the audio, I suspect we would not be able to tell the difference between lossless or Dolby or DTS, and generally wouldn't care. Most of the games I see going past use OGG/MP3/XMA or XWMA compression for their samples, which tend to offer about 10:1 ratio.

 

[tuna]

There are some folks on AVSForum who _insist_ they can tell the difference between lossless, and losslessly compressed audio, despite the two files being bit-for-bit identical when played.

Do you have any links to these crazy individuals posts'?

 

[bkilian]

Do you have any links to these crazy individuals posts'?

http://www.avsforum.com/t/350910/beware-wma-lossless-files-are-not just as an example, and it's not the only one.

Imagine my disappointment, when the WMA files, while generally sounding good, definitely had lost the airiness of the original CD and sounded flatter. This was noticeable in vocals, where on the CD I could hear the singer's faint breathing into the mike, but the WMA file had stripped it away. Also WMA stripped away subtle piano, drum etc. decay.

The bottom line for me is that WMA lossless, while "mathematically identical" to the original, is not acceptable for serious music archiving and listening.

There was one individual who insisted that simply copying the WAV from one hard drive to another resulted in "jitter" he could hear. This individual was not just some random guy, but a move professional, who insisted that he always received the same hard disk that the audio mix was created on.

We used to have some spirited discussions in the HD DVD days where folks would insist they could tell the difference between 5.1 PCM lossless and 5.1 dolby true-hd or 5.1 DTS HD MA.

 

[TheWretched]

Yeah... comparing lossless to lossless and hearing a difference is quite astonishing, really^

I once read an article about a blind test of "audiophile" people, where they let them test hear audio stuff using different cables. One was in fact a coat hanger, and that actually got some of the best grades in that test.

 

[onQ]

isn't one of the PS4 rumors that it will have a audio processor that can stream 200 MP3s at a time?