Spacial stereo processing

[Shifty Geezer]

I've had my Samsung Pebble sm2032mw for a few days now and noticed that the stereo in Uncharted was extremely spacious, with the waterfall sounding way out on the left. At first I thought it was the TV doing some processing, but the same spacial audio wasn't present in other games. I tried some surround output modes in case the TV supported them (manual doesn't mention audio at all!) but it didn't. So I guess the spacial audio is actually happening in game.

Which gets me thinking how intensive is this? I thought not very, in which case couldn't a library make it common to all games? And better yet, could developers target headphones for that 'in the action' feeling achieved with binaural recordings? Is there a strong reason to forsake stereo and headphone use and rely solely on standard 5.1 sound formats? AFAIK headphone use is still strong and reproducing 5.1 on stereo headphones is surely going to find a lot of happy users?

 

[Johnny_Physics]

I seem to remember Silent Hill 2 used a new mixing library or something that Sony developed that would get a 3d-feel out of an ordinary stereo tv, and it worked pretty decently IIRC. I don't know how much adoption it saw, but perhaps they're cooking up something similar for the PS3?
Perhaps there's patents in the way? I know Aureal had their mixing system that worked great with headphones but they got stomped by Creative that then bought them, and Creative isn't always nice and reasonable("Carmack's reverse") so that might be a roadblock even though it seems very unlikely.


Also, something slightly related, I had a friend that worked for a state funded tech house that was full of bright people just fiddling with things and trying to create cool stuff without a necessary profit in mind, and they had this one group that had determined what would be, and then constructed, some kind of perfect surround system that was, to the naked eye, just one huge loud speaker that you faced. It was an advanced mixer and a mono-output or something.

 

[Hazuki Ryu]

I've had my Samsung Pebble sm2032mw for a few days now and noticed that the stereo in Uncharted was extremely spacious, with the waterfall sounding way out on the left. At first I thought it was the TV doing some processing, but the same spacial audio wasn't present in other games. I tried some surround output modes in case the TV supported them (manual doesn't mention audio at all!) but it didn't. So I guess the spacial audio is actually happening in game.

Which gets me thinking how intensive is this? I thought not very, in which case couldn't a library make it common to all games? And better yet, could developers target headphones for that 'in the action' feeling achieved with binaural recordings? Is there a strong reason to forsake stereo and headphone use and rely solely on standard 5.1 sound formats? AFAIK headphone use is still strong and reproducing 5.1 on stereo headphones is surely going to find a lot of happy users?

Do you have xbox 360 ? Pgr 4 has got a lot of sound options I wonder how different they fell I'm gonna try it right now in 5.1 and stereo with some different settings I dont know if it uses any headphones mode though.

Edit: so I tried pgr 4 on the following settings

Console set to usual Dolby Digital 5.1 and my reciver set to 2 channel stereo
Console set to digital stereo and game set to stereo reciver in straight mode (or neutral)

I didnt really find any difference though testing was very superficial I'll post if i find anything

 

[Deepak]

I think this is OT but Dreamcast opening screen music always sounds wow-some on my crappy CRT TV.

 

[Shifty Geezer]

My first experience of spacial stereo was a demo of the CDTV playing the BitMap Brothers' Xenon II. This was shortly after the tech was introduced on the Tomorrow's World TV program. I still recall the amazing experience of sounds from outside the speakers. Since then there's been a whole host of technology advances, and we have spatial processing of standard stereo streams to add room on lots of music player, but I haven't encountered much in the way 2 speaker spacial/surround audio short of a few demos. Surround has been implemented through physical speaker setups. It was only recently I had my first experience with binaural recordings on headphones which were amazing, and yet the technique goes back a a hundred years!

I'm curious why stereo implementations of volumetric audio (that's a cool term. Think I'll trademark it!) aren't way more commonplace, especially when some games it appears do.

 

[betan]

I'm curious why stereo implementations of volumetric audio (that's a cool term. Think I'll trademark it!) aren't way more commonplace, especially when some games it appears do.

I guess because it requires knowledge of speaker and player positions or luck, and is limited to front 1D.

I assume you are using PS3's stereo output.

 

[Squeak]

AFAIK headphone use is still strong and reproducing 5.1 on stereo headphones is surely going to find a lot of happy users?

Still?! If anything it's on the rise.
Quality headphones offer SQ much higher than any speaker for the same price or speakers that costs much more than the headphones.
And can also offer much better imaging with the right kind of mic setup/filters.

The only weakness is low bass and especially bass so low that you can feel it. That, and the fact the you are bound by a wire, but that isn't much of an issue when sitting down watching a movie or playing a game.

 

[Hazuki Ryu]

Still?! If anything it's on the rise.
Quality headphones offer SQ much higher than any speaker for the same price or speakers that costs much more than the headphones.
And can also offer much better imaging with the right kind of mic setup/filters.

The only weakness is low bass and especially bass so low that you can feel it. That, and the fact the you are bound by a wire, but that isn't much of an issue when sitting down watching a movie or playing a game.

Though i agree with you in the quality for the price, if you want to have 5 people listening to music or a movie in your living room you need 5 headphones and 5 headphone connections or a spliter, most people don't like to use headphones me included I only use them when i have to and no matter how good they are they will eventually hurt your years if you use them for extended periuds of time.

 

[betan]

Though i agree with you in the quality for the price, if you want to have 5 people listening to music or a movie in your living room you need 5 headphones and 5 headphone connections or a spliter, most people don't like to use headphones me included I only use them when i have to and no matter how good they are they will eventually hurt your years if you use them for extended periuds of time.

I don't know about most people, but I'm in the market for a decent surround headphone, if nothing else for noise reduction at late night.
Plus I expect headphones to be the only practical way of obtaining real 3d sound (including up and down) in the near future.

 

[Mobius1aic]

Plus I expect headphones to be the only practical way of obtaining real 3d sound (including up and down) in the near future.

I think thats arguable. Also it's necessary to talk about how many speaker satellites we'd need in order to satiate people. That's OT so I won't go into it.

 

[ShadowRunner]

I recently purchased a pair of quality headphones (Senn. HD650) and THIS to go along with them. It basically applies "Dolby Headphone" processing to a 5.1 surround signal sent to it, and works with any pair of stereo phones. From my experience so far its great, the sound positioning for me seems better than my 5.1 system (Logitech Z5500) and i will be using it exclusively from now on for solo gaming. Condemed 2 just jumped up from a 8/10 to 9.5/10 on the review scale because of this setup

Now this is all done by using a 5.1 signal, if the processing was done in-game there would be infinite "channels" from which to create the final stereo output and i should imagine the effect would therefore be far better. It realy is a shame not much is being done with this tech from what i can see.

Uncharted actualy has a headphone setting in the audio options, have you tried that setting shifty? I think ill give it a try tomorrow, i have a feeling some sort of processing of the audio must be going on (or else whats the difference the stereo and headphone settings?).

 

[ShadowRunner]

Couldnt wait til the morn, works gonna be a pain tomorrow
Tried Uncharted with the headphone setting and the effect is similar to what you get with Dolby Headphone. Switching back to Stereo in the settings removes this effect so there is definatly something going on.

I never even noticed these settings before which is making me wonder how many other games have them, but just dont go about advertising the fact in any way.

 

[ShadowRunner]

I think thats arguable. Also it's necessary to talk about how many speaker satellites we'd need in order to satiate people. That's OT so I won't go into it.

Its arguable yes, but in my opinion almost certainly true. Being able to simulate an infinite number of satellites through a pair of headphones is far more practical than actually having an infinite number of satellites around your room

 

[betan]

I think thats arguable. Also it's necessary to talk about how many speaker satellites we'd need in order to satiate people. That's OT so I won't go into it.

First of all, this seems perfectly on topic to me.
Second, currently what's minimal 3d surround solution 3 layer 22.2? I don't really know.

For headphones possibly 6 speakers would be more than enough, because in addition to phase shifting and filtering tricks that can be used safely, headphones also have ear lobe positioning advantage over external surround systems.

Though current 2-speaker headphones may not provide full 2d localization, I wouldn't be surprised if future stereo headphones provide more than satisfactory 3d localization by emulating any filtering auditory system does based on position of sound source.

Is there a reason you think that's not possible?

 

[Squeak]

We only have two ears, so all we need to simulate any number of "channels" or sound positions is two transducers and the right kind of software.

Don't buy surround headphones, they are bogus. All you do, is get extra cruddy little drivers, instead of two good quality ones. And most likely the surround effect will be much worse than with a good software solution.

The main problem is that peoples ears are quite different, so we will need some kind of calibration to get the best result, but that could be made as simple as adjusting screen position and colours with the picture.

 

[FutureCTO]

We only have two ears, so all we need to simulate any number of "channels" or sound positions is two transducers and the right kind of software.

Don't buy surround headphones, they are bogus. All you do, is get extra cruddy little drivers, instead of two good quality ones. And most likely the surround effect will be much worse than with a good software solution.

The main problem is that peoples ears are quite different, so we will need some kind of calibration to get the best result, but that could be made as simple as adjusting screen position and colours with the picture.

You are correct we only have two ears. We also only have two eyes. Yet black and white, single pulse and wane are not our perceptual limits. When looking at an object we can only perceive light from the relative angles of our eyes in relation to the objects positioning and the amplitude of brightness and visual spectrum being cast and reflected...

I don't want to get too elaborate (me, big posts?) but having two ear does not mean two speakers are enough to accurately production the full spectrum of intermixed sound that we can perceive. Though two is enough for seperated sound. Cross over two way audio frequencies exist for a reason.

My preferred audio system uses asymmetric reflected sound. This produces the greatest sound field depth and most evenly fills the audio space. The only headphones I can wear are "open" or pass-through. Closed headphones hurt my ears immensely and result in ringing (= hearing loss) even with mild audio levels.

To sum things up. The world is an asymmetrically unified electromagnetic field. All things exist simultaneously or cancel each other out. So long as one wave travels out of phase with another they both may exist. The co-existence of multiple frequencies on multiple planes, angles and reflective/refraction/scattering is the color of the world as we best perceive it in both audio and visual.

 

[vazel]

I recently purchased a pair of quality headphones (Senn. HD650) and THIS to go along with them. It basically applies "Dolby Headphone" processing to a 5.1 surround signal sent to it, and works with any pair of stereo phones. From my experience so far its great, the sound positioning for me seems better than my 5.1 system (Logitech Z5500) and i will be using it exclusively from now on for solo gaming. Condemed 2 just jumped up from a 8/10 to 9.5/10 on the review scale because of this setup

Now this is all done by using a 5.1 signal, if the processing was done in-game there would be infinite "channels" from which to create the final stereo output and i should imagine the effect would therefore be far better. It realy is a shame not much is being done with this tech from what i can see.

Uncharted actualy has a headphone setting in the audio options, have you tried that setting shifty? I think ill give it a try tomorrow, i have a feeling some sort of processing of the audio must be going on (or else whats the difference the stereo and headphone settings?).

Your discrete surround setup must be setup wrong then. You probably have to raise the volume of the rear speakers. I found I had to do that to make the rear speakers stand out more.

Although audio recorded specifically as binaural with special arrangement of microphones is impressive from my experience real time surround sound virtualization is only good at reproducing front right/left and rear right/left so you're only getting 4.0 sound. And even then the surround positioning varies. Only 'standout' sounds are positional. You're losing out on the atmosphere created from the soundfield of music or environmental sound playing on all speakers. And you're losing out on the most impressive part of a 5.1 setup, the subwoofer. The rumble a subwoofer brings to audio is just so magnificent I find it an even better effect than surround positioning. A 2.1 setup would be better than a Dolby Headphone setup imo.

 

[Shifty Geezer]

I agree sounds from behind aren't so strong in the binaural demo's I've heard, and the behind audio is one of the key strengths to a surround audio solution. It'd be nice to hear someone sneaking up behind you in Warhawk rather than them getting a free knife kill! I can't say I've ever experienced a good surround system though. Generally it seems to me sounds are 'discrete', sounding like they're from the speaker behind you, rather than an arbitrary position. I'm sure proper calibrated speakers and seating can solve this, but software can't target only that demographic of high-end AV enthusiasts! Solutions have to be found for Average Joes. The latest 5.1 setup I heard was my freinds new HDMI AMP that had an automatic calibration function using a microphone placed on seating, and the result still isn't great. If the effort to actually build a system that delivers perfect surround is too much, most people won't bother. That's where I think headphones provide the best compromise, providing the simplest implementation for the end user with the most accurate spatial representation. Seating position and speaker position aren't variables to wrestle with as they're fixed. Even non-calibrated solutions are pretty effective.

I've only got the Uncharted demo and I didn't see a headphone option in that, but I'll borrow a copy and try it out. Though this TV plays random 'notes' on the audio out that messes up headphones as they're clearly audible in quiet areas - my only gripe with it.

 

[vazel]

Actually I meant binaural audio recorded specifically as that with specially arranged microphones did an impressive job of surround sound(including rear). It's realtime surround virtualization from a 5.1 signal that isn't as good.

I can't say I've ever had that experience of a surround setup sounding too 'discrete'. But then again this thread is for people trying to justify headphone surround sound virtualization.

 

[Squeak]

You are correct we only have two ears. We also only have two eyes. Yet black and white, single pulse and wane are not our perceptual limits. When looking at an object we can only perceive light from the relative angles of our eyes in relation to the objects positioning and the amplitude of brightness and visual spectrum being cast and reflected...

I don't want to get too elaborate (me, big posts?) but having two ear does not mean two speakers are enough to accurately production the full spectrum of intermixed sound that we can perceive.

Sound is not like light at all, and ears and eyes are two very different receptors.
What you get in through your ears is solely a one dimensional signal. The only "3d" is in the hints from amplitude change, frequency change and phase change between the ears. But that's what your brain is for.
Two drivers is enough. We only need better headphones and better software to make the 3d experience better.