This Filtering Worthwhile?
- Thread starter Deathlike2
- Start date
Well, multi-sampling will always be much more effcient than super-sampling...MaxSt said:
Yes, it uses bilinear filtering on that part. But since many texures in games are low contrast you would constantly be using bilinear.
Yes, but there's no smooth transition between the filter modes. And the parameter that decides which filter to use would be rather arbirtrary or have to be done by the artist. I see hq3x does it automatically but I wonder how it works on a low-contrast image, in practice. It looks great on a static image that lends itself perfectly for vector graphics, I'm just not convinced it would look ok for real games.
Well in the links I gave above I think it looks great.
Whoa--the hq3x filter is incredible. I'd like to see Nintendo incorporate it with a SNES emulator in their upcoming console, with the option to buy & download classic SNES games online with the ability to play them at what looks like hand-drawn quality. Now THAT would be a smart online business decision, IMO.
Well actually bicubic is not perfect yet. It's clearly illustrated in one of the other links I gave above that only a sinc can be used for perfect re-sampling. But it's not really attractive because of the cost. So several approximations are used. Bilinear filtering has a triangular impulse answer, while bicubic filtering is a better approximation of the sinc function, but still not perfect. And you could have a bicubic filter which has no negative component, which would avoid the 'halo' effect. Unfortunately I don't have Photoshop so I don't even know if it's possible to use your own filter.MaxSt said:
Would you be so kind to try a few textures from real games with hq3x?
Sinc is perfect for band limited signals sampled at or below the nyquist frequency in as such that it perfectly reconstructs the original band limited signal.
Our eyes care much less about this shit than our ears though (and even there sampling theory isnt the be all and end all of things). Images are not sampled at the nyquist frequency, and even if they were we can reconstruct better looking images than the bandlimited one.
Good magnification is edge sensitive, which usually has to mean it has to look at a significant amount of pixels to determine the orientation (usually this isnt done explicitly, but inherent in the magnification filter). Because our original signal is not band limited the area of pixels which are actually used for the weighted average tends to be small though (you can get good results with just 4).
My guess is that hqx3 analyzes a 3x3 area and does the final weighted average using the 4 closest texels. The example image has some rather regular angles BTW
PS. oh I didnt notice that hqx3 had source available and a slightly less contrived example image (where indeed the edge orientation is not captured accurately). All I can make from the source quickly browsing it is that it indeed uses a 3x3 area :/
Our eyes care much less about this shit than our ears though (and even there sampling theory isnt the be all and end all of things). Images are not sampled at the nyquist frequency, and even if they were we can reconstruct better looking images than the bandlimited one.
Good magnification is edge sensitive, which usually has to mean it has to look at a significant amount of pixels to determine the orientation (usually this isnt done explicitly, but inherent in the magnification filter). Because our original signal is not band limited the area of pixels which are actually used for the weighted average tends to be small though (you can get good results with just 4).
My guess is that hqx3 analyzes a 3x3 area and does the final weighted average using the 4 closest texels. The example image has some rather regular angles BTW
PS. oh I didnt notice that hqx3 had source available and a slightly less contrived example image (where indeed the edge orientation is not captured accurately). All I can make from the source quickly browsing it is that it indeed uses a 3x3 area :/
Nick said:
Many Photoshop filters, like "smart blur", use such parameters.
But that's still a valid concern. Sooner or later I'll have to do something about it.
Nick said:
I'm not convinced. Bicubic on those links looks as blurish as bilinear, just less jaggish.
Until I'll see an executable and try it on my test image, I'll remain sceptical.
MaxSt.
Well, there is no perfect general form of filtering. Information has been lost in the sampling. The original information can only be "perfectly" reconstructed if very specific assumptions are made about the original data. If including, for example, a fourier transform in the filtering process, one may make the assumption that the original data included no components of higher-frequency than the nyquist limit for the sample frequency chosen.Nick said:
What is a sinc filter, anyway? It was only mentioned in a cursory manner in the link I scanned...
It is a perfect reconstruction method for a signal sampled at or below the nyquist frequency. It is really simple, at each pixel (-center) you originate a 2D sinc signal with a frequency of one cycle per pixel and the amplitude the same as the original pixel value ... then at the location you want the interpolated value you sum all the local values of all the sinc functions, and hey presto there is the interpolated value.
The assumption that you've then made is that you have a signal that can be simply broken up into fourier components, and none of those components exceed this nyquist frequency.MfA said:
In particular, images in general won't conform to any such assumption. Sound is a bit better, as one doesn't need to reproduce fourier components beyond our range of hearing.
Anyway, what's the form of this 2D sinc signal?
Okay, did some looking.MfA said:I dunno, is all too long ago, I guess it might just be the seperable combination of 1D sinc signals ( sin(pi*t/T)/(pi*t/T) ) by multiplication or addition or whatever ... just forget what I said and do your own googling
Yes, that's right, as long as T is the distance between samples (spatial, temporal, whatever). Looks like the idea behind the sinc function was to design a basis set that was based upon an ideal low-pass filter. So, if your data set is infinitely long, and you can produce the sinc functions correctly at each and every site (of which there are an infinite number), then it would perfectly reconstruct a signal that was originally first passed through a low-pass filter.
Obviously this is not possible, and it must be recognized that while successive sites will see less and less contribution from each further site, since there are an infinite number in the original math, if you suddenly make the sampling finite, you could leave out a very important part of the sampling. The only way, therefore, to realistically do sinc sampling would be in an analytic manner (i.e. instead of figuring out the value of each sinc function, just figure out a formula that takes them all into account). It is typically impossible for an analytic solution to be found for each and every case.
Nick said:
Sync depends on a particular mathematical model of the image.
Imagine the yellow apple on the green table. Should the yellow/green border be sharp? Yes. Will Sync make it sharp? No. That's because the math model doesn't fit.
Nick said:
hq3x wouldn't be very good in that case. You are looking at it from the wrong perspective. Remember, the filter is a "proof of concept", it's a start.
Also remember - I specifically tagreted this first version to be realtime, and I sacrificed some image quality for that. Realtime is an artifical limitation, it's just because I wanted to see the filter in action (in zsnes).
I don't expect the final version to be realtime, there is no actual need for that. But I do expect the final version to handle ALL images well, not just the cartoonish ones, like the current version.
MaxSt.
For a nice example of relatively state of the art magnifcation methods check this (papers on edge directed interpolation can be found here).
One thing most of the algorithms I have read about ignore is texture/noise. I think a decent improvement of the state of the art could still be gained by trying to fit a model to the original image and synthesize texture/noise for the interpolated values, this should make images look better at high magnification.
Oh and I think NEDI could be significantly improved by multiscale methods.
One thing most of the algorithms I have read about ignore is texture/noise. I think a decent improvement of the state of the art could still be gained by trying to fit a model to the original image and synthesize texture/noise for the interpolated values, this should make images look better at high magnification.
Oh and I think NEDI could be significantly improved by multiscale methods.
cybamerc said:
Oki, I understand what you mean now. Well, yes, bicubic isn't so much suitable for typical cartoon or vector-graphics style images, but for typical texture images it works very well and unlike bilinear creates no ugly cross formed artifacts when magnified. When used for minification though bicubic tends to create aliased images. This is why for instance Paint Shop Pro when resizing with the "smart resize" option uses bicubic for magnification and bilinear for minification.
Probably but I'm not a coder so the various algorithms are a bit lost on meNick said:
I swear by Photoshop though and so do many others. I would think that their bicubic filter works as intended.His bicubic implemention leaves something to be desired though. It is extremely blurry and looks horrible to say the least, especially when upsampling. In Photoshop you would have to use a blur filter after using the bicubic filter to make it look like that. Blurring the image also helps reduce the halo effect.Nick said:
I'm really impressed with the result. Too bad it doesn't work as well with photos (complex shapes). I'd still prefer a regular bilinear filter for most uses. Though I have to say that your filter is vastly superior to many commercial resampling solutions which are gimmicky at best.MaxSt said:
scaled 300% - hq3x filter.
The halo artifact is shown quite clearly here. A bilinear filter would produce a much better result.MaxSt said:
scaled 300% - bicubic filter from Photoshop.
That Xin Li filter is very impressive. Gives a very natural result. Do you know if it is possible to get an application or a plug-in that supports it?MfA said:
Well, it's been a while since I used PSP but with Photoshop it's the other way around. Bicubic is superior for downsampling while bilinear works better for upsampling. I've also noticed that Photoshop generates a far less blurry result when using bicubic filter than the examples Nick linked to. So either Photoshop's bicubic implementation is indeed wrong (which I doubt) or the other apps add a little blur to remove jaggies and halo effect.Humus said:
I'd just like to restate I really like it. Although the only 'practical' use I see to it is increasing the texture quality for old games.
In the case of non-proprietary formats ( which is very rare though, sadly ), this could be really great! Or the programmer could do it...
Uttar
In the case of non-proprietary formats ( which is very rare though, sadly ), this could be really great! Or the programmer could do it...
Uttar
The site with the demo images has nagware which implements it. Never tried it though. Personally I feel his hybrid methods produces an image which looks too much like a mosaic.
I should add that Xin Li uses a least means square optimization of a 4 pel (or larger) filter across a neighbourhood of the point to be interpolated (the filter tries to approximate a pixel by its horizontal and vertical neighbours). Some might recognise this as a MMSE predictor, calculating this predictor involves inversion of a 4x4 matrix ... which for 2D unfortunately is unstructured. You wouldnt want to do this in a pixel shader
Even in new games we often have a smaller than 1 texel/pixel ratio, better magnification isnt just usefull for old games.
I should add that Xin Li uses a least means square optimization of a 4 pel (or larger) filter across a neighbourhood of the point to be interpolated (the filter tries to approximate a pixel by its horizontal and vertical neighbours). Some might recognise this as a MMSE predictor, calculating this predictor involves inversion of a 4x4 matrix ... which for 2D unfortunately is unstructured. You wouldnt want to do this in a pixel shader
Even in new games we often have a smaller than 1 texel/pixel ratio, better magnification isnt just usefull for old games.
Similar threads
