[B3D Article] "Ripping off the veil: The mysterious PS3 hardware scaler exposed"

[Mintmaster]

This is what that image would look like without any loss in vertical resolution.

I think he is. I tried scaling the 1920x1080 picture to 960x1080, then back up, and the difference in vertical resolution/detail compared to his 960x1080 image is huge.

Getting 2x supersampling makes a huge difference too.

When scaling to 960x1080, do it using point sampling (i.e. nearest neighbour). You can do it using MS Paint, actually. Then upscale it however you've been doing it. The only difference I see between this and macabre's pic is artificial texture detail from the higher mipmap selection of 1920x1080 vs. 960x1080. I can see aliasing within the texture - the kind that would shimmer in motion.

macabre, I think the best thing you can do is post the natively rendered 960x1080 and 1280x720 pics to prove the skeptics wrong.

 

[deathkiller]

Getting 2x supersampling makes a huge difference too.

When scaling to 960x1080, do it using point sampling (i.e. nearest neighbour). You can do it using MS Paint, actually. Then upscale it however you've been doing it. The only difference I see between this and macabre's pic is artificial texture detail from the higher mipmap selection of 1920x1080 vs. 960x1080. I can see aliasing within the texture - the kind that would shimmer in motion.

macabre, I think the best thing you can do is post the natively rendered 960x1080 and 1280x720 pics to prove the skeptics wrong.

I think that the problem is using a b-spline filter that blurs the whole image.

This is how the image looks using nearest neighbor for downscaling and lanczos for upscaling:

The difference is huge in textures with the macabre's 960 image (the aliasing looks the same):

 

[Thowllly]

I made a test myself. Using the VSM demo from elsewere on this board, I took a screenshot of the same view (the standard light view, since it doesnt compensate for aspect ratio when I resized the window), once with a 1280x1024 viewport, and once with a 640x1024 viewport, both with AA off. I cropped away the window borders with paint, then I rescaled the 640 image to 1280. Finally I cropped the images some more and saved them as jpg with Gimp at 95% quality (saving them as jpg from Paint gave very poor quality).

Full resolution:

Scaled up 2x:

The scaled image is blurrier and vertical edges are very jagged, but it's nothing like the huge difference there is in those images I commented on.

 

[one]

The Vista analogy is frankly meaningless in this case as we are dealing with a closed platform with a number of components selected, set, and in some cases built, by the same company making the software, with relatively narrow targets in mind - Vista is built in order to be both secure whilst still having provisions for having a much wider range of interoperability, internal and external hardware configurations all from vendors other than the software vendor.

Then I think you are underestimating the scope of the PS3 OS as a new venture - if Sony plan to build new Cell-powered products, obviously they'll try to reuse software too. And this is no more a closed platform world, they also declared they'd release PS3s with different hardware configurations much quicker - hardwares that run games are virtualized by the OS, not to mention the CPU virtualization is a premise of the PS3 OS while Vista supported it as the first Windows product. In addition to that the web browser is completely in-house built, not bought from somewhere.

 

[Mintmaster]

The scaled image is blurrier and vertical edges are very jagged, but it's nothing like the huge difference there is in those images I commented on.

There's no detailed chainmail texture in there. The only sharp texture detail (speckles on the ground) are indeed much blurrier. Besides, what do you see in macabre's comparison besides poor vertical edges and blurrier textures?

The difference is huge in textures with the macabre's 960 image (the aliasing looks the same):

Nope, there's much more texture aliasing in your first image. Look at the jagged patterns in the helmet. Look at the sparkles in the chainmail (well, sparkles frozen in time ). It's exactly what you'd see with a negative LOD bias.

 

[Thowllly]

There's no detailed chainmail texture in there.

No, but even the armours metal texture is super blurry.

The only sharp texture detail (speckles on the ground) are indeed much blurrier.

Yeah, but nowhere as extreme as the blurring on the original images

Besides, what do you see in macabre's comparison besides poor vertical edges and blurrier textures?

Nothing. It's just that the textures are way too blurry for the resolution they've been rendered at.

 

[deathkiller]

Nope, there's much more texture aliasing in your first image. Look at the jagged patterns in the helmet. Look at the sparkles in the chainmail (well, sparkles frozen in time ). It's exactly what you'd see with a negative LOD bias.

I was talking about edge aliasing in reference that I filtered the image to not apply 2x supersampling to the borders when downscaling. I know that doing that filters I create texture aliasing but my point was that b-spline filter cause a lot of bluring to the image.

 

[Thowllly]

I was at first only looking at the textures, but after looking at the edges I noticed something that will put an end to this discusion Scaling a image horizontaly should only add horizontal blurring, but macabres 960x1080 image clearly shows both horizontal and vertical blurring on the edges. To illustrate:

- First image is from 1920x1080 image, showing no blurring.
- Second is how a stretched 960x1080 image should look like, only horizontal blurring.
- Third image is the same as the second, but with a vertical blur filter applied.
- Fourth image is from the original scaled 960x1080 image, showing clear horizontal and vertical blurring.

There you have it, macabres example images are not correct, since the scaled image have clearly been unnecesarrily blurred vertically.

 

[macabre]

Okay here are the links to the full-size pics, so people can apply whatever filter they like:

720:
http://mitglied.lycos.de/opposed/720sc.jpg

960:
http://mitglied.lycos.de/opposed/960.jpg

Edit: sorry screwed up the 720 size ,I`ll upload a new one
Edit2: new one is up

 

[Thowllly]

Okay here are the links to the full-size pics, so people can apply whatever filter they like:

720:
http://mitglied.lycos.de/opposed/720sc.jpg

960:
http://mitglied.lycos.de/opposed/960.jpg

Edit: sorry screwed up the 720 size ,I`ll upload a new one
Edit2: new one is up

Thank you these images do have a lot more detail after scaling than the first scaled images you posted... ...but (sorry to be a pain in the ass) the new pictures don't exactly match the old pictures (apart from the different gamma), when I scale them up to 1920x1080 the character becomes much bigger than the one in the original images. Could you post a new 1920x1080 one that matches these new pictures in scale? pretty please?

 

[assen]

Thank you these images do have a lot more detail after scaling than the first scaled images you posted... ...but (sorry to be a pain in the ass) the new pictures don't exactly match the old pictures (apart from the different gamma), when I scale them up to 1920x1080 the character becomes much bigger than the one in the original images. Could you post a new 1920x1080 one that matches these new pictures in scale? pretty please?

The original 1920x1080 image doesn't matter - what we are arguing here is whether scaling up from 960x1080 or from 1280x720 results in a better 1920x1080 image. If 360 and PS3 could comfortably render at 1920x1080 in the first place, the word "scaler" would hardly be known even on this forum

 

[Thowllly]

The original 1920x1080 image doesn't matter - what we are arguing here is whether scaling up from 960x1080 or from 1280x720 results in a better 1920x1080 image. If 360 and PS3 could comfortably render at 1920x1080 in the first place, the word "scaler" would hardly be known even on this forum

Yeah, that what's the discussion should be about, but what I (and others) were arguing was that the original scaled image examples were flawed, (the 1280x720 one was flawed too, but it wasn't as obvious to me, because I expected there to be some vertical blurring there, while to 960x1080 was very obviously flawed to me, since it shouldnt have any vertical blurring). Anyway, even thought that's a slight derailing of the original discussion, it would still be interesting to see exactly how 1280x720 and 960x1080 compares to 1920x1080.

...hmm, or maybe it's not interesting at all... lets get back on topic; I think I'll do some tests with AA on, to see how much it can improve on those ugly edges....


Edit:
My previous test case, but this time with 4x AA:

1920x1080:

960x1080:

1280x720:

960x1080 looks nice, I think, but that might be just this particular scenario, I think 1280x720 should generally look better.

 

[macabre]

Thank you these images do have a lot more detail after scaling than the first scaled images you posted... ...but (sorry to be a pain in the ass) the new pictures don't exactly match the old pictures (apart from the different gamma), when I scale them up to 1920x1080 the character becomes much bigger than the one in the original images. Could you post a new 1920x1080 one that matches these new pictures in scale? pretty please?

http://mitglied.lycos.de/opposed/1080p.jpg

Yeah the pictures are a bit different, I didn`t save that scene after I did the first test.

 

[assen]

960x1080 looks nice, I think, but that might be just this particular scenario, I think 1280x720 should generally look better.

On 960x1080 the nearly horizontal lines are much better, while the nearly horizontal are worse; the 1280x720 is an equal opportunity jagfest. This is to be expected - nearly horizontal lines need good vertical detail, which the 960x1080 image provides.

 

[rounin]

I think the 960x1080 one looks better than the 720p one. It seems like the latter has more jaggies and blurrier.

 

[Mintmaster]

Yeah, that what's the discussion should be about, but what I (and others) were arguing was that the original scaled image examples were flawed, (the 1280x720 one was flawed too, but it wasn't as obvious to me, because I expected there to be some vertical blurring there, while to 960x1080 was very obviously flawed to me, since it shouldnt have any vertical blurring). Anyway, even thought that's a slight derailing of the original discussion, it would still be interesting to see exactly how 1280x720 and 960x1080 compares to 1920x1080.

The reason you don't see the flaws in the new images is because they don't have a lower mipmap selection than the 1920x1080 pic anymore. The camera is zoomed in by a factor of ~1.5, so now the new 720p pic has a model the same size (pixel-wise) as the old 1080p pic. Also, the lighting angle is different so the lower mipmaps in the old 960x1080 and 1280x720 pics mattered more for the bump mapping.

960x1080 looks nice, I think, but that might be just this particular scenario, I think 1280x720 should generally look better.

I never thought 720p would be a blowout win, but I did think the aliasing would be much less compared to 960x1080, as we have seen in these examples. The one thing bugging me about this scene you chose is that it looks really squished vertically, and thus most edges are horizontal. Any reason for that?

Anyway, the big thing to take away is that 12.5% more pixels can degrade image quality if you arrange those pixels in a strange way. However, since the 960x1080 mode is probably only going to be used by 1080i-only TVs, and since the 720p mode won't be abandoned, it should be a barely adequate solution.

 

[Shifty Geezer]

A selection of shifty scalings...​

Original Source Image

Creating a 720p upscale example

1. 720p = 66% of 1080p(i)
2. Downscale source to 66% with point sampling to simulate rendering directly to lower resolution
3. Upscale this to original dimensions with bilinear sampling

Pic on left is downsized original. Pic on right is 720p simulation scaled to 1080

Creating a 960x1080 upscale example

1. 960x1080 = 50% of 1080p(i) width
2. Downscale source to 50% with point sampling to simulate rendering directly to lower resolution
3. Upscale this to original dimensions with bilinear

Pic on left is downsized original. Pic on right is 960x1080 simulation scaled to 1080

Caveats

• Possible extra jagginess from pointwise downsampling versus proper rendering to target resolution.
• Opportunities for more complex scaling algorithms than straight interpolation
• Source image has a high contrast edge that might not be indicative of average edge contrast.
• Manual horizontal sampling. The software used didn't provide proper 1 dimensional bilinear sampling, so a manual simulation was produced with a 1D matrix blur and interleave masking. The exact result was weighted to reduce the blockiness a bit.

Try it - you might like it! Pick your own source images. They don't have to be full 1080p screenshots; you only need to use the right scaling ratios on any sized image. To get 720p, downsample to 66% of original than upscale with whatever algorithm you want. For the half-1080, downsample to 50% width, same height, and apply whatever upscaling algorithm you want.

 

[Thowllly]

The reason you don't see the flaws in the new images is because they don't have a lower mipmap selection than the 1920x1080 pic anymore. The camera is zoomed in by a factor of ~1.5, so now the new 720p pic has a model the same size (pixel-wise) as the old 1080p pic. Also, the lighting angle is different so the lower mipmaps in the old 960x1080 and 1280x720 pics mattered more for the bump mapping.

Wait.... what?! What exactly are you getting at? Are you still denying that the original scaled images were flawed? Didn't you bother looking at post #168? Where I clearly show that that the upscaled picture has a large amount of vertical blur applied? I mean, it's clear as day...

I never thought 720p would be a blowout win, but I did think the aliasing would be much less compared to 960x1080, as we have seen in these examples.

same here, but as I said, it might just be that special scenario...

The one thing bugging me about this scene you chose is that it looks really squished vertically, and thus most edges are horizontal. Any reason for that?

Yeah, I already explained that in an earlier post. I choose to use the light camera because it always keeps the same viewport aspect ratio when I resize the window to strange pixel ratios. (And the reason I choose the VSM demo in the first place was that it was the only usable thing I have on my harddrive at the moment). But of course this looks wrong, since the light camera probably uses a 1:1 aspect ratio.

What I could do is; instead of rendering at 960x1080 and 1280x720 and scaling up to 1920x1080, I could render at 480x1080 and 640x720 and scale up to 960x1080, that should look about right...

 

[Thowllly]

Sorry about the "double post", but I want to keep my discussion with Mintmaster about the flawed old pictures separate from the 960x1080 vs 1280x720 vs 1920x1080 tests.

Now all the source pictures have half the width they should have, to better match the viewport aspectratio, but that doesn't matter, since I crop them to fit on this page anyway.

"1920x1080" (960x1080):

"960x1080" (480x1080):

"1280x720" (640x720):

Now the "1280x720" image is clearly better on the majority of the edges, as it should be

 

[Mintmaster]

Wait.... what?! What exactly are you getting at? Are you still denying that the original scaled images were flawed? Didn't you bother looking at post #168?

Yup, I did, but my own analysis of the original pics tells me the reduced detail is from a reduced mipmap. I also see a much bigger difference between your third and fourth images than you do. If there is any vertical blurring, it is extremely subtle (much less than you did in the third pic) and not enough to explain the reduced texture quality. Reduced mipmap level also explains why the 720p pic has muddy highlights. Doing the same point sampling downscale from 1080p gives 720p more artificial texture detail also, so I think the comparison between the old 960x1080 and 1280x720 pics was still fair.

Anyway, it doesn't matter. We have new source images from macabre. I've attached another comparison in this post. It's not a tremendous difference, but you see vertical edge aliasing standing out and textures being blurrier, and the better horizontal edges are little consolation.

The lesson for the day is more pixels aren't always better.