Pixel Shaders: float & integer colors

[asmatic]

Hello to everyone!

I'm using pixel shaders in order to do some very basic watermarking stuff with images.
I read a image as a A8R8G8B8 texture and mix it with another texture (same format) that contains the watermark data in a pixel shader,

However, in spite of being the two textures of integer format mi pixel shader must return a float4 value, not a int4 as I was testing or else I obtain artifacts in the final image. Even more, the original image must be sampled as a float, while sampling the watermark texture as a int or float gives the same results.

I'm worried because the integer data fits better with the needs of the watermarking and i'm fearing of loose information at de-watermarking stage.

Is there any way to work in a pixel shader as 100% integer data?

Thanks

[Cryect]

Curious why does a watermarking algorithm need a method to remove the watermark. In theory, if you want someone to have the watermark free version you just give it to them instead of the algorithm to remove the watermark. Especially considering information is lost when a watermark is added.

[asmatic]

sorry, rather than de-watermarking I should have named the function "detecting watermark"

The main problem is that I have to make some integer ops to put and to detect the watermark.

The watermark texture consists in 0's and 1's, but if I try to sample it in the shader as

int4 pix2 = tex2D(text,In.uv); I only obtain 0's
using:

float4 pix2 = tex2D(text,In.uv); I obtain 0 for 0's and 0'0039 for 1's.

Tha main problem is that when I found a 1 in the watermark texture (sampler text) I have to write a odd Integer color to the ouput, or an even if it was a 0.

So i must work with integers rather than floats in the pixel shaders howaever, the shader seems to be forced to work with floats.

Sould I compile to 1.x to work with integers?



I'm using HLSL and compile with 2.0 targets.

[Riff]

The specification states that color components are converted to floating point representation before being passed to the shader stage (at least this is the case in GLSL). These floating point components are always in the range [0.0, 1.0] so it should be no surprise when nearly all of the values get truncated to 0.0 if you store them to an integer variable. Most built-in functions don't care about the range of values passed into them as parameters. You are free to convert from floating point color components to integer color components, so long as you convert them back to floating point before they become shader stage outputs. If the color components are 8-bit, simply multiply your vector by (256.0, 256.0, 256.0, 256.0). This will map 0.0 to 0.0 and 1.0 to 256.0. Similarly, divide by the same vector to convert back to floating point.

The issue is that while integer data types are defined, there really arent any integer-specific operations defined. In most cases, there is no need to distinguish between integer and floating point values. Usually the only time it matters is when you want to use boolean logic requiring knowledge about how the value is represented internally. Until OpenGL and D3D support bitwise boolean operators I think its likely that you will have to simulate bitwise operations by using Floor, Ceiling, multiplication and division.

That being said, even if there were bitwise operators and a bigger distinction between floating point and integer representations, texture color components are still more than likely to remain passed in as floating point values.

[Cryect]

Just minor correction to point out shouldn't it be multiply by 255 not 256.

[Chalnoth]

Just minor correction to point out shouldn't it be multiply by 255 not 256.
Nope, 256. You're mapping 256 values between 0 and 255. I believe the proper interpretation of the fixed-point values is [0,1).

[Simon F]

Just minor correction to point out shouldn't it be multiply by 255 not 256.
I'd agree with that. 255 is the correct value.

[asmatic]

Yes, thanks I noticed that it sould be 255.
I'm using now floor() and forcing convesions by dividing and multplying.
Sadly i have discovered now a seriously bug: pixel / texels not correctly aligned. I need to work on that. :(

[Chalnoth]

No, it shouldn't be 255.

Multiplying by 256 would expand the [0,1) space between 0 and 255, as desired.

Multiplying by 255 would map 256 values onto 255 numbers. That wouldn't be good.

[asmatic]

But the space is [0, 1]. Because white color is {1,1,1} in the pixel shader and {255,255,255} in photoshop for example.

Regards

[Chalnoth]

I believe {1,1,1} would be clamped to {255,255,255}, not an exact representation of that number.

[Simon F]

No, it shouldn't be 255.

Multiplying by 256 would expand the [0,1) space between 0 and 255, as desired.
Chalnoth,
The range is [0,1], so 255 is the correct scale factor.

[LeGreg]

I agree with every other one: multiply to 255..

Chalnoth, you're mistaken here.
if you multiply by 255 you still have 256 different integer values (you have to count zero as a value).

OT : What is funny though is because of that you don't have a correct integer representation of 0.5 ;)

[Chalnoth]

Well, the only problem I have with this argument now is, with on-chip support for higher precision fixed point, things just won't map properly.

That is, if you map 11 bits to a number between 0 and 2 (as is done with FX12), then the 8-bit input from a texture won't map directly unless it's mapped as [0,1).

[Mintmaster]

I've stumbled on this as well when thinking about this subject.

While I too agree that the factor is 255, I'm left wondering how the math is done. How do IHV's build their multipliers? 1x1=1, but taking the high eight bits of 255x255 gives you 254. Whatever solution you have, all the math must be consistent.

Anyone know the details?

[Xmas]

NVidia details:
FX12 is [-2048/1024, 2047/1024]
FX9 is [-256/255, 255/255]

Add is straightforward, as well as FX12 mul. FX9 mul should add the MSB to the result.

[asmatic]

back to the original watermarking algoritm, i have made it work using floors, and scaling by 255 but only using Reference Rasterizer. Any attempt to use the algoritm with a HAL devide fails :( :cry:

[bloodbob]

I believe DXNEXT may require signed 32 bit ints wait till then I guess.

[aranfell]

I think that part of the confusion about the scale factor is due to the OpenGL spec sometimes using the term "fixed point" in a non-standard way. When producing an N-bit "fixed point" frame buffer value, OpenGL defines that the scale factor is (2^N)-1, not 2^N, so that the numbers [0..(2^N)-1] map to [0..1] (e.g., [0..255] map to [0.0..1.0]). This is explained in an obscure section at the end of one of the chapters of the OpenGL spec. The justification for this is simple: a pixel representation is not very useful if it can't exactly represent 1.0.

Arithmetic on these "fixed point" numbers has been handled a variety of ways in hardware, with varying degrees of accuracy. For example, one simple way to adjust the product of two such numbers is to treat them as true fixed point and then post-multiply by 256/255. That can be approximated by adding the high order bit of the result into the low order bit of the result, since 256/255 is approximately 1+1/256. Converting true fixed point to "fixed point" is simpler, since it requires multiplying by 255/256, which just requires a shift and a subtract.

[ehart]

Part of the confusion with the math here is that it is somewhat better to describe the math typically done for graphics in the range [0,1] as repeating fraction rather than fixed-point. Fixed point implies you are representing an exact number. Repeating fraction isn't exact, in that the digit sequence is logically repeated infinitely. (The number is rational, but it does not terminate, except in the 0 case.) I don't know that repeating fraction is the correct technical term, but it is the best one I have heard to differentiate this.

Now, back to the real problem.

Can you post a quick snip of the code? I imagine you are doing something like this for a single pixel in psuedo-C/GLSL:

ivec4 in; //input image
ivec4 out; //result
ivec4 mark; //watermark (no data except the last bit)

out = (in & 0xfe) | ( mark & 0x01);

I would advise trying to code it like this:

vec4 in;
vec4 out;
vec4 mark;

out = floor(in*127.0)/127.0 + mark;

I would probably also suggest folding the 1/127 into a multiply yourself, so you can control the operation more.

-Evan

[asmatic]

Of course, this is the coda thet puts the watermark/message in the original texture:
Dx's HLSL code:

struct PS_INPUT {
float2 uv : TEXCOORD0;
};

sampler image : register(s0); // original image
sampler watermark :register(s1); // texture containing the bit pattern of the watermark in the blue channel

float4 main (PS_INPUT In) : COLOR
{
float4 pix1 = tex2D(image,In.uv);
float4 pix2 = tex2D(watermark,In.uv); // read the bit
float tmp = floor(pix1.b*255); // transform it to integer space
if(pix2.b == 0) // it is a 0?
{
pix1.b = ((int)tmp/(int)2)*2; // then put an even number in the output image's blue channel
}
else
{
pix1.b = ((int)tmp/(int)2)*2+1; // put an odd value
}

pix1.b = pix1.b/255; // un-transform to float
return pix1; //put value
};

this is the shaders that makes the detection of the watermark:

struct PS_INPUT {
float2 uv : TEXCOORD0;
};

sampler image : register(s0);

float4 main (PS_INPUT In) : COLOR
{
float4 pix1 = tex2D(image,In.uv); // read texel
pix1.b = floor(pix1.b*255); // transform to integer
float tmp = fmod(pix1.b,2); // calculate module
if(tmp == 0) // if module was 0, the number was even
{
pix1 = 0; // so put a {0,0,0,0} to the output
}
else
{
pix1 = 1; // it was a odd number: put white {1,1,1,1} to the output
}
return pix1;
};

It works perfectly when the device is the REF reference software rasterizer, however, fails when using HAL device. Sometimes it returns inverted bit patterns, and often completely fails and returns random bit patterns.

The watermark texture is a black A8R8G8B8 texture. His n-th texel has b = 255 if the n-th bit of the watermark is 1, or 0 otherwise.

With REF rasterizer, after the detection phase I should obtain a black texture with white texels where a even odd number (watermark = 1) was found. There sould be no clues of the original image, only a bit pattern.
However, with HAL device, after the dection phase the result texture is like this:

http://campus.uab.es/~2066821/deswatermark.GIF

regards

[ehart]

I am guessing this is coming down to rounding rules etc. I would suggest a couple changes to your code to enhance your chances, and portabbility:


Don't use any integer variables, DX allow these to be emulated as floats. I am not sure how much wiggle room it allows for the handling.
Minimize your operations. Each one is adding error. You scale the numbers 4 times in your code. You only need to do it twice.


I would try this:



struct PS_INPUT {
float2 uv : TEXCOORD0;
};

sampler image : register(s0); // original image
sampler watermark :register(s1); // texture containing the bit pattern of the watermark in the blue channel

float4 main (PS_INPUT In) : COLOR
{
float4 pix1 = tex2D(image,In.uv);
float4 pix2 = tex2D(watermark,In.uv); // read the bit
float tmp = floor(pix1.b*127.0); //clip the last bit

pix1.b = tmp/127.0 + pix2.b/255.0; //add 8th bit based on watermark

return pix1; //put value
};


Getting bit exact integer math is pretty hard with floats. Would you care if I stuck this in a piece of sample code?

Assuming it works, it is something that several developers would like to see.

-Evan

[Xmas]

I'd suggest this:

struct PS_INPUT {
float2 uv : TEXCOORD0;
};

sampler image : register(s0); // original image
sampler watermark :register(s1); // texture containing the bit pattern of the watermark in the blue channel

float4 main (PS_INPUT In) : COLOR
{
float4 pix1 = tex2D(image,In.uv);
float4 pix2 = tex2D(watermark,In.uv); // read the bit
pix1.b = (floor(pix1.b * 127.9) * 2 + pix2.b) / 255.0;
return pix1; //put value
};

struct PS_INPUT {
float2 uv : TEXCOORD0;
};

sampler image : register(s0);

float4 main (PS_INPUT In) : COLOR
{
float4 pix1 = tex2D(image,In.uv); // read texel
float tmp = pix1.b * 255;
float bit = tmp - (floor(pix1.b * 127.9) * 2); return float4(bit);
};

[Xmas]

Evan, nice to see you here :D

I hate to say it, but your LSB clipping code won't work. When you insert 2/255 as pix1.b, you will get 0/255 as result. 255/255 will result in 255/255, but should be 254/255, so you can add 1/255 without overflowing.

The correct scaling factor is 127.5, but for numerical reasons it is better to use 127.9

[ehart]

Yeah, after I read your post, I realized it. I think I have been staring at too much code this weekend. :roll:

-Evan

[Simon F]

OT : What is funny though is because of that you don't have a correct integer representation of 0.5 ;)
Ahh but you can represent 1/3 :-)

[asmatic]

thanks Evan & Xmas.
The code now works perfectly with HAL device's.

I still don't completely get the trick of using 127.9
Yet I see that there is a lot to improve in my shaders , your solution is a lot more elegant.

Getting bit exact integer math is pretty hard with floats. Would you care if I stuck this in a piece of sample code?

Assuming it works, it is something that several developers would like to see.
Of course you can! In fact a lot of my work it's inspired your article at ShaderX 2.

Regards

[Xmas]

I still don't completely get the trick of using 127.9
Let's assume your texture contains integer values ranging from 0 to 255, which get mapped to float values ranging from 0.0 to 1.0 by dividing them by 255.
So
0 -> 0/255
1 -> 1/255
2 -> 2/255
etc.

Now you multiply those values by 127.5, use floor(), multiply by 2 and get this:
0 -> 0/255 -> 0 -> 0 -> 0
1 -> 1/255 -> 0.5 -> 0 -> 0
2 -> 2/255 -> 1 -> 1 -> 2
3 -> 3/255 -> 1.5 -> 1 -> 2
4 -> 4/255 -> 2 -> 2 -> 4
etc.

Seems to work... but only if x/255 can be represented perfectly accurate! Imagine the float representation of 42/255 is slightly lower than 42/255 itself. Then you get something like 20.99999 instead of 21, and the floor function gives you 20 instead of 21.
Using 127.9 makes sure the result is slightly above 21.

[asmatic]

Aha, I see your point. :wink:

I appreciate very much your help!

Thank-you!