B3D Shader Competition - any chance of Benchmarks too?

G

g__day

Regular
Reading the gathering body of evidence about how NVidia's top end performs relative to ATi's in DX9 PS 2.0 performance got me wondering.

If B3D are sponsoring a massive shader showcase - could you consider requesting benchmarks too? If we end up with a fantastic library of shading routines - it would be great if they showcased not only NVidia and ATi capabilites - but their relative performances level too.

What do you think?
 
g__day said:
Reading the gathering body of evidence about how NVidia's top end performs relative to ATi's in DX9 PS 2.0 performance got me wondering.

If B3D are sponsoring a massive shader showcase - could you consider requesting benchmarks too? If we end up with a fantastic library of shading routines - it would be great if they showcased not only NVidia and ATi capabilites - but their relative performances level too.

What do you think?
Click to expand...


I think it's unlikely because that would take focus away from shaders and ATI, and move it to relative performance and Nvidia.
 
Two birds, one stone.

Do you really want shaders that don't perform but look pretty? Is 3 fps fine for you? Weren't these shaders meant to be of the kind suitable to run on todays most advanced videocard hardware for an advanced game reading Dave's competition notice? To me that implies perfomance is critical too - and if ATi delivers but NVidia can't - or we see more of their relative strengths and weaknesses isn't this a good thing?

I'm not saying turn this into a benchmarking competition - just consider measuring shader throughput too.
 
g__day said:
Two birds, one stone.

Do you really want shaders that don't perform but look pretty? Is 3 fps fine for you? Weren't these shaders meant to be of the kind suitable to run on todays most advanced videocard hardware for an advanced game reading Dave's competition notice? To me that implies perfomance is critical too - and if ATi delivers but NVidia can't - or we see more of their relative strengths and weaknesses isn't this a good thing?

I'm not saying turn this into a benchmarking competition - just consider measuring shader throughput too.
Click to expand...

This is a competition, with a marketing focus. It's meant to show how good shaders are and how good shaders are on ATI hardware. If people want to run the shaders on Nvidia hardware and see a slideshow, they can, but I'm sure the last thing the bods at ATI marketing want is to get Nvidia mentioned a lot in the middle of their competition.

If people really want to measure shaders, I'm sure there will be those who will break out FRAPS, but making a solid benchmark that shows something useful is a bit more difficult than just putting a framecounter in the corner.
 
Bouncing Zabaglione Bros said:
This is a competition, with a marketing focus. It's meant to show how good shaders are and how good shaders are on ATI hardware. If people want to run the shaders on Nvidia hardware and see a slideshow, they can, but I'm sure the last thing the bods at ATI marketing want is to get Nvidia mentioned a lot in the middle of their competition.
Click to expand...

Oh contrae, it would be a marketer's wet dream to show NVidia couldn't run DX9 software as well or as fast as ATi can :)

Bouncing Zabaglione Bros said:
If people really want to measure shaders, I'm sure there will be those who will break out FRAPS, but making a solid benchmark that shows something useful is a bit more difficult than just putting a framecounter in the corner.
Click to expand...

And writing a solid shader will require a bit of tuning and debugging, the framework of benchmarking software analysis tools may assist do this.

Rather than just say no its bad - why not ponder with an open mind if you couldn't achieve both goals with little extra effort?

Does anyone else have a view?
 
g__day said:
Oh contrae, it would be a marketer's wet dream to show NVidia couldn't run DX9 software as well or as fast as ATi can :)
Click to expand...


Depends whether you want to run a negative campaign and spend your time getting your competitors name in the middle of your marketing.


g__day said:
And writing a solid shader will require a bit of tuning and debugging, the framework of benchmarking software analysis tools may assist do this.
Click to expand...

There's quite a lot more work between writing a shader and writing a "framework of benchmarking software analysis tools". You've turned a focussed shader competition into a request for benchmarking tools with eyecandy. The first rule in answering a question correctly is to answer the question that is being asked, not some other question altogether.

g__day said:
Rather than just say no its bad - why not ponder with an open mind if you couldn't achieve both goals with little extra effort?

Does anyone else have a view?
Click to expand...

I did ponder with an open mind, and I gave you my opinion and my reasoning. Just because you don't agree with it, you seem to be dismissing it as being "close-minded". The competition is set the way it is because of it's specific goals. Trying to change those goals to something else at this stage seems (IMNSHO) unlikely to succeed, and dilutes from the specific aim of showing some shader excellence.

However, there is nothing to stop someone putting in a benchmarking mode in their demo if they want to - I just think it would be extraneous to the stated aims of the competition.
 
Don't you think NVidia will one day run the same shaders - or has B3D and ATi got an agreement restricting NVidia from ever seeing them? If they haven't I'd expect them to appear on NVidia class h/w sooner or later.

The shader rules don't seem to mention performance is in or out - so why are you saying performance is out if these shaders are intented to be of the calibre to run in a future game engine? I have queried the scope of the competition to see if performance is in - in any way, shape or form - and if so how will it be measured. Doesn't this seem reasonable to you?

To your last point a clarification is arequest - not a change of rules because reading thru I didn't see if this was explicitly in or out of scope. I queried if your mind was closed because for some reason you believe it is well outside of scope. If this appears rude I wish to add I desire to insult no one here and apologise if you take offence.

Do you think a shader is excellent if it only runs at 0.001 fps? Well the answer of course is it depends on what you're after. A shader that could go into a future game engine has to be a bit faster than that - so why not measure its speed?
 
On the official competition page, we said that a benchmark mode would be welcomed. Whether Beyond3D or the entrants posts benchmark scores is undecided, since we will need to study the entries properly.

We have received number of registrations in the 20s. Deadline mentioned is supposed to be 31 August (italics... Dave should be making an announcement about why this is in italics). We have thus far received 3 actual submissions. Just FYI.
 
Reverend said:
We have received number of registrations in the 20s. Deadline mentioned is supposed to be 31 August (italics... Dave should be making an announcement about why this is in italics). We have thus far received 3 actual submissions. Just FYI.
Click to expand...

I don't know who made the submissions thus far, and who didn't...but the number of actual submissions should not impact the deadline.

If I was one of the participants who met the deadline, and then had it extended to meet the needs of others, I would consider that pretty unfair.
 
As an aside (and I'm not sure I'm remembering this correctly), doesn't RenderMonkey run on Nvidia cards? At the 9800 launch I'm sure ATi showed RM running realtime shaders alongside the only Nvdia DX9 card they could get hold of at the time (a workstation card) and the Nvidia card was very, very much slower.

Has anyone tried running RenderMonkey on the 5900U recently ? I don't have RM installed at the moment, so I can't remember if it has some kind of framecounter when you preview the shaders in realtime.
 
I think the benchmarking is a good thing.

This is how it should work; however, the competition should go ahead with no benchmarking, it gets judged winners are awarded and everybody congratulates them and the participants.

After a while, when a B3D staff member magically has time -- that's gotta be a level 9 spell. Then the benchmarking can begin and it'll be twice the mileage from one event. On the hole, I think it'd be a good idea.
 
Joe DeFuria said:
Reverend said:
We have received number of registrations in the 20s. Deadline mentioned is supposed to be 31 August (italics... Dave should be making an announcement about why this is in italics). We have thus far received 3 actual submissions. Just FYI.
Click to expand...

I don't know who made the submissions thus far, and who didn't...but the number of actual submissions should not impact the deadline.

If I was one of the participants who met the deadline, and then had it extended to meet the needs of others, I would consider that pretty unfair.
Click to expand...
Yes, we should be extending the deadline (Dave needs to announce this soon) but it is not to meet any participants needs.
 
Reverend said:
On the official competition page, we said that a benchmark mode would be welcomed. Whether Beyond3D or the entrants posts benchmark scores is undecided, since we will need to study the entries properly.

We have received number of registrations in the 20s. Deadline mentioned is supposed to be 31 August (italics... Dave should be making an announcement about why this is in italics). We have thus far received 3 actual submissions. Just FYI.
Click to expand...

Hey, this is a software development contest! Do you really expect a significant amount of submissions to arrive before Aug. 30, 23:30? :)
 
Snyder said:
Hey, this is a software development contest! Do you really expect a significant amount of submissions to arrive before Aug. 30, 23:30? :)
Click to expand...

You meant Aug 31, I assume. :) Every programmer knows that a deadline of Aug 31 means the end of the day. Or rather the morning of the next one. I'd have to query what time zone Beyond3D is on, so that I could hopefully get a few more hours into the next day. :) Still, I do plan to submit by the deadline. That's why I'm sitting right now by my computer, playing Warcraft, er, browsing the internet, er, I mean programming, yeah, that's what I meant.

I'm not planning to include a benchmark for now. That's because I'm getting 1FPS in a tiny window using the REF device.

A pity the deadline is to be extended. That'd make me feel like I need to add more stuff to the program, or alternatively will make me waste some more time instead of finishing the program. Besides, I need to finish something else I haven't started by Tuesday, and submitting after Sunday will eat into that.
 
Shader contest extention

Common, it's not really fair to the people who worked hard and got the shaders in on time to extend the deadline for people who blew it off.
 
Reverend said that the reason for the deadline extension doesn't have to do with people being late. While I can see that anyone who submitted the entry by the deadline would prefer to have fewer contestants, I don't think it's inherently unfair to give people more time. It's the shader that's being judged, not the person's work speed.
 
ET said:
Reverend said that the reason for the deadline extension doesn't have to do with people being late. While I can see that anyone who submitted the entry by the deadline would prefer to have fewer contestants, I don't think it's inherently unfair to give people more time. It's the shader that's being judged, not the person's work speed.
Click to expand...

Now, keep in mind that I'm not involved in this competition at all. So it's no skin off my back whichever way this goes.

It is unfair to those who meet the deadline, to extend the deadline. Particularly when it's this close.

When a deadline is established, this sets up certain work parameters, goals, and plans of attack. If someone knew they would have more time to submit their entry from the get go, a whole different approach / shader might be attempted. Now, this late in the game, you can't just give that time back.

If the reason for the supposed "extension" has nothing to do with the number of entries, then the solution is simple. Stick to the original deadline for submissions, and just postpone judgement / publishing of the results.

Perhaps if the deadline were extended a few weeks ago, I could see it being no problem. But this late? I would be royally pissed if I had worked around the premise of one deadline, got it done, and at the 11th hour, there was an extension.
 
yes, but the people who submit after the deadline will have more time then the ones who got down to it, and did the work. After all if we don't care about work speed, then give me ~10 years, and I'll win for sure :)
 
Joe, with all due respect, you are not in the competition, have not submitted an entry and therefore this competition does not affect you. Thanks for your comments, it is understandable as well as appreciated.

Let us handle this.
 
You must log in or register to reply here.
Back
Top