A very useful reply from Futuremark

[K.I.L.E.R]

Note that he said users are encouraged to speak out if IHVs are suspected of any way breaching the rules?
This reply should place many people's minds at rest.

-----------------------------Start e-mail
Dear K.I.L.E.R.,

Thank you for your feedback. We truly appreciate your interest and passion
towards our products - it is users like you who bring the satisfaction to
our work.

I want to assure you that we have told all the information we could to the
public. I understand that there are lots of rumors and also various
companies communicate in various channels information that sometimes is
erroneous. We have been hard at work in order to bring tangible good
developments that benefit the consumers of the PC technology products. I
lament that this process has taken quite some time, but it is essential that
we follow strict process and allow every company to have their voice heard
and their point of view taken into account.

As regards your specific questions:

1. Who is going to enforce these rules?

We will communicate strict and clear rules to IHVs, the media and users of
our products. Ultimately Futuremark will see that the rules are honored. If
we find out that any company is breaching the rules, we will investigate and
publish the results of our analysis. We encourage also our users to give us
feedback if they heavily suspect a breaching of the rules occurring.


Will the publicly announced rules apply to IHV such as nVIDIA?

The rules apply to all products from all companies. In short, no 3DMark
result will be valid and a comparable result unless the drivers and hardware
fulfill the rules.


I hope that this answers your question. Please don't hesitate to contact me
for any additional feedback.

Best regards,


Tero Sarkkinen
Executive Vice President of Sales and Marketing

Futuremark Corporation
tero@futuremark.com, http://www.futuremark.com
________________________________________________________________
Futuremark Corporation (US)
12930 Saratoga Avenue, Suite B-7. Saratoga, CA 95070 USA
Tel. +1-408-517-0131, fax +1-408-517-9119, cell +1-408-858-7662
________________________________________________________________


-----Original Message-----
From: K.I.L.E.R [mailto:ksaho@optushome.com.au]
Sent: Saturday, September 20, 2003 11:12 AM
To: tero@futuremark.com
Subject: A question about the upcoming guidelines


I have read this thread
(http://www.beyond3d.com/forum/viewtopic.php?t=8016) which was created
by a Futuremark employee informing the readers of the Beyond3D forum
about the upcoming rules. The term 'guidelines' was not used so
logically speaking there will be rules that reviewers and the general
public must comply with if the results were to be allowed for submission
via the ORB and granted official status.

As a consumer I have become distasteful over the handling of the nVIDIA
and Futuremark issue over the past months; the fact that nVIDIA could be
allowed to get away with posting results on the ORB that were classified
as “officialâ€￾ whilst using non-WHQL Detonator drivers. Futuremark’s
official stance at the time was that it would not allow for non-WHQL
drivers to be made official on the ORB, the worst part about this was
that Futuremark employees knew about nVIDIA’s tactics used in those
drivers. Things like lowering texture quality, detecting screen grabs
and upping the image quality when a screen grab was detected, replacing
shaders, lowering precision are among some of the things nVIDIA had done
to make their products look good in 3dmark03 and 3dmark01.

My main concern is that the upcoming rules will not be heeded by IHVs
such as nVIDIA and will allow for misleading ORB results.

I have 2 questions, which I would be exceptionally grateful if they can
be answered.

Who is going to enforce these rules?
Will the publicly announced rules apply to IHV such as nVIDIA?

Thank You

--
K.I.L.E.R
(Kinetic Intelligent Lifeform Engineered [for] Repair)

 

[CorwinB]

On the one hand, I agree with you : that's an encouraging reply you got.

On the other hand, I won't be quite satisfied until :
- I see the exact content of those rules
- I see how far exactly FM is ready to go to make sure the results on the ORB are actually relevant (both related to WHQL-drivers, and to removal of "butcheatoptimized" results)

At any rate, it's important to keep a level head and a "wait&see" approach WRT this announcement. For example, it could have such rules as "IHVs can submit optimizations to FutureMark, but the optimization should be mathematically exact, should be usable in a gaming context (ie no clipping planes), and FM gets the last word about wether this optimization is legit or not. Any driver-side optimizations unapproved by FM gets all the results of this drivers yanked from the ORB.". If enforced, said rules could be good.

 

[just me]

That appears to be an encouraging reply, let's see what manifests itself out of that tho'.

As far as WHQL'd drivers > FM has already implemented a filter for them on the ORB. Only problem with that > M$ no longer WHQL tests Win98/Me drivers. For those on 2000/XP: it doesn't matter, but not everyone is running 2000/XP & it is marketed as a 98/Me/2000/XP proggie. We've also seen WHQL doesn't mean the drivers don't have specific 'optimizations' for benchmarks.

 

[dan2097]

I personally think they should removed that filter and replace it with a futuremark approved /not futuremark approved driver thing