FutureMark & NVIDIA Statements
Posted by: [NT] on: 06/03/2003 05:51 PM [ Print | 18 comment(s) ] · 3527 views
We just received this email from Futuremark:
Dear Member of the Press:
Following is a joint statement from Futuremark and NVIDIA relating to 3DMark03 and optimizations.
Futuremark Statement: For the first time in 6 months, as a result of Futuremark's White Paper on May 23rd, 2003, Futuremark and NVIDIA have had detailed discussions regarding NVIDIA GPUs and Futuremark's 3DMark03 benchmark. Futuremark now has a deeper understanding of the situation and NVIDIA's optimization strategy. In the light of this, Futuremark now states that NVIDIA's driver design is an application specific optimization and not a cheat. Read More
The world of 3D Graphics has changed dramatically with the latest generation of highly programmable GPUs. Much like the world of CPUs, each GPU has a different architecture and a unique optimal code path. For example, Futuremark's PCMark2002 has different CPU test compilations for AMD's AthlonXP and Intel's Pentium4 CPUs. 3DMark03 is designed as an un-optimized DirectX test and it provides performance comparisons accordingly. It does not contain manufacturer specific optimized code paths. Because all modifications that change the workload in 3DMark03 are forbidden, we were obliged to update the product to eliminate the effect of optimizations identified in different drivers so that 3DMark03 continued to produce comparable results. However, recent developments in the graphics industry and game development suggest that a different approach for game performance benchmarking might be needed, where manufacturer-specific code path optimization is directly in the code source. Futuremark will consider whether this approach is needed in its future benchmarks. NVIDIA Statement NVIDIA works closely with developers to optimize games for GeForceFX. These optimizations (including shader optimizations) are the result of the co-development process. This is the approach NVIDIA would have preferred also for 3DMark03. Joint NVIDIA-Futuremark Statement Both NVIDIA and Futuremark want to define clear rules with the industry about how benchmarks should be developed and how they should be used. We believe that common rules will prevent these types of unfortunate situations moving forward.
Dear Member of the Press:
Following is a joint statement from Futuremark and NVIDIA relating to 3DMark03 and optimizations.
Futuremark Statement: For the first time in 6 months, as a result of Futuremark's White Paper on May 23rd, 2003, Futuremark and NVIDIA have had detailed discussions regarding NVIDIA GPUs and Futuremark's 3DMark03 benchmark. Futuremark now has a deeper understanding of the situation and NVIDIA's optimization strategy. In the light of this, Futuremark now states that NVIDIA's driver design is an application specific optimization and not a cheat. Read More
The world of 3D Graphics has changed dramatically with the latest generation of highly programmable GPUs. Much like the world of CPUs, each GPU has a different architecture and a unique optimal code path. For example, Futuremark's PCMark2002 has different CPU test compilations for AMD's AthlonXP and Intel's Pentium4 CPUs. 3DMark03 is designed as an un-optimized DirectX test and it provides performance comparisons accordingly. It does not contain manufacturer specific optimized code paths. Because all modifications that change the workload in 3DMark03 are forbidden, we were obliged to update the product to eliminate the effect of optimizations identified in different drivers so that 3DMark03 continued to produce comparable results. However, recent developments in the graphics industry and game development suggest that a different approach for game performance benchmarking might be needed, where manufacturer-specific code path optimization is directly in the code source. Futuremark will consider whether this approach is needed in its future benchmarks. NVIDIA Statement NVIDIA works closely with developers to optimize games for GeForceFX. These optimizations (including shader optimizations) are the result of the co-development process. This is the approach NVIDIA would have preferred also for 3DMark03. Joint NVIDIA-Futuremark Statement Both NVIDIA and Futuremark want to define clear rules with the industry about how benchmarks should be developed and how they should be used. We believe that common rules will prevent these types of unfortunate situations moving forward.
« New Turtle Beach Santa Cruz Driver For Windows · FutureMark & NVIDIA Statements
· IT-News From India »
Comment
|
JakeMcLain Unregistered |
#44455 Posted on: 06/03/2003 06:09 PM
So basically: NVIDIA is very good at optimizing (+24%) and ATi is very bad at cheating (+2%). (This is a joke, nofi) |
Comment
|
shaderboy Unregistered |
#44456 Posted on: 06/03/2003 06:55 PM
This is disugusting. Up to this point i was all behind futuremark. 3DMark now has no value whatsoever. What caused this i wonder? nvidia strongarm tactics ? will nvidia be rejoining the beta program(along with its money)? So tell us futuremark, what exactly is a cheat now then? It the detonator 50's render a totally blank screen and score 100,000 points is that fair? Is it an application specific optimisation? It must be !! 3DMark has already been uninstalled from my system and wont be going back on ever, after all i have a 9800 pro, the fasteest real world card you can get, i no longer need 3DMark to tell me that, there are other applications that will. |
Comment
|
El_Coyote Unregistered Posts: 503 Joined: 2002-12-18 |
#44459 Posted on: 06/03/2003 07:25 PM
oooh this is sooooo fucked up :P "In the light of this, Futuremark now states that NVIDIA's driver design is an application specific optimization and not a cheat ." <-- i wonder how many money that was worth. gods optimations doesnt work by cropping the IQ to bits! |
Comment
|
Beastage Unregistered Posts: 0 Joined: 2003-06-26 |
#44460 Posted on: 06/03/2003 07:28 PM
There is nothing on FM website as one that is a regular in FM chat and boards I say wait for FM statement first ... as long as it nvidia only I wouldn't trust it |
Comment
|
shaderboy Unregistered |
#44461 Posted on: 06/03/2003 07:30 PM
maybe but why then are treads about the nvidia cheating issue being deleted at light speed on the futuremark forums ? |
Comment
|
Gemini Unregistered |
#44464 Posted on: 06/03/2003 08:52 PM
Hail to the allmighty Dollar... |
Comment
|
robertmarley Unregistered |
#44465 Posted on: 06/03/2003 09:04 PM
"why then are treads about the nvidia cheating issue being deleted at light speed on the futuremark forums ?" Probably because saying how "fucked up" things are, isn't the kind of material any respected company wants running around their forums. Apart from that, if this is touchy ground, they don't want any more trouble that what is necessary. I believe people are missing the what the e-mail was about to be honest. I'm not sure if anyone reads the comments made by John Carmack and others, and what the actual difference between nVidia's "optimization" and ATi's "optimization" was. I think not.  It seemed ATi's optimization (atleast the one I saw) left the outputted image totally unmolested, where as nvidia's approach was not really a dynamic one.
There's no point beating up benchmarking companies because of this - They are probably between the devil and the deep blue sea at this moment in time. |
Comment
|
shaderboy Unregistered |
#44466 Posted on: 06/03/2003 09:15 PM
Rubbish, futuremark are guilty and they know it. The way is now open for GPU makers to do anything they like. Catalyst maker has said on the rage3D forums that he will be producing a 'special' driver just to see how high the 3dmark score can be cheated up to. This driver will not be an official release of course (ATI are not Nvidia!). Futuremark should have stuck to their guns and refused to allow any optimisations at all. |
Comment
|
RiCO Unregistered Posts: 268 Joined: 2002-05-28 |
#44469 Posted on: 06/03/2003 10:41 PM
'application specific optimization' are just nicer words for 'cheating' |
Comment
|
robertmarley Unregistered |
#44473 Posted on: 06/03/2003 11:43 PM
Be quiet, ShaderBoy.. I have yet to be reduced to the low-level of immaturity that brings me to all of this stupid "anti-everything behaviour". By the way.. 'ShaderBoy', when taking part in a debate like this, where are your intelligent comments, which i've yet to see. Next, we will have a barrage of "nVIDIA SUCKS!!!!" posts from you, followed by a stream of late anti-war protests, and whatever else you feel the need to get involved in. To be pedantic, SSE, 3DNow and whatnot are company specific optimizations, yet we accept them with open arms in benchmarks. Albeit, AMD have now got Intel's MMX, SSE and as of late, SSE2, although they tend to be weaker than the "real thing" so to speak. You will turn around and tell me that CPU specific optimizations have nought to do with Benchmarks, and dodgy frame-for-frame clipping techniques, but the general idea has been said for actual optimizations that do not destroy the meaning of the benchmark's code at all. (Unlike nVidia's pretty bad move) |
Comment
|
GlowWorm Unregistered |
#44474 Posted on: 06/03/2003 11:50 PM
+ using the word cheat = i'll sue your butt off. |
Comment
|
GlowWorm Unregistered |
#44476 Posted on: 06/03/2003 11:51 PM
+ accusing a company of cheating = i'll sue your butt off unless you retract...! |
Comment
|
Mertsch Senior Member Posts: 2708 Joined: 2002-08-22 |
#44492 Posted on: 06/04/2003 01:44 AM
who cares about cheating on a stupid benchmark ?! come on FM get 3D Mark04 and forget about the BS03 |
Comment
|
neubsi Unregistered |
#44495 Posted on: 06/04/2003 02:02 AM
Seems Nvidas lawyers have doen their job very good;) |
Comment
|
popalazarous Unregistered Posts: 5 Joined: 2004-01-03 |
#44517 Posted on: 06/04/2003 08:29 PM
am i getting this straight. ATI optimized, nvidia changed code. We all love optimization of drivers which is fair enough. But changing code is different, thats cheating?. Is this the truth of it. Nvidia have a lot of clout and money so thats why they can almost do what they like. personally i prefer ati because they were honest about what happened. I have no preference over which is best. I had an nvidia card nut changed to ati because it was better value for money thats it! |
Comment
|
popalazarous Unregistered Posts: 5 Joined: 2004-01-03 |
#44518 Posted on: 06/04/2003 08:31 PM
money talks! |
Comment
|
shaderboy Unregistered |
#44525 Posted on: 06/04/2003 10:40 PM
Umm excuse me ??? My name may suggest i know what im talking about, i actually wite shader code! Just because i disagree with you it doesnt mean im immature, just more informed maybe. I dont hate nvidia at all, why is it that anyone whos got a bad word to say about them must be an ATI fanboy ??? |
Comment
|
Melll Unregistered |
#44537 Posted on: 06/04/2003 11:37 PM
Wow.. This is either going to break the word we all know as Benchamarking, or it will be known to us all as CHEATING the products!!! WHY do they charge $500.00 for a card that people want for the best quality period, and have them get a reduced piece of crap that we all thought would provide us with THE WAY ITS MEANT TO BE PLAYED yeah that the way we want to play??? By NVIDIAS RULES?!!! CMon this is making me furious... I will never touch an Nvidia product ever again, period |
