What Is Wrong With UT2003 Benchmark?

Published by

Just saw this @ G256.com:

UT2003 seems to have some kind of coding problem. The benchmark numbers do not add up. UT was a CPU dependant game, and Quake3 was a graphics card dependant game. UT2003 seems dependant on everything: RAM, Video, and CPU. But, it's framerate results are highly inconsistant. One thing I know is, Mark Rein is no John Carmack. I think the EPIC boys are pushing the limits of what they can handle.Take a look at this mess of benchmarks run on multiple cards. It makes absolutely no sense that at 1024x768x32 a GeForce3 TI-500 and a GeForce4 4600 get identical benchmark results.

Well Warp2Search has the answer for YOU! Hit Read More. ;-)

Benchmarking

The Unreal Tournament 2003 Demo comes with a large variety of benchmarking functionality. The easiest way to utilize it is by starting Benchmark.exe in the System directory of your Unreal Tournament 2003 installation directory. This will bring up a dialog box asking you to pick a resolution and when you press okay, it will run 4 consecutive tests (two level flybys and two botmatches). It is important to not touch the mouse before the final dialog with the results pops up. The result dialog displays the average framerate of the two flybys and the two botmatches.

The benchmark will always try to run at the highest detail settings so results will not be directly comparable with cards on which the game can't render at the highest detail level (e.g. TNT2 and Kyro I/II cards).

For the botmatches the Benchmark.exe is running a game with just bots, with the player in spectator mode. In case the engine can't run at the highest detail settings the outcome of the botmatch might differ leading to a different botmatch and therefore numbers not comparable to other results. This could also happen when comparing different kinds of CPUs. So either watch the botmatches carefully when benchmarking to ensure that they are the same every time or compare the number in brackets in benchmark.log. If the number is the same, the exact same sequence was rendered.

In the Benchmark directory you will find a couple of batch files which can be used to run individual tests and to create a big batch file for automating the whole process of running at multiple resolutions.

The game will log a lot of information and stats while in benchmark mode and the results can be found in the subdirectories of the Benchmark folder. Of the most interest is the CSV folder, which contains the myriad of stats the engine collects on a per frame basis in the form of a "comma separated variables" file which e.g. can be opened in Microsoft Excel.