For some, benchmarks provide bragging rights. For others, they’re a valuable comparative tool and a useful instrument for performance tweaking.
Benchmarks…what are they good for? Absolutely nothing, say some people. For others, they provide bragging rights. But for a select few, they’re a valuable comparative tool and a useful instrument for performance tweaking.
I like to presume that you’re in the latter camp, which is why many of my columns include benchmark results. And it’s also why I’m taking the time this month to assist you in interpreting those results and to help you employ the benchmark software tools effectively.
Benchmarks serve two purposes: They enable us to objectively compare various computers and components, and they provide a method for tweaking system performance. Whether we’re dealing with processors, memory, video cards, hard drives, optical drives, or other related components, benchmarks help us measure performance in a somewhat consistent, reproducible manner.
Two kinds of benchmark software enter into the performance equation: synthetic and application-based or real-world. I use both to provide the best overall performance analysis. I also utilize software that is affordable and readily available. In synthetic benchmarks, there are a slew from which to choose, including 3DMark03, 3DMark2001 SE, Atto Utilities, CDSpeed2000, HD Tach, IOMeter, PCMark2002, PowerStrip, Sisoft Sandra, SysMark2002, WCPUID, Winbench, and Winstone.
In real-world benchmarks (i.e.,. games), the most popular are “Comanche 4,” “Dungeon Siege,” “Half-Life,” “Jedi Knight II,” “Quake 3,” “Return to Castle Wolfenstein,” “Serious Sam SE,” “Serious Sam 2,” “Tom Clancy’s Splinter Cell,” and “Unreal Tournament 2003.” I’ve used many of them, but have settled on just a few as the “required” contingent of my default benchmarking suite.
With regard to synthetic benchmarks, 90 percent of my needs are met in three programs: 3DMark03, 3DMark2001 SE, and PCMark2002, all from Futuremark (formerly MadOnion). The first two benchmarks cover 3-D video testing, with 3DMark03 also providing basic CPU marks. The latter offers tests for CPU, memory, and hard drives.
In gaming benchmarks, my current suite consists of “Jedi Knight II,” “Return to Castle Wolfenstein,” and “Unreal Tournament 2003” (though I expect this list to change in the fall with the release of new cutting-edge titles). These real-world tests focus exclusively on video frame-rate performance.
What do these programs do? For the most part, synthetic benchmarks try to simulate what actually happens in software applications, but in a way that data can be viewed in greater detail. Real-world benchmarks, like games, provide results based on actual workload, but lack the minutia that synthetic benchmarks offer. Both have their place in the framework of performance testing. The bigger question is, what do these tests actually tell us?
Benchmarks like 3DMark03, 3DMark2001 SE, and application-based tests provided by games primarily tell us how well our system performs thanks to the video card stuffed under the hood. They inform us, for instance, whether our game rig runs better with an ATI Radeon 9800 Pro or an NVIDIA GeForce FX 5900 Ultra.
They also serve as a means of objectively tweaking video performance. For example, these tests will let us know if increasing or decreasing AGP aperture settings in the system BIOS or adjusting Direct3D or OpenGL settings in Windows has a positive or negative effect on video performance, and to what extent.
Software that tests CPU, memory, and hard drive performance individually is important, too. That’s where programs like PCMark2002 and, to a limited extent, 3DMark03 come into play. They tell us, for instance, if increasing the frontside bus or CPU multiplier settings or if adjusting the memory timing CAS latency and DRAM command rate boost performance and by how large a margin. They also inform us if defragmenting the hard drive, changing Virtual Memory settings (the size and/or location of the Windows page file), or limiting the programs and device drivers that load on boot-up have any beneficial or adverse effects, as well as if drive performance degrades as the drive heats up under sustained operation (an indication additional cooling is necessary).
Once you decide what benchmarks you’ll use, what procedure do you follow? If you’re starting from scratch with a fresh install of all your software and the operating system (I recommend Windows XP), first configure everything to your liking, then run Scandisk and Disk Defragmenter (or the scanning and defragmenting utilities of your choice) to check drive integrity and optimize your hard drive(s). Then image your system with a program like Drive Image or True Image as a safeguard. (I’ll discuss the issue of drive imaging in a future column, but briefly, it’s the process of making a compressed copy of a drive, complete with the operating system, and then either transferring it to a new machine or restoring it on an old drive whose data has become damaged.) Next, install the benchmark software and image the drive again. Now you can perform all your benchmarking and tweaking with the peace of mind that should you mess anything up royally on the software side, you can always restore your system to its default installation.
If you’re not beginning with a fresh installation of Windows XP (or your preferred OS), you’ll want to start by defragmenting your hard drive. Then run your scanning and defragmenting utilities to verify integrity and optimize your hard drive(s), image your system, install the benchmarks, and image again.
A word of caution: When utilizing benchmarks to optimize system performance, it’s vitally important not to rush the job. Make incremental changes one at a time and retest. Don’t make multiple changes all at once, or drastic ones. That’s a sure way to get yourself in trouble and end up with a system that won’t run. As for what to tweak first, there’s really no prescribed order. However, I suggest starting with the CPU, then the system memory, followed by the hard drive, and finally the video card.
In general, a resolution of 1,024 x 768 is good for general performance testing. You’ll want to run higher resolutions when it comes to benchmarking and tweaking your video card, but lower resolutions are fine for all other tests. If you really want to tweak and overclock heavily, I suggest researching the subject at sites like Tweak 3D and Motherboards.org as well as in such books as “The Book Of Overclocking” by No Starch Press.
Finally, while not benchmarking in the strict sense, make sure that your optical drives are operating at their best, too, as well as your modem and Internet connection (for multiplayer fans especially). Every little increase adds to total system performance. Verify, as well, that your case has sufficient airflow. Tweaking your system for utmost speed only to have it slow down due to overheating is a self-defeating process. Of course, you’ll want to tweak your OS for best performance. All tweaks combined will bear some effect on benchmark results and the speed at which your games strut their stuff.