You often hear in the software industry that performance optimization is one of the last steps in the software development process. That bodes well for Windows 8, considering at the early stage of Developer Preview—even before we've seen an actual beta—the nascent operating system is getting widespread praise for its performance, particularly in startup times.
Anecdotal evidence is always encouraging, but I decided to run some very early tests on the OS to see if the reports were wishful thinking or if there was a real, measurable boost in speed. Along with startup and shutdown times, I used several standard industry benchmarks to compare Windows 8 performance with that of Windows 7 running on the same machine. Long story short: unless the code takes a turn for the worse in the next year or so, we can look forward to some speedier computing once Windows 8 is released.
I tested on a 3.4GHz quad-core PC with 4GB RAM running Windows 8 Developer Preview and Windows 7 in dual-boot configuration. Both OS installations were pristine, with no extraneous apps or data other than what came with the OSes to slow them down. I ran each test three times and took the average (aside from PCMark 7, which takes about a half hour to run and has its own built-in redundancy).
Startup Time
The time it takes before you can start using your computer is critical. The less waiting for the system to be ready, the better. Before Windows 7 was released, one of the hopes planted by Microsoft was that the OS would start up in less than 15 seconds. For most of us, that never came true. This is a topic that Microsoft's Windows 8 team has blogged about at length, and it's one of the more compelling aspects of Windows 8.
I've long been frustrated by how Windows 7, even after the promises of quicker startup time, takes significantly longer to get going than Ubuntu does on the same machine. Windows 8 Developer Preview, despite its pre-release status, makes huge advances in this area, and with newer hardware that uses a UEFI (Unified Extensible Firmware Interface) hardware boot process. And Windows 8 running on ARM-based mobile tablets will likely produce even faster startup times, but I wanted to compare with existing Windows 7 hardware, which Microsoft intends to fully support in addition to the more lightweight newcomers.
But even on less-forward-looking hardware, Windows 8 does some new tricks to speed startups: It takes advantage of multiple processor cores, uses stored kernel session data in the Windows hibernation file , and saves time by not completely enumerating all of a system’s drivers every time it boots.
The times I recorded were from pushing the power button on the PC to desktop display. This included the 15 seconds for BIOS boot, and that plus Windows 8's multiboot utility made the times longer than either OS would be without. Nevertheless, the improvement in Microsoft's next OS was striking: Windows 8 took a full minute less time to boot than Windows 7 on the same fairly powerful machine.
Shut Down Time
Shutdown time, long ago a major point of contention for windows users, was another story altogether: I saw no improvement whatsoever in this yardstick when moving from Windows 7 to Windows 8. The time remained between 10 and 11 seconds, which isn't shabby, but I recall Apple's pulling off a remarkable improvement in shutdown time for its update from Leopard to Snow Leopard.
PC Benchmarks
Geekbench 2.1 is designed to test processor and memory performance, but the operating system plays a part in the performance of any app being run, including benchmarks. It generates a number using a baseline configuration equal to 1000, so a higher score equates to better performance. If nothing else, running the test can verify whether a change in OS hasn't slowed down the benchmark. On the 64-bit version of the test, my PC produced a result of 8955 for Windows 7 and 9014 for Windows 8 Developer Preview. I was surprised to actually find this slight improvement from Windows 7 to Windows 8, to the tune of 0.6 percent, and the results on each of the multiple test runs were very consistent. This is a slightly bigger bump on the test than I found when comparing Windows 7 with Vista.
PCMark7 runs through a battery of tests designed to evaluate storage, computation, image and video manipulation, web browsing and gaming performance. It produces a result in PCMarks, with a higher number begin better. Windows 8 Developer Preview bested its parent with a score of 2221 compared with Windows 7's 2075, for a 7 percent improvement. This is less than the nearly 15 percent improvement I saw in moving from Vista to Windows 7, but that was on released code. Windows 8 could equal that advance by the time it's fully optimized by Microsoft and released. Still, a 7 percent gain isn't too shabby for pre-release software, where you might expect to see a drop-off rather than an improvement.
Here's the full table of my results:
| Benchmark | Windows 7 | Windows 8 | Percent improvement |
|---|---|---|---|
| Startup time (min:sec) | 1:32 | 0:32 | +65 |
| Shutdown Time | 10 | 11 | -10 |
| Geekbench 2.2 (higher is better) | 8955 | 9014 | +0.6 |
| PCMark 7(higher is better) | 2075 | 2221 | +7 |
| Google V8 version 6 (higher is better) | 2741 | Metro: 1874 Desktop: 3066 |
Metro: -31 Desktop: +12 |
| Mozilla Kraken 1.1(lower is better) | 13897 | Metro: 14288 Desktop: 13385 |
Metro: -3 Desktop: +4 |
(Best scores on green background.)
Browser Benchmarks
I tested both of Windows 8's Internet Explorer 10 apps—both the new Metro style version and the Desktop version, against Internet Explorer 9 on Windows 7. I used two browser benchmarks, Mozilla Kraken 1.1 and Google's V8 Benchmark Suite v.6, since these show up a lot more variation among browser versions than the former favorite, SunSpider.
The general trend of my results was a bit odd: When I ran the benchmarks in Windows 8's Metro-style version of IE10, performance was a bit slower than IE9 on Windows 7, but when I ran them in the Desktop version of IE10, they returned slightly better performance than in Windows 7.
Google's V8 Benchmark Suite performs several JavaScript calculations, such as ray tracing simulation and decryption, and memory management. It returns a number relative to a reference system, so a higher score is better. Windows 8's Metro version of IE10 lost ground from Windows 7 at a 31 percent clip, while the Desktop version of Windows 8's IE10 browser improved on IE9 by 12 percent. We can only hope that as we get closer to release, the Metro browser will at least equal its Desktop counterpart.
The same story showed up on Mozilla's Kraken benchmark. This test takes longer to run, and Mozilla claims it's the most real-world among the JavaScript benchmarks. Its results are timed in milliseconds, so a smaller result is better.
Windows Speeds Up
These tests put some data behind what anyone who's played around at all with Windows 8 Developer Preview knows: It sure feels faster than Windows 7 on the same computer. Microsoft clearly still has some optimization to do, especially for the Metro version of the Internet Explorer 10 browser and for shutdown speed. But the stellar startup-time improvement and the overall snappiness of this first publicly available version of Windows 8 are certainly encouraging for the future of Windows. Have you tried Windows 8 yet? If so, be sure to tell us what you think of its performance in the comments.
For more from Michael, follow him on Twitter @mikemuch.