(Credit: Intel)
CPU benchmarking application Geekbench has issued a warning about Intel Core Ultra 200 and 300 CPUs and their support for Intel's Binary Optimization Tool (BOT).
The tool is designed to enhance chip performance in certain games and applications, but John Poole, founder and president of Geekbench developer Primate Labs, suggests that it actually modifies the benchmark's code to improve its scores, VideoCardz reports.
Although that's not explicitly nefarious, Geekbench notes that with so few games and apps currently supported, the Intel BOT could misrepresent what the new chips can do.
Intel describes the BOT as "an added feature that users can choose to enable, which optimizes software to run games and applications more efficiently on Intel hardware." Geekbench, however, found that the BOT modifies the benchmark's code during execution to change the types of instructions it delivers.
Geekbench says Intel's BOT reduced the number of instructions required to complete the benchmark. In internal testing, it found that total instructions dropped from 1.26 trillion to 1.08 trillion when using BOT, a 14% reduction. That kind of application optimization is impressive and suggests there's real potential for BOT to enhance workloads in different apps, but with it so selectively applied, enhancing benchmark results has the potential to be misleading.
(Credit: Primate Labs)It could also misrepresent the results. Geekbench is designed to offer a range of workloads written in different styles, but Intel's BOT replaces some of those workloads with binaries designed and optimized for that CPU. With BOT enabled, you could argue that Geekbench isn't testing the CPU at all. Intel is.
"If BOT worked with every application, we wouldn’t have concerns with its use with Geekbench," Poole argues. "It is an interesting optimization technique that has some drawbacks (the two-second startup delay being one of them, especially for short-lived processes)."
To counter this, the next version of Geekbench, Geekbench 6.7, will include BOT detection and inform anyone running the benchmark that it may enhance results. Geekbench's internal tests suggest that enabling BOT in Geekbench 6.3 nets a 5.5% increase in single-core and multi-core benchmark scores.
Older versions of Geekbench, such as 6.3 and the more recent 6.6, include a standard warning that some Intel Core Ultra 200 and 300 CPUs may show enhanced scores due to BOT.