Initial benchmarks for Qualcomm's Snapdragon 845 CPU, which we'll see in the upcoming Samsung Galaxy S9, tip a 20-40 percent increase in web browsing performance over the Snapdragon 835, and a 15-30 percent improvement in graphics performance.
That's great, but Qualcomm says it only tells part of the story.
First, the Good News
The best benchmarks—mostly, PCMark—aren't awful. Rather than just churning the CPU, they try to tell you something about real-world task performance. And looking at AnandTech's benchmark report (the most comprehensive one to come out so far), real-world task performance is good. The Snapdragon 845 reference device easily outpaced all existing Android phones on simulated work tasks, although on web browsing, Apple's notoriously efficient Safari browser keeps iPhones ahead.
AnandTech notes that the Snapdragon 845 does better on "web workloads" rather than overall CPU performance, but I don't see that necessarily as a bad thing. The chip is optimized for actual applications, as opposed to synthetic number-crunching.
This will, hopefully, calm nerves set alight by earlier reports that the Samsung Exynos version of the Galaxy S9 outperforms the Qualcomm version on Geekbench, which is one of those "churn the CPU" benchmarks. Geekbench is frequently used in comparisons because it's very fast to run, reports simple result scores, and works across multiple OSes, but it's very limited in what it actually measures.
Both versions are ARM instruction-set compatible, but the Exynos 9810 is designed differently from the Snapdragon 845. They're both 10nm chipsets, but the 9810 uses four custom Samsung cores, four smaller ARM Cortex-A55 cores, and an ARM GPU, while the Snapdragon uses four ARM Cortex-A75 derivative cores, four A55 derivatives and a custom GPU.
What Benchmarks Don't Test
But the traditional CPU/GPU benchmarking applications only test a small part of these chipsets' performance, as there are a lot of other parts of the chipset that Qualcomm is quite proud of.
Start with the X20 LTE modem. Modem performance matters a lot in real-world smartphone use; it doesn't matter how fast the CPU can render web pages if it's just spinning its wheels waiting for data. It takes an expensive piece of equipment or a large, crowd-sourced data set to be able to tell the differences between LTE modem performance.
Fortunately, we have both, which is how we discovered that Qualcomm-based iPhones outperform Intel-based iPhones. But you can't test modems using an inexpensive benchmarking application.
The Hexagon DSP and Spectra ISP, for instance, aren't heavily tested by benchmarks. The DSP often reduces the power consumption of tasks, rather than increasing their speed, and the ISP takes care of image processing from the phone's built-in cameras. Benchmarking applications tend not to test camera or battery.
So What Do We Think?
The initial Snapdragon 845 benchmarks are encouraging. They also set expectations: this is going to be a stepwise improvement in CPU performance, and new smartphones won't necessarily feel much faster in basic UI tasks.
The Snapdragon 845 technologies I'm most curious about, and most want to see, are Ultra HD Premium displays, active depth sensing AR cameras, and improved battery life, especially for true wireless headsets. None of that stuff shows up on benchmarks, and all of that stuff is dependent on other components: new screens, new cameras, and new earbuds.
Unlike with Apple, Qualcomm can't tightly tie its processor improvements to other smartphone features it's building, because Qualcomm doesn't build smartphones. So there's a bit of an act of faith there. Qualcomm has laid down exciting possibilities, but the Snapdragon 845 will only be truly exciting if smartphone makers decide to pick those possibilities up.