Nearly All Mobile Device Makers Cheat on Benchmarks, Except Apple and Motorola
Following Tuesday's report that Samsung artificially inflates its benchmarking scores, well-respected hardware review site AnandTech has published evidence suggesting nearly all mobile manufacturers, with the exception of Apple and Motorola, use CPU/GPU optimizations to game benchmark tests.
Samsung and other OEMs use a variety of methods to enhance device performance when a benchmark is detected. For example, with its Galaxy S 4 Samsung raised its thermal limits (and max GPU frequency) to get an edge on certain benchmarks and also raised its CPU voltage/frequency to its highest state when a benchmark was sensed, a tactic engaged by multiple manufacturers like LG, HTC, and ASUS as well.
In the table below, Anandtech highlights devices that detect benchmarks and immediately respond with max CPU frequency.
With the exception of Apple and Motorola, literally every single OEM we’ve worked with ships (or has shipped) at least one device that runs this silly CPU optimization. It’s possible that older Motorola devices might’ve done the same thing, but none of the newer devices we have on hand exhibited the behavior. It’s a systemic problem that seems to have surfaced over the last two years, and one that extends far beyond Samsung.
AnandTech notes that it’s a continual "cat and mouse" game discovering which devices have optimized for which benchmarks, because targeted benchmarks must be avoided.
The only realistic solution is to continue to evolve the suite ahead of those optimizing for it. The more attention you draw to certain benchmarks, the more likely they are to be gamed. We constantly play this game of cat and mouse on the PC side, it's just more frustrating in mobile since there aren’t many good benchmarks to begin with. […]
There's no single solution here, but rather a multi-faceted approach to make sure we’re ahead of the curve. We need to continue to rev our test suite to stay ahead of any aggressive OEM optimizations, we need to petition the OEMs to stop this madness, we need to work with the benchmark vendors to detect and disable optimizations as they happen and avoid benchmarks that are easily gamed.
Despite all of the effort that OEMs put into benchmark optimizations, the gains are negligible. The impact on CPU tests revealed a 0 to 5 percent performance increase, and a less than 10 percent increase on GPU benchmarks.
Popular Stories
Macworld's Filipe Espósito today revealed a handful of features that Apple is allegedly planning for iOS 26.4, iOS 27, and even iOS 28.
The report said the features are referenced within the code for a leaked internal build of iOS 26 that is not meant to be seen by the public. However, it appears that Espósito and/or his sources managed to gain access to it, providing us with a sneak peek...
Software from an iPhone prototype running an early build of iOS 26 leaked last week, giving us a glimpse at future Apple devices and iOS features. We recapped device codenames in our prior article, and now we have a list of some of the most notable feature flags that were found in the software code.
In some cases, it's obvious what the feature flags are referring to, while some are more...
Last week, details about unreleased Apple devices and future iOS features were shared by Macworld. This week, we learned where the information came from, plus we have more findings from the leak.
As it turns out, an Apple prototype device running an early build of iOS 26 was sold, and the person who bought it shared the software. The OS has a version number of 23A5234w, and the first...
The first foldable iPhone will feature a series of design and hardware firsts for Apple, according to details shared by the Weibo leaker known as Digital Chat Station.
According to a new post, via machine translation, Apple is developing what the leaker describes as a "wide foldable" device, a term used to refer to a horizontally oriented, book-style foldable with a large internal display....
Apple is testing iOS 26.3, the next version of iOS 26 that will launch around January. Since iOS 26.3's testing is happening over the holidays, it is a smaller update with fewer features than we've seen in prior betas.
We've rounded up what's new so far, and we'll add to our list with subsequent betas if we come across any other features.
Transfer to Android
Apple is making it simpler...
Apple today released iOS 26.2, the second major update to the iOS 26 operating system that came out in September, iOS 26.2 comes a little over a month after iOS 26.1 launched. iOS 26.2 is compatible with the iPhone 11 series and later, as well as the second-generation iPhone SE.
The new software can be downloaded on eligible iPhones over-the-air by going to Settings >...
Apple seeded the second iOS 26.2 Release Candidate to developers earlier this week, meaning the update will be released to the general public very soon.
Apple confirmed iOS 26.2 would be released in December, but it did not provide a specific date. We expect the update to be released by early next week.
iOS 26.2 includes a handful of new features and changes on the iPhone, such as a new...
Apple released the AirPods Max on December 15, 2020, meaning the over-ear headphones launched five years ago today. While the AirPods Max were updated with a USB-C port and new color options last year, followed by support for lossless audio and ultra-low latency audio this year, the headphones lack some of the features that have been introduced for newer generations of the regular AirPods and the ...
