(Credit: Jim Fisher)
We test a lot of different types of photo equipment at PCMag, but our focus is on cameras and lenses. To properly evaluate them, we conduct standardized lab tests in a controlled environment and go out into the real world to capture images and video. Doing so allows us to assess image quality in a predictable way and to challenge modern autofocus systems with various types of subjects a photographer is likely to encounter in real life. As PCMag's lead camera reviewer, I have nearly 15 years of experience covering the space and have worked to develop and refine our testing methodologies. Here's a look at what we do in each set of tests.
How We Test Cameras
In the lab, we evaluate image quality across the entire ISO range to see how photos will look in varying levels of light and to determine whether lenses can deliver crisp images on modern, high-resolution sensors. With an interchangeable lens camera, we always use the highest-quality lens we have available at its best f-stop to evaluate noise and detail. We rely on an X-Rite Color Checker Classic swatch chart to obtain a quantitative noise value from out-of-camera JPGs, and we frame it next to a foreign bank note to allow qualitative analysis of noise and detail for both JPG and Raw images.
We can also confirm the accuracy of manufacturer-stated shooting rates and buffer capacities, as well as test performance with multiple memory formats. We mainly rely on field tests to evaluate subject recognition modes, but also assess how well cameras handle on-screen images of people, animals, and other common subjects as needed.
Image quality isn't as huge a differentiator among digital cameras as it was in years past. The majority of the cameras we review match peers with the same size sensor in terms of picture quality. There are outliers, of course, which is why we still test everything. Autofocus intelligence, sensor readout speed, rolling shutter distortion, and video features are the areas in which today's models differ significantly.
On that note, we now put more emphasis on testing cameras outside the lab. Autofocus performance has become particularly important. To that end, we evaluate how well a camera tracks moving subjects and whether features like face and eye detection work as advertised.
Taking the camera out into the world also informs our impression of its ergonomics. Battery life, build quality, handling, viewfinder, and weather protection are all important factors to consider. If a manufacturer bills a camera as rugged or waterproof, we do our best to put it through the wringer. We drop tough cameras on the ground and take water-protected systems out in the rain. We're not able to perform teardowns, but certainly take them into account when they're available.
Field testing also allows us to determine how a camera handles different types of scenes, including those that push dynamic range. Capturing Raw images in these scenarios lets us see what photographers can expect when they lift shadows, curb highlights, and perform other edits in post. We use Adobe Lightroom Classic as our standard photo editing software.
Video tests are qualitative. We record at the highest quality and use the camera's available color profiles. When a flat profile is an option, we see how well it grades and utilize a standard LUT in Apple Final Cut Pro for processing. We pan the camera to look for rolling shutter distortion, check how well autofocus works in video, and try any special features, such as slow-motion capture or Raw recording. We also record long-form clips to see if a camera is prone to overheating.
Finally, we look at the camera's support system. If it's an interchangeable lens model, we consider the cost and quality of both first- and third-party lenses. We also test how well a camera's companion app works for image transfers or tethered shooting in the studio.
How We Test Lenses
Your choice of lens has a greater influence on the pictures you take than your camera. As such, we conduct thorough reviews and image analyses of optics, both in our test studio and in the real world.
We use one of two SFRplus test charts to check a lens' resolution, and for any distortion and vignetting. Our large chart is ideal for high-pixel cameras, but our testing space requires us to use a smaller, less precise chart for extreme telephoto angles. We evaluate a lens at various points across its focal range (for zooms) and at every full-stop aperture available. We include the figures in our reviews, using a scaled metric to account for the fact that sensor resolution influences the quantitative figures; high-resolution sensors score higher than cameras with lower pixel counts in our test software. Our reference scale follows below:
We run JPGs through Imatest software for analysis using an MTF50 baseline with line width per picture height (lw/ph) figures and refer to these numbers colloquially as "lines" in our reviews.
We perform all tests on a tripod and enable a self-timer to reduce the chance of vibrations affecting results. We aim to run every lens lab test at the camera's lowest ISO setting and with the electronic shutter, taking multiple samples to reduce the risk of misfocus or vibration. Occasionally, we use a higher ISO with longer lenses, in addition to the base ISO, since those optics are susceptible to vibration, even on a tripod. We've performed thousands of lab tests, and can recognize an oddball result from missed focus, shutter shock, or an unintended vibration.
Resolution is an important factor in lens performance, though not a hugely differentiating one: Most modern lenses are very, very sharp. As such, we also look at other aspects of optical performance. We check lenses for false color, for instance, in high-contrast areas on the plane of focus (lateral chromatic aberration) and in transitions between the plane of focus and background blur (longitudinal chromatic aberration, or LoCA).
We also consider the quality of the background blur itself, or bokeh. We scrutinize the shape of defocused highlights to see whether they are rounded or exhibit a cat's-eye effect, and we check for any onion-skin texture, extra-bright soap-bubble edges, or purple/green color shift (another form of LoCA).
We also try to get sunstars from lenses, an artistic effect that is commonly seen in landscape photography. Sunstars result from optical diffraction, so they tend to appear at very small apertures, usually f/16-22. At these settings, the lens draws small points of light with multi-point starbursts, an effect that some landscape specialists find desirable. We look to see whether the star's tines are crisp or streak into several points, and check for any false-color effects. At small apertures, most color issues come from internal reflections in the lens optics. These appear as green or rainbow-color orbs (commonly called ghosts) in images.
Flare is another concern for lenses when you take photos with the sun or another bright light source in or just outside the frame. We check how a lens handles backlit scenes in terms of contrast, false color, and internal reflections. If a lens comes with a protective hood, we make sure to properly attach it before testing.
Lenses for mirrorless camera systems commonly rely on in-camera corrections to compensate for distortion and vignette. We consider this an integral part of their design, so we don't knock them for these issues as long as the corrections don't detract from the image. But we do report any distortion that's visible in uncorrected Raw pictures and call out instances where distortion correction is so extreme that it affects picture quality.
Finally, we examine the focus performance of a lens. We test how quickly and accurately it locks onto a subject and drives focus across its range, and we also evaluate the manual focus experience. For the latter, we consider the size and dampening of the focus ring, as well as whether it supports a linear or nonlinear response (or both). For cinema use, a linear response is desirable because it allows you to repeat takes from shot to shot. For video-friendly lenses, we test the throw distance, the angle of rotation a lens requires to move from close to distant focus.
We also take focus breathing into account, since the angle of view for many lenses blooms or shrinks with changes in the plane of focus. This matters for video shots that rack from one subject to another, and for stills, this effect can make it difficult to stack multiple exposures to achieve a larger depth of field or a macro composition. Most photo lenses breathe to some extent, but we note when the effect is significant.
For our latest coverage, check out our camera page, or for more pointed buying advice, peruse the list of the best cameras we've tested.