Nikon’s Z9 and potential for computer photography

Nikon recently announced its new flagship camera, the Z9. This camera represents Nikon’s first mirrorless camera fully geared for professional photographers and hybrid shooters, with a huge spec list designed for use in the demanding fields of photojournalism, sports, nature, bird watching and any other usage situation that requires an incredible camera fast in high resolution. The $ 5,500 Z9 is also Nikon’s first camera to omit a traditional mechanical shutter, allowing it to achieve new levels of speed and autofocus performance.

Faster speeds are great, especially for sports photographers. But it is interesting to think about where this technology can be used to take traditional cameras in the future. This may be the first step towards larger format cameras that use the computational smartness that smartphone cameras have embraced for years.

The Z9 has a 3.2-inch multi-angle LCD screen that can tilt in both directions but cannot face forward.
Image: Nikon

Nikon has not mentioned things like computer photography for HDR-style images or the cyclic buffering that smartphones do to simultaneously capture up to nine or 10 images and combine them with each press of the shutter button. But the new 45.7-megapixel full-frame backlit stacked CMOS sensor is not far from what has been in phones for years, at least in terms of core design. This type of construction uses a sandwich architecture of sensor, logic board and dedicated RAM – which provides incredibly fast read speeds.

Today, it enables the Z9 to use a full-time electronic shutter with the fastest shutter speed of 1 / 32,000 of a second and achieve incredibly fast burst recording. It can capture 20 frames per second in RAW / JPG at full resolution or as fast as 120 fps at 11 megapixels, all without making any audible sounds (optional false shutter sounds can be activated for an audible cue). The new Expeed 7 processor and dual CFexpress / XQD card slots give the Z9 an alleged 1,000-shot full-resolution buffer in a high-efficiency compressed RAW, but it’s the fast readout speed of the stacked sensor that may be the key to the computational photography puzzle.

As the first of the major camera manufacturers to drop the mechanical shutter, it puts Nikon ahead of its competitors in the race against computer photography. Sony’s A1 and A9 lines have already used stacked sensors for fast read speeds, making electronic shutters viable for full-time work, and Canon’s upcoming R3 will use the same technology. Moving to a fully electronic shutter has been the logical next development for cameras, though it is up to Nikon to prove that its electronic shutter is up to the daily tasks and requirements of professional photographers right now.

The Z9’s stacked CMOS sensor is fast enough for blackout-free shooting while writing images to the card.
GIF: Nikon

Until now, camera makers ‘efforts to implement computer photography have been limited to features such as Olympus’ Live ND and Panasonic’s post-focus and in-camera focus stacking. Convenient features, yes, but these are page views compared to the paradigm shift that full computer photography implemented with each press of the shutter button could one day be. OM System, the recently renamed Olympus, recently promised to use computer photography technology in its next camera, but we’ll have to see if that’s the main focus or just another feature on the page.

Deep Learning, used in the new Z9’s object recognition autofocus system, has also been used to some capacity in the past by Olympus, Panasonic and Canon. It serves to improve autofocus tracking performance, but in the end, a mirrorless camera still captures a single image that is limited by the dynamic range of the sensor.

The primary barrier that is most likely to prevent cameras like the Z9 and other mirrorless cameras at the pro or enthusiast level with stacked sensors that go fully computationally, may lie in the data flow and image processing pipeline. Ten images captured simultaneously from a 45-megapixel full-frame sensor and combined into one file will be exponentially larger than the same collection of images taken from a smartphone sensor at a fraction of the size.

In addition, cyclic buffering is required to constantly write and rewrite images to the camera’s buffer in the background before pressing the shutter button. Even the Z9’s new processor may not be up to the task. In the smartphone space, CPUs are designed to be suitable for this treatment, even sometimes using dedicated hardware, but cameras are not built the same way. It is possible that there is still a need for more innovation at the CPU level from the camera manufacturers.

Processing power can be the ultimate technological barrier to full-size camera photography.
Image: Nikon

There are some obvious benefits to using computer photography. Almost any modern smartphone can create a balanced exposure with well-lit subjects, shadows full of visible details and visible clouds – all in the same frame. Advances like Night Sight and Night Modes let you do things that are much harder to accomplish with a standard camera, while Google continues to bring new computer tricks to keep subjects sharp when in motion, and Apple even allows RAW files with calculation data.

On the other hand, a photo taken with even the most advanced mirrorless camera today – while superior in sharpness and resolution – results in some victims, such as to blow out highlights or smash the shadow details in high-contrast day scenes. Achieving the same look as most smartphones requires at least some finishing and editing, ideally from a RAW file to be exported as JPG or another universal format. Computing photography that comes with dedicated camera systems can revive the camera market, though it can also take camera makers to finally figure out connected Wi-Fi apps that aren’t terrible – admittedly, another big order.

Cameras like the Z9 can be the bridge to that path, something that might be appreciated by even professional photographers who could spend less time editing to achieve the look that many of their clients seek. It might just make full-size cameras a little more exciting again, though it might also blur the lines of “what’s a picture?”

Leave a Comment