Sophisticated sensor meets sophisticated processing: DxO PhotoLab ⁵ now supports Fujifilm X-Trans

DxO’s latest software brings exciting news for Fujifilm photographers: DxO PhotoLab ⁵ now processes files from X-Trans sensors (beta) and produces remarkable levels of detail.

What is it about X-Trans cameras that make them different to other cameras on the market, and how is machine learning revolutionizing the way that raw files are processed? Head Scientist Wolf Hauser discusses the pros and cons of X-Trans and how DxO’s approach to processing them leads to significant advances in image quality.

Never a company afraid to try something different, Fujifilm introduced the X-Trans sensor in 2012. Given that the rest of the camera industry almost exclusively uses Bayer sensors, this was a bold move and the last nine years have seen many heated debates about whether X-Trans brings genuine benefits to photographers or is little more than an elaborate marketing trick. As will be explored below, there are certainly advantages and disadvantages to X-Trans and the algorithms used to interpret the raw data from this sensor are critical for getting good results. Fujifilm enthusiasts have long searched for the best software to process their images and the latest iteration of DxO PhotoLab include beta support for X-Trans raw files, offering clean images from Fujifilm cameras with fantastic detail rendition.

Before we can understand what makes X-Trans different from Bayer, it’s useful to remind ourselves how sensors capture light, how moiré comes about, and how the raw data from a sensor is turned into the images that we see on our screens.

How to Make a Camera Sensor See in Color

The pixels on a camera’s sensor, whether it’s the smartphone in your pocket or a medium format body, only capture the intensity of light. The solid-state photosites count photons, but they have no means of understanding the wavelength — and thus the color — of the light that they are receiving. To solve this problem, manufacturers creating the earliest digital cameras invented the Color Filter Array (CFA). This mosaic of red, green, and blue sits in front of the sensor and allows the camera to observe different colors through different pixels.

In order to create an image, the next step is to interpolate this data through a process called demosaicing which uses sophisticated algorithms to calculate the missing red, green, and blue values for each individual pixel based on the surrounding pixels.

This design was inspired by nature: the human eye also has red, green, and blue receptors, although a critical difference is that these receptors are spread completely randomly across the retina. Our brains process this stream of continuously shifting data at incredible speed, expertly filling in any blanks using experience and assumptions — none of which is easily replicated in a camera. Instead, the typical camera sensor uses a uniform grid named after its inventor, Bryce Bayer, who came up with a beautifully simple design back in 1974.

The foundation of the Bayer pattern is a block of four pixels with one red and one blue pixel sitting diagonally opposite one another, with the two remaining pixels both green. In effect, the sensor has been split into three: one-half of the pixels sees only green, one-quarter sees only red, and the remaining quarter sees only blue. As a result, the camera has to guess twice as many red values and twice as many blue values as it does green.

Moiré Explained

Capturing light through the means of a uniform grid of pixels can produce some strange visual effects. Moiré is an interference phenomenon that can occur when two grids interact, with patterns often appearing as waves or ripples. In the real world, we tend to see them most often when one dense, wire mesh fence sits behind another.

Cameras are particularly prone to creating these patterns for the simple reason that moiré is the result of two regular grids interacting, and one already exists in the form of the neatly arranged rows of pixels that make up the camera’s sensor. When the scene contains a regular pattern that is as finely detailed as the pixel grid, moiré may appear.

The diagram below simplifies the phenomenon by showing a single row of pixels. The sensor carves up the incoming texture, averaging its intensity within each pixel. Signal processing engineers call this process "sampling": converting a continuous signal into (spatially) discrete values. In the first instance, the sensor can accurately understand the scene, despite the simplification that occurs. Difficulties arise if the details of the pattern become finer than the pixel grid. As can be seen in the second instance, the high frequency of the signal does not match with the lower frequency of the pixels, and the pattern breaks down. Instead of the original high frequency, we observe a lower frequency that never was part of the scene, albeit with strongly reduced amplitude.

Things get even worse if there are gaps between pixels. Suppose that we take out all pixels at odd positions and observe the signal only through pixels at even positions. As before, the incoming signal intensity is averaged over even pixels — but whatever hits the odd pixels is lost completely.

On the two previously shown textures, this will have no decisive impact. However, there is a range of frequencies that could be captured well without gaps and which transmute into moiré patterns when we introduce gaps. Again, we observe a lower frequency that was not in the scene, and this time its amplitude is almost unchanged compared to the original signal.

You might wonder "why would we introduce gaps?" Well, that's exactly what a color filter array does. Our so-called RGB sensors are actually a red sensor with many gaps, a green sensor with fewer gaps, and a blue sensor, also with many gaps. Having more gaps means a greater chance of creating moiré patterns. More gaps in red and blue are the reason why you frequently observe color moiré, i.e. false hue patterns.

How to Avoid Moiré

As we have seen above, moiré arises when there is a mismatch between the resolving power of the lens and the resolution of the sensor, with the lens trying to force too much fine detail into too few megapixels. With sensor resolutions steadily increasing, and the pixel grid becoming finer, this is becoming less of a problem over time.

But most cameras don’t yet have high enough resolution to mitigate this problem, so manufacturers have had to find other means of dealing with moiré. One radical solution would be to remove the color filter array completely, creating a monochrome camera that is capable of capturing extra detail by virtue of the fact that the light reaching the sensor is no longer being filtered through three different colors. Obviously, for those of us who want color photographs, this is not ideal. Manufacturers typically use a slightly less brutal solution: blurring the image so that dense, fine patterns are eliminated before the light hits the sensor — effectively reducing the resolution of the image. This is achieved by placing an anti-aliasing (AA) filter (also called an optical low pass filter) in front of the Bayer filter, sacrificing a little detail in order to avoid moiré. Of course, placing an AA filter behind an ultra-sharp lens that cost a lot of money is not ideal, either. Some manufacturers now produce alternative versions of their high-end, high-resolution cameras that do not feature the AA filter for photographers who wish to capture as much detail as possible, even if that means risking moiré.

2012 saw Fujifilm make a bold design move. For several of its new cameras, it chose to get rid of this AA filter, claiming to have found an ingenious solution to the moiré problem: X-Trans.

As opposed to the two-by-two repeating pattern of the Bayer filter, the X-Trans filter is far more complex: a repeating pattern of six-by-six. This is still a long way from the random arrangement of the receptors in the human eye, but the increased periodicity means that the patterns which trigger moiré in Bayer filter cameras are less of a problem for X-Trans.

The more random-like pixel arrangement becomes more obvious when we look at, for instance, only the red pixels:

If we hide the green and blue pixels, the gaps between the red pixels become more apparent. Bayer on the left, X-Trans on the right.

Note that with X-Trans, any given row or column is capable of “seeing” all three colors. By contrast, an individual row or column on a Bayer filter is always missing either a red or blue pixel:

Every row and every column on the X-Trans sensor (left) contains all three colors — even on the rows that have the fewest red and blue pixels. By contrast, on the Bayer sensor, every row and every column is always missing either red or blue pixels.

For regular patterns consisting of either only horizontal or only vertical high frequencies, this gives X-Trans a real advantage. In this instance, it actually avoids the gaps in the red and blue channels. We have seen above how these gaps can cause moiré, and X-Trans is definitely less prone to color moiré than Bayer as a result.

However, there is no such thing as a free lunch and the advantages of X-Trans bring with them certain disadvantages. Across its six-by-six base, only 8 pixels are red and only 8 are blue, while the same sized area of a Bayer sensor would have 9 of each. As a result, the X-Trans sensor is about 11% less sensitive to both red and blue. More of a factor, however, is that pixels of the same color can be further apart on the X-Trans filter compared to Bayer. On a Bayer filter, a red pixel is never more than one pixel’s width away from another red pixel. By contrast, on an X-Trans filter, it can be twice that.

On irregular textures and details as they typically occur in nature (foliage, flowers, faces, etc), the regular grid of the Bayer pattern does no harm, and X-Trans brings no benefits. By contrast, an X-Trans sensor’s lower density of red and blue pixels means that it will observe slightly less color detail. If a small color spot happens to fall in that fairly huge block of four green pixels, it is simply not observed at all.

Bayer Versus X-Trans, Pepsi Versus Coke?

As readers may have noticed, discussing the finer details of camera technology can trigger some strong reactions, and debates over the advantages and disadvantages of X-Trans have played out on message boards and social media networks since it was launched. However, the color filter array is only one feature out of many that define a camera and few customers would have it foremost in mind when buying a camera. Fujifilm photographers appreciate the distinctive look and feel of their cameras, enjoying the ergonomics and ease of use, not to mention the results that can be achieved straight out of camera. Fujifilm’s engineers are experts when it comes to color following more than 70 years of experience in creating color photographs and devotees love Fujifilm’s film simulations which draw on the company’s rich history of producing film stock. Names such as Astia and Velvia give their cameras a sense of authenticity.

The Bayer versus X-Trans argument has parallels with the ongoing battle between ARM processors and those designed by Intel. Apple's marketing department will claim their iPad is better because it features their new ARM chip, while Microsoft wants you to believe that Surface is better because it uses the latest Intel chip. This allows aficionados of both brands to spend their nights in heated forum discussions about ARM vs Intel, RISC vs CISC — but 99% of all users don't actually care. They chose their tablet because they prefer the look and feel of one over the other, the user experience, and ultimately, the branding.

Beyond some technical differences deep inside the system, for most users, the Fujifilm versus Canon/Nikon/Sony/etc decision might be not so different from choosing between Pepsi and Coke.

The Consequences of Complexity

tech-news_fuji-xtrans.fp29

Whatever the criteria for choosing a camera, squeezing the best possible performance out of its sensor is still important, and it’s useful to see what happens when you zoom into images at 100% and more.

It’s worth remembering that the final image is not solely the result of the sensor itself; whether it’s in-camera or through software, a variety of processes take place, notably demosaicing, the algorithm which fills in the gaps between the red, green, and blue channels. It’s a combination of these algorithms with the color filter array that determines the result.

One of the reasons that the Bayer filter has proven so tenacious is that engineers are used to processing its data. Finding the best recipe for demosaicing Bayer sensors has kept researchers busy for four decades, and the highly sophisticated algorithms developed over the years have allowed them to mitigate many of the limitations fundamental to its design. Even fairly simple algorithms, such as those embedded in the first digital cameras, yield fairly good results.

The increased complexity of the X-Trans pattern, on the other hand, entails a demosaicing process that is far more complex. Fujifilm’s engineers are said to have spent five years waiting for their cameras’ processing power to catch up before introducing X-Trans in the X-Pro1 in 2012. At the same time, the research community has published far fewer papers about X-Trans demosaicing than they have for Bayer; not only is it a more complex problem but less research effort has been spent solving it. It seems fair to assume that current X-Trans demosaicing algorithms are still some distance away from achieving a theoretically optimal solution. This is one reason that Fujifilm enthusiasts often find themselves jumping between software packages in search of a solution that delivers the best results.

Overcoming Complexity Through Machine Learning

At DxO, we have attempted to create better X-Trans processing in the past, but we were never quite satisfied with the outcome. The main challenge was that our processing traditionally took a different route to every other software: while most raw processors demosaic a raw file before denoising, DxO always did it the other way around — one of the reasons that our software often produces cleaner results. As a consequence, adapting our raw processes to cater for X-Trans sensors would not only have required a new demosaicing process, but also a new denoising process. Applying raw denoising to X-Trans was again incredibly complicated compared to Bayer and rebuilding the process never yielded truly satisfactory results.

Today, image processing is being revolutionized by machine learning, particularly by a technology called convolutional neural networks. Within a few years, this new class of algorithms — no longer hand-crafted by researchers and engineers, but learned empirically by a computer from looking at millions of training examples — has made decades of research effort obsolete. With Bayer demosaicing, for instance, neural networks now easily beat the very best algorithms designed by humans.

While certainly frustrating for researchers who spent their life on demosaicing algorithms, this revolution is actually a huge opportunity. Not only are the results better, it also boosts productivity: computers find a state-of-the-art demosaicing algorithm in days or weeks rather than in years or decades. Machine learning is particularly well adapted to problems that have clearly defined inputs and expected outputs but where the mapping between them is too complex to be formulated as a classical algorithm. Image and speech recognition were the first examples but machine learning turns out to be such a powerful tool that it has proven useful in domains where decent classical algorithms already existed — such as demosaicing.

X-Trans demosaicing is a great candidate for machine learning. Being more complex than Bayer demosaicing, the advantage of machine learning over traditional engineering should be even greater than that achieved with Bayer demosaicing. Our counterparts at Adobe demonstrated exactly this when they introduced their machine learning-powered “Enhance Details” feature in early 2020. Reviewers concluded that, while the difference for Bayer images was rather subtle, it was a significant improvement for X-Trans images.

At DxO, we have leveraged machine learning in DxO PhotoLab ⁴ to solve another highly complex task: our new raw conversion technology — DxO DeepPRIME — uses a single, huge convolutional neural network to apply demosaicing and denoising at the same time. After 10 days of intensive work, our computer developed a highly sophisticated algorithm that outperforms our traditional demosaicing at low ISO, and both our traditional demosaicing and denoising at high ISO.

DxO PhotoLab ⁵ Features DeepPRIME for
X-Trans

Obviously, these 10 days of training our neural network were preceded by years of research. We had to define the “shape” of the network manually while the computer only determined its millions of parameters. But the most challenging part, on which we spent 80% of our effort, was ensuring that the problem was being described to the deep convolutional neural network as accurately as possible through the use of very precise training data.

Once work on Bayer sensor images was complete, it became clear that making changes to accommodate X-Trans raw files was no longer such a daunting task because the procedure for generating training data could be reused with very few modifications. It was still challenging as we had to fundamentally change the network shape to accommodate for the complex X-Trans pattern, but it was conceivable and proved to be an exciting challenge. The results are exciting, too. Let’s have a look at two examples.

This landscape shot was taken with a Fujifilm X-T2 at ISO 200. The image looks reasonably good in terms of color and exposure, even without post-processing (top). However, when zooming in and examining the details — details that become important when making large prints — we notice that DxO PhotoLab (bottom right) does a much better job at preserving color details than the camera itself (bottom left). The camera fails to distinguish the varying hues of human skin, the wooden fences, and the grass; all end up being more or less uniformly greenish. By contrast, DxO PhotoLab manages to distinguish these features, producing a more natural image. It also preserves the texture of the grass in the foreground more effectively. As a result, the image appears to be at a higher resolution and will reproduce better when printed in a large format.

This low light indoor action shot was taken with a Fujifilm X-T3 at ISO 6400. The original photo was underexposed so we pushed it by two stops during post-processing — the equivalent of ISO 25600 (top). Such heavy exposure adjustment cannot be performed on JPEG images, so the comparison here is not with the camera but against a well known raw converter: Adobe Lightroom with Enhance Details (bottom left). When we look at the faces more closely, we can see that DxO PhotoLab (bottom right) yields a significantly cleaner result. Because it uses a neural network to run demosaicing and denoising at the same time, DeepPRIME does a better job at removing noise while at the same time preserving more detail in terms of both luminance and color.

Obviously, it takes more than DxO DeepPRIME on its own to fully support X-Trans in software as sophisticated as DxO PhotoLab. Many internal tools used by our lab to calibrate the color and noise model of each camera body had to be adapted. Several other processing blocks had to be designed from scratch, such as the demosaicing algorithm used to display a preview while the user makes adjustments.

Because so many things were done in parallel, and because X-Trans is still more or less unknown territory for us, we feel that our technology has not quite reached a stage where it performs consistently to our high standards. At the same time, we really want to share our new technology with Fujifilm fans. We believe that photographers will appreciate how our DxO DeepPRIME technology can draw out color detail that was previously missing, bring new life to old photographs, and transform high ISO images. You can help us to complete our refinements by reporting bugs and providing feedback via Send a request.

What are Linear DNG files and how should you use them in your photo-editing workflow?

Photo editing software such as Lightroom and Capture One might give you creative freedom and flexibility, but that doesn’t necessarily mean that you’re getting the best possible image quality from your RAW files. Linear DNG files are part of a solution that allows you to combine different software to get ultimate image quality without having to overhaul your entire workflow.

Linear DNG files are RAW files that have been partially developed, having undergone some complex mathematical processing to lock in demosaicing. Depending on its intended purpose, a Linear DNG might also complete other parts of the RAW conversion process such as denoising and lens corrections. You can then take this optimized file into your photo-editing software of choice to complete the creative process — everything from white balance and exposure through to tone curves and color grading — with the knowledge that your final image will be the highest possible quality. As you edit your files, you might even find that you have greater flexibility than with the original RAW file.

By taking a RAW file and converting it to a Linear DNG file with one application and then editing it in another, you get to combine the strengths — one mathematical, one creative — of two different pieces of software.

This image was edited as a RAW file using Adobe Lightroom. In the centre, you can see the results of Noise Reduction in Lightroom, and on the right is the optimization as a result of using DxO PureRAW  ³.

The basics: RAW conversion, linearity, and DNG files

Every image sensor in every camera produces RAW files that need to be converted into a format that is capable of being displayed on-screen or printed on paper. It’s worth remembering that a camera does not “see” an image; instead, it captures information that must be transformed using sophisticated digital image processing. Software — either in the camera or on your computer — takes data that has been measured by each pixel and converts it into a color image. Unprocessed information is called “RAW data” and the process of transforming it into something visible is called “RAW conversion.” If you shoot in RAW, you’re saving the unprocessed data from the sensor and saving it to your memory card, ready to be processed by software such as DxO PhotoLab or Adobe Lightroom.

In some ways, a RAW image can be compared to a frame of negative film in analog photography. It contains all of the information that was captured when taking the photograph but more processing is needed before an image can be revealed.

Why should you shoot RAW?

Why would you capture data that still needs to be processed rather than JPEG files that are ready to be viewed? There are two reasons.

1. The first is image quality. As mentioned above, RAW conversion is complex. Doing it outside of the camera allows you to use more sophisticated algorithms, yielding better images.

2. The second reason is that it gives you control over certain aspects of the RAW conversion, such as exposure, color, and contrast.

Again, comparing it to analog photography is a useful illustration: JPEG files are a bit like Polaroid instant film. You get immediate results and that’s a huge advantage. However, the image quality cannot match that of negative film. Furthermore, with negatives, you can adjust exposure and color while developing your prints in the darkroom.

Of course, you can change the exposure and color of JPEGS, but RAW files contain a lot more information. There’s more dynamic range, a greater array of colors, and more levels of gray that allow you to preserve smooth tone transitions

RAW data is linear

RAW data is linear — but what does this mean? In mathematics, a linear system is one where multiplying an input by a certain amount means that the output will be multiplied by the same amount. Going back to your algebra at school, consider this example: In a linear system where 2x = 5y, then 4x = 10y, and 6x = 15y. Double one factor and you double the other, triple one factor and you triple the other, etc. Mathematicians would describe a linear system as one where scaling an input by a factor means that the output is always scaled the same factor: in other words, f(αx) = α f(x) for all α.

A photographic lens is a linear system. If you double the intensity of your studio lighting or double the exposure time while keeping the aperture unchanged, the amount of light reaching the image sensor in the camera will also double.

Just like the lens, the camera’s sensor is also linear. As a result, the data in the RAW file is a linear function of the amount of light in the scene. Notably, the first few steps of RAW conversion are linear. However, some non-linear steps soon take place — most notably, color rendering and gamma correction (i.e., the process of making data fit more closely to human eye’s understanding of how brightness works). To be able to retain quality, you want your exposure adjustments to be applied before these non-linear functions occur. This is where a JPEG file falls short as it doesn’t allow you to do that.

By contrast, when editing a RAW file, all of these changes are made before color rendering and gamma correction take place. Exposure, white balance, highlight recovery, boosting shadows — all of this can be achieved before irreversible steps are imposed and the editing process remains non-destructive.

An example can be seen in this photograph of a color checker card, captured with a white balance that had not been set correctly:

If you take the RAW file from the camera, you can use the eyedropper in your photo-editing software on a part of the image that is neutral gray to allow it to calculate the correct white balance:

The result is accurate (taking into account the limitations of the camera). Adjusting white balance on your camera before the shot or as part of the editing process achieves the same result because, just like the camera, the software applies this change in the linear domain. However, if you take the JPEG produced by the camera and attempt to correct the image in the same way, the results are noticeably different:

In reality, all of the grays in this scene are completely neutral. However, in this JPEG, each of the grays has a slight color cast. In addition, the individual color tiles are both quite far from their true hues and have lost some of their intensity. Recovering an accurate reproduction of the colors is impossible because the linearity of this information was given up when the RAW data from the sensor was converted into a JPEG.

What is DNG?

DNG stands for Digital Negative and is a RAW image file format designed by Adobe. Adobe’s goal was to create an open standard for storing RAW data as an alternative to the proprietary formats used by camera manufacturers — ARW from Sony, NEF from Nikon, CR3 from Canon, etc. All of these files can be converted to DNG simply by changing how the pixel values are stored. The values themselves remain unchanged.

The DNG format also provides a standardized means of storing metadata. Metadata — called XMP (Extensible Metadata Platform) — might include information such as shooting parameters, copyright, and star ratings, but they also contain settings for Adobe’s RAW conversion algorithms.

So what is Linear DNG?

Typically, DNG files store unprocessed RAW data. By contrast, Linear DNG files store some intermediate results of RAW conversion. Some initial processing steps have been applied — but only linear ones. This means that none of the data is lost to one-way, non-linear processes. As a result, Linear DNGs have exactly the same flexibility as RAW files when it comes to editing aspects such as brightness, color, and contrast.

Linear DNG files become useful when you want more flexibility in your workflow. If your preferred photo-editing software does a poor job of performing those initial processing steps, you might want to have another piece of software do it for you instead as a means of ensuring better image quality. With Linear DNG files, you can combine the strengths of different tools, squeezing more from your RAW files in terms of quality before switching to different software that gives your preferred mode of creativity.

Objective and subjective steps when processing RAW files

RAW conversion consists of many steps, both linear and non-linear. At DxO, we regard each step as either restoration or rendering. Here’s how they differ.

Restoration is where you are trying to achieve a perfect image from imperfect RAW data. There is an objective truth — the scene in front of your camera — that you are trying to recreate in a photograph as accurately as possible. Two fundamentals of this process are demosaicing — the conversion of values from individual pixels for red, green, and blue into accurate colors — and denoising — the removal of inaccurate pixels created as a result of interference and heat inside your camera, both of which become amplified when ISO is increased. Other important steps are the removal of vignetting, geometric distortions, and chromatic aberrations, and the correction of a lack of sharpness in the corners.

Restoration is considered an objective task in that there is a ground truth that you are trying to achieve as closely as possible. For example, with demosaicing, you want to create accurate colors as though they were seen with the human eye rather than the millions of individual values for red, green, and blue, as captured by the pixels on the camera’s sensor. In denoising, you want to restore the pixel values as though the image had been captured under better lighting conditions and using a lower ISO. With both of these tasks, computers are far better than humans.

Rendering, on the other hand, is fundamentally subjective. Even exposure and white balance are subjective. Trade-offs have to be made between highlights and shadows, and for many, the goal is not to faithfully reproduce the scene as it appeared in front of their camera. Instead, you might want to create a certain atmosphere or emotion. Even though computers might become better at guessing what sort of rendering humans might appreciate, many photographers prefer to have control over this part of the process.

90% of the sliders in your RAW conversion software — whether it’s DxO PhotoLab, Adobe Lightroom, or Capture One — are dedicated to rendering. Only 10% of the sliders relate to restoration, even though the software spends 90% of its computing time on restoration tasks. Furthermore, this is in spite of the fact that the restoration can determine how much flexibility a photographer has when choosing how to edit a photograph, and can have an effect on other aspects such as the potential to crop without compromising too much detail, or how large a print can be achieved.

Splitting your RAW conversion into rendering and restoration

A photographer’s choice of RAW processing software is highly subjective, and most are guided by the software they find the most intuitive in terms of rendering their images — even though the choice for restoration should be a completely objective decision. Unfortunately, the best tool for one is not necessarily the best tool for the other.

For restoration, those evaluating cameras and RAW conversion software — whether they are photo industry journalists or keen amateurs — have been running analyses using test scenes and evaluation targets for decades. These assessments have long confirmed that DxO’s optical corrections are the most accurate and that our DeepPRIME technology is better at removing noise while preserving more detail than any other RAW converter on the market. In terms of restoration, DxO DeepPRIME should be the obvious choice for any photographer seeking to maximize the quality of their images.

However, when it comes to rendering, we appreciate that the choice is more personal. Not only are there preferences for certain colors or levels of contrast, there is also habit. Your photographic style might be tied up in the tools that are most familiar to you, and we understand that. Changing your RAW converter would require learning a completely new system and potentially losing what makes your images unique in terms of look and feel. Switching away from Adobe Lightroom or Capture One and over to DxO PhotoLab might seem too much of a hurdle, just as switching between Canon and Nikon would also mean a big commitment.

With this in mind, we decided that photographers should be able to get the best of both worlds. What if you could take advantage of DxO’s renowned restoration algorithms while keeping your existing workflow? You could get superior image quality and files with greater flexibility — all without having to switch to new software. This is where Linear DNG files and DxO PureRAW come into play.

A unique sensor now matched by next-generation processing: DxO DeepPRIME and DeepPRIME XD now support Fujifilm X-Trans

DxO’s latest software brings exciting news for Fujifilm photographers: both DxO PhotoLab ⁶ and DxO PureRAW  ³ now process files from X-Trans sensors, producing remarkable levels of detail.

What is it about X-Trans cameras that make them different to other cameras on the market, and how is machine learning revolutionizing the way that raw files are processed? Head Scientist Wolf Hauser discusses the pros and cons of X-Trans and how DxO’s approach to processing them leads to significant advances in image quality.

Never a company afraid to try something different, Fujifilm introduced the X-Trans sensor in 2012. Given that the rest of the camera industry almost exclusively uses Bayer sensors, this was a bold move and the last ten years have seen many heated debates about whether X-Trans brings genuine benefits to photographers or is little more than an elaborate marketing trick. As will be explored below, there are certainly advantages and disadvantages to X-Trans and the algorithms used to interpret the raw data from this sensor are critical for getting good results. Fujifilm enthusiasts have long searched for the best software to process their images and DxO PhotoLab ⁶ and DxO PureRAW  ³ now support for X-Trans raw files – in both DeepPRIME and DeepPRIME XD modes – offering clean images from Fujifilm cameras with fantastic detail rendition.

Before we can understand what makes X-Trans different from Bayer, it’s useful to remind ourselves how sensors capture light, how moiré comes about, and how the raw data from a sensor is turned into the images that we see on our screens.

How to Make a Camera Sensor See in Color

The pixels on a camera’s sensor, whether it’s the smartphone in your pocket or a medium format body, only capture the intensity of light. The solid-state photosites count photons, but they have no means of understanding the wavelength — and thus the color — of the light that they are receiving. To solve this problem, manufacturers creating the earliest digital cameras invented the Color Filter Array (CFA). This mosaic of red, green, and blue sits in front of the sensor and allows the camera to observe different colors through different pixels.

In order to create an image, the next step is to interpolate this data through a process called demosaicing. This uses sophisticated algorithms to calculate the missing red, green, and blue values for each individual pixel based on the surrounding pixels.

This design was inspired by nature: the human eye also has red, green, and blue receptors, although a critical difference is that these receptors are spread completely randomly across the retina. Our brains process this stream of continuously shifting data at incredible speed, expertly filling in any blanks using experience and assumptions — none of which is easily replicated in a camera. Instead, the typical camera sensor uses a uniform grid named after its inventor, Bryce Bayer, who came up with a beautifully simple design back in 1974.

The foundation of the Bayer pattern is a block of four pixels with one red and one blue pixel sitting diagonally opposite one another, with the two remaining pixels both green. In effect, the sensor has been split into three: one-half of the pixels sees only green, one-quarter sees only red, and the remaining quarter sees only blue. As a result, the camera has to guess twice as many red values and twice as many blue values as it does green.

Moiré Explained

Capturing light through the means of a uniform grid of pixels can produce some strange visual effects. Moiré is an interference phenomenon that can occur when two grids interact, with patterns often appearing as waves or ripples. In the real world, we tend to see them most often when one dense, wire mesh fence sits behind another.

Cameras are particularly prone to creating these patterns for the simple reason that moiré is the result of two regular grids interacting, and one already exists in the form of the neatly arranged rows of pixels that make up the camera’s sensor. When the scene contains a regular pattern that is as finely detailed as the pixel grid, moiré may appear.

The diagram below simplifies the phenomenon by showing a single row of pixels. The sensor carves up the incoming texture, averaging its intensity within each pixel. Signal processing engineers call this process “sampling”: converting a continuous signal into (spatially) discrete values. In the first instance, the sensor can accurately understand the scene, despite the simplification that occurs. Difficulties arise if the details of the pattern become finer than the pixel grid. As can be seen in the second instance, the high frequency of the signal does not match with the lower frequency of the pixels, and the pattern breaks down. Instead of the original high frequency, we observe a lower frequency that never was part of the scene, albeit with strongly reduced amplitude.

Things get even worse if there are gaps between pixels. Suppose that we take out all pixels at odd positions and observe the signal only through pixels at even positions. As before, the incoming signal intensity is averaged over even pixels — but whatever hits the odd pixels is lost completely.

On the two previously shown textures, this will have no decisive impact. However, there is a range of frequencies that could be captured well without gaps but which transmute into moiré patterns when we introduce gaps. Again, we observe a lower frequency that was not in the scene, and this time its amplitude is almost unchanged compared to the original signal.

You might wonder “why would we introduce gaps?” This is exactly what a color filter array does. Our so-called RGB sensors are actually a red sensor with many gaps, a green sensor with fewer gaps, and a blue sensor, also with many gaps. Having more gaps means a greater chance of creating moiré patterns. More gaps in red and blue are the reason why you frequently observe color moiré, i.e. false hue patterns.

How to Avoid Moiré

As we have seen above, moiré arises when there is a mismatch between the resolving power of the lens and the resolution of the sensor, with the lens trying to force too much fine detail into too few megapixels. With sensor resolutions steadily increasing, and the pixel grid becoming finer, this is becoming less of a problem over time.

But most cameras don’t yet have high enough resolution to mitigate this issue, so manufacturers have had to find other means of dealing with moiré. One radical solution would be to remove the color filter array completely, creating a monochrome camera that is capable of capturing extra detail by virtue of the fact that the light reaching the sensor is no longer being filtered through three different colors. Obviously, for those of us who want color photographs, this is not ideal. Manufacturers typically use a slightly less brutal solution: blurring the image so that dense, fine patterns are eliminated before the light hits the sensor — effectively reducing the resolution of the image. This is achieved by placing an anti-aliasing (AA) filter (also called an optical low pass filter) in front of the Bayer filter, sacrificing a little detail in order to avoid moiré. Of course, placing an AA filter behind an ultra-sharp lens that cost a lot of money is not ideal, either. Some manufacturers now produce alternative versions of their high-end, high-resolution cameras that do not feature the AA filter for photographers who wish to capture as much detail as possible, even if that means risking moiré.

2012 saw Fujifilm make a bold design move. For several of its new cameras, it chose to get rid of this AA filter, claiming to have found an ingenious solution to the moiré problem: X-Trans.

As opposed to the two-by-two repeating pattern of the Bayer filter, the X-Trans filter is far more complex: a repeating pattern of six-by-six. This is still a long way from the random arrangement of the receptors in the human eye, but the increased periodicity means that the patterns which trigger moiré in Bayer filter cameras are less of a problem for X-Trans.

The more random-like pixel arrangement becomes more obvious when we look at, for instance, only the red pixels. If we hide the green and blue pixels, the gaps between the red pixels become more apparent. Bayer on the left, X-Trans on the right.

Note that with X-Trans, any given row or column is capable of “seeing” all three colors. By contrast, an individual row or column on a Bayer filter is always missing either a red or blue pixel:

Every row and every column on the X-Trans sensor (left) contains all three colors — even on the rows that have the fewest red and blue pixels. By contrast, on the Bayer sensor, every row and every column is always missing either red or blue pixels.

For regular patterns consisting of either only horizontal or only vertical high frequencies, this gives X-Trans a real advantage. In this instance, it actually avoids the gaps in the red and blue channels. We have seen above how these gaps can cause moiré, and X-Trans is definitely less prone to color moiré than Bayer as a result.

However, there is no such thing as a free lunch and the advantages of X-Trans bring with them certain disadvantages. Across its six-by-six base, only 8 pixels are red and only 8 are blue, while the same sized area of a Bayer sensor would have 9 of each. As a result, the X-Trans sensor is about 11% less sensitive to both red and blue. More of a factor, however, is that pixels of the same color can be further apart on the X-Trans filter compared to Bayer. On a Bayer filter, a red pixel is never more than one pixel’s width away from another red pixel. By contrast, on an X-Trans filter, it can be twice that.

On irregular textures and details as they typically occur in nature (foliage, flowers, faces, etc), the regular grid of the Bayer pattern does no harm, and X-Trans brings no benefits. By contrast, an X-Trans sensor’s lower density of red and blue pixels means that it will observe slightly less color detail. If a small color spot happens to fall in that fairly huge block of four green pixels, it is simply not observed at all.

Bayer Versus X-Trans, Pepsi Versus Coke?

As you may have noticed, discussing the finer details of camera technology can trigger some strong reactions, and debates over the advantages and disadvantages of X-Trans have played out on message boards and social media networks since it was launched. However, the color filter array is only one feature out of many that define a camera and few customers would have it foremost in mind when buying a camera. Fujifilm photographers appreciate the distinctive look and feel of their cameras, enjoying the ergonomics and ease of use, not to mention the results that can be achieved straight out of camera. Fujifilm’s engineers are experts when it comes to color following more than 70 years of experience in creating color photographs, and devotees love Fujifilm’s film simulations which draw on the company’s rich history of producing film stock. Names such as Astia and Velvia give their cameras a sense of authenticity.

The Bayer versus X-Trans argument has parallels with the ongoing battle between ARM processors and those designed by Intel. Apple’s marketing department will claim their iPad is better because it features their new ARM chip, while Microsoft wants you to believe that Surface is better because it uses the latest Intel chip. This allows aficionados of both brands to spend their nights in heated forum discussions about ARM vs Intel, RISC vs CISC — but 99% of all users don’t actually care. They chose their tablet because they prefer the look and feel of one over the other, the user experience, and ultimately, the branding.

Beyond some technical differences deep inside the system, for most users, the Fujifilm versus Canon/Nikon/Sony/etc decision might be not so different from choosing between Pepsi and Coke.

The Consequences of Complexity

Whatever the criteria for choosing a camera, squeezing the best possible performance out of its sensor is still important, and it’s useful to see what happens when you zoom into images at 100% and more.

It’s worth remembering that the final image is not solely the result of the sensor itself; whether it’s in-camera or through software, a variety of processes take place, notably demosaicing, the algorithm which fills in the gaps between the red, green, and blue channels. It’s a combination of these algorithms with the color filter array that determines the result.

One of the reasons that the Bayer filter has proven so tenacious is that engineers are used to processing its data. Finding the best recipe for demosaicing Bayer sensors has kept researchers busy for four decades, and the highly sophisticated algorithms developed over the years have allowed them to mitigate many of the limitations fundamental to its design. Even fairly simple algorithms, such as those embedded in the first digital cameras, yield fairly good results.

The increased complexity of the X-Trans pattern, on the other hand, entails a demosaicing process that is far more elaborate. Fujifilm’s engineers are said to have spent five years waiting for their cameras’ processing power to catch up before introducing X-Trans in the X-Pro1 in 2012. At the same time, the research community has published far fewer papers about X-Trans demosaicing than they have for Bayer; not only is it a more complex problem but less research effort has been spent solving it. It seems fair to assume that current X-Trans demosaicing algorithms are still some distance away from achieving a theoretically optimal solution. This is one reason that Fujifilm enthusiasts often find themselves jumping between software packages in search of a solution that delivers the best results.

Overcoming Complexity Through Machine Learning

At DxO, we have attempted to create better X-Trans processing in the past, but we were never quite satisfied with the outcome. The main challenge was that our processing traditionally took a different route to every other software: while most raw processors demosaic a RAW file before denoising, DxO always did it the other way around — one of the reasons that our software often produces cleaner results. As a consequence, adapting our RAW processes to cater for X-Trans sensors would not only have required a new demosaicing process, but also a new denoising process. Applying RAW denoising to X-Trans was again incredibly complicated compared to Bayer and rebuilding the process never yielded truly satisfactory results.

Today, image processing is being revolutionized by machine learning, particularly by a technology called convolutional neural networks. Within a few years, this new class of algorithms — no longer hand-crafted by researchers and engineers, but learned empirically by a computer from looking at millions of training examples — has made decades of research effort obsolete. With Bayer demosaicing, for instance, neural networks now easily beat the very best algorithms designed by humans.

While certainly frustrating for researchers who spent their life on demosaicing algorithms, this revolution is actually a huge opportunity. Not only are the results better, it also boosts productivity: computers find a state-of-the-art demosaicing algorithm in days or weeks rather than in years or decades. Machine learning is particularly well adapted to problems that have clearly defined inputs and expected outputs but where the mapping between them is too complex to be formulated as a classical algorithm. Image and speech recognition were the first examples but machine learning turns out to be such a powerful tool that it has proven useful in domains where decent classical algorithms already existed — such as demosaicing.

X-Trans demosaicing is a great candidate for machine learning. Being more complex than Bayer demosaicing, the advantage of machine learning over traditional engineering should be even greater than that achieved with Bayer demosaicing. Our counterparts at Adobe demonstrated exactly this when they introduced their machine learning-powered “Enhance Details” feature in early 2020. Reviewers concluded that, while the difference for Bayer images was rather subtle, it was a significant improvement for X-Trans images.

At DxO, we leveraged machine learning in DxO PhotoLab to solve another highly complex task: our RAW conversion technology — DxO DeepPRIME and DeepPRIME XD — uses a single, huge convolutional neural network to apply demosaicing and denoising at the same time. After 10 days of intensive work, our computer developed a highly sophisticated algorithm that outperforms our traditional demosaicing at low ISO, and both our traditional demosaicing and denoising at high ISO.

DxO PhotoLab ⁶ and DxO PureRAW  ³ Feature DxO DeepPRIME and DeepPRIME XD for X-Trans

Of course, those 10 days of training our neural network were preceded by years of research. We had to define the “shape” of the network manually while the computer only determined its millions of parameters. But the most challenging part, on which we spent 80% of our effort, was ensuring that the problem was being described to the deep convolutional neural network as accurately as possible through the use of very precise training data.

Once work on Bayer sensor images was complete, it became clear that making changes to accommodate X-Trans raw files was no longer such a daunting task because the procedure for generating training data could be reused with very few modifications. There were still obstacles to overcome as we had to fundamentally change the network shape to accommodate for the complex X-Trans pattern, but it was conceivable and proved to be an exciting challenge. The results are exciting, too. Let’s have a look at two examples.

This landscape shot was taken with a Fujifilm X-T2 at ISO 200. The image looks reasonably good in terms of color and exposure, even without post-processing (top). However, when zooming in and examining the details — details that become important when making large prints — we notice that DxO DeepPRIME (using DxO PhotoLab ⁶, bottom right) does a much better job at preserving color details than the camera itself (bottom left). The camera fails to distinguish the varying hues of human skin, the wooden fences, and the grass; all end up being more or less uniformly greenish. By contrast, DxO DeepPRIME manages to distinguish these features, producing a more natural image. It also preserves the texture of the grass in the foreground more effectively. As a result, the image appears to be at a higher resolution and will reproduce better when printed in a large format.

This low light indoor action shot was taken with a Fujifilm X-T3 at ISO 6400. The original photo was underexposed so we pushed it by two stops during post-processing — the equivalent of ISO 25600 (top). Such heavy exposure adjustment cannot be performed on JPEG images, so the comparison here is not with the camera but against a well known raw converter: Adobe Lightroom with Enhance Details (bottom left). When we look at the faces more closely, we can see that DxO DeepPRIME (using DxO PhotoLab, bottom right) yields a significantly cleaner result. Because it uses a neural network to run demosaicing and denoising at the same time, DeepPRIME does a better job at removing noise while at the same time preserving more detail in terms of both luminance and color.

More than just machine learning

Obviously, it takes more than DxO DeepPRIME on its own to fully support X-Trans in software as sophisticated as DxO PhotoLab and DxO PureRAW  ³. Many internal tools used by our lab to calibrate the color and noise model of each camera body had to be adapted. Several other processing blocks had to be designed from scratch, such as the demosaicing algorithm used to display a preview while the user makes adjustments.

Ready for your photos

After an intense period of research and development, both DxO PhotoLab ⁶ and DxO PureRAW  ³ are now ready to bring dramatic improvements to your RAW files. We believe that photographers will appreciate how our DxO DeepPRIME and DeepPRIME XD technology can draw out color detail that was previously missing, bring new life to old photographs, and transform high ISO images. Download a free trial and discover what DxO DeepPRIME can do for your photos.

How the new DxO Wide Gamut helps PhotoLab 6 deliver truer colors and better images (Article only available in English)

For DxO PhotoLab ⁶, we created our most versatile and flexible color space yet. Here’s how it works and why it’s better for your photos.

Color is one of the most important aspects of any picture and the capacity to reproduce and adjust color to match the photographer's intent is fundamental to any image-editing software. Essentially, accurate color is the foundation on which you edit.

To make better adjustments and provide more intense and lifelike results, DxO PhotoLab ⁶ brings three major advances in terms of the way it handles color.

• At its heart is a new working color space providing a much wider gamut for accurate reproduction of saturated colors and more headroom for color adjustments like hue, saturation, and luminance.
• The algorithms that manage colors as they pass through your image-editing workflow have been completely reengineered so that they deliver better results in conjunction with the new working color space.
• Finally, a new Soft Proofing mode ensures that the colors and tones that you see on your monitor match what comes out of your printer, or what is displayed on your other devices.

Get to grips with color science lingo

To understand why we created a new wide gamut color space – and why it’s so effective – it’s important to have a working knowledge of color science. Below are some of the main components. If you’re comfortable with color science and just want to know more about how PhotoLab 6 handles color, just click here to skip to that section. Link to below

Color

Color is a sensation that the human eye perceives as a result of light being reflected by an object or emitted by a source. More precisely, this perception occurs as various parts of the spectrum of light interact with the three different types of cone photoreceptor cells on our retina. This stimulation is converted into an electrical signal which we interpret as color.

If all wavelengths present similar levels of intensity, we perceive achromatic colors such as gray or white, but when certain parts of the spectrum dominate, it is perceived as color. When more energy is concentrated in a small part of the spectrum, a color will appear more saturated and intense.

Surface colors

---

Spectral colors

---

Color space

---

XYZ and CIE color spaces

There are also color spaces that are defined independently of any device, such as the XYZ space designed in 1931 by the International Commission on Illumination (abbreviated to CIE from its French name, Commission Internationale de l'Éclairage). This space attempts to model color perception along three axes: X, Y, and Z.

The illustration below shows the spectral power distribution of the light reflected by two different objects — a piece of paper and a tomato – when the objects are illuminated under daylight. Each graph also indicates the corresponding surface color expressed as XYZ values. To the human eye, the paper is perceived as white, while the tomato is perceived as red.

Fig 1: The light reflected by two objects, shown in graph form.

Gamut

A gamut is the range of colors that can be reproduced by a given monitor or printer, or the colors that can be represented within a certain color space. For printing, gamut depends on both the inks and the paper being used. The larger the gamut, the more colors a device can reproduce.

If you are soft proofing an image using a color profile for a printer, your software might notify you that certain colors in your image simply cannot be reproduced by the printer — they are “out of gamut.” See below for more on soft proofing and why it’s important.

Wide gamut

Wide gamut is a loose term that describes any device or color space that can reproduce more colors than average. Since most of today’s monitors use the relatively small sRGB color space, any color space exceeding the gamut of sRGB tends to be called “wide gamut” by the relevant marketing team. Not all wide gamut devices are equal, however; the term can describe color spaces that are slightly larger than sRGB through to those that are far, far bigger.

Chromaticity

Chromaticity is a measurement that distinguishes different colors of equal luminance. It’s therefore commonly used in charts that illustrate color space. Several color spaces organize color so that one axis represents luminance, and the other two axes represent chromaticity, for instance in the form of hue and colorfulness.

The CIE has derived a chromaticity diagram from their XYZ color space where colors are represented along two axes, x and y. This diagram is popular for comparing color gamuts, mainly because it can be presented more easily than a three-dimensional chart.

Fig 2: Comparison of the AdobeRGB and Display P3 color spaces in the CIE XYZ color space (left) and CIE chromaticity diagram (right). Notice how, although AdobeRGB and Display P3 are of similar size on the chart, neither is a subset of the other – AdobeRGB contains more cyan, while Display P3 contains more red and yellow.

Color profile / ICC profile

A color profile – also called an ICC profile because the format was standardized by the International Color Consortium – characterizes the color space of a specific device, typically a monitor or a printer. They are normally supplied by the vendor or can be created by the user by measuring the device, for instance by using a colorimeter.

Color management

Color management is the process of converting colors from one color space to another while preserving the perception. Typically, when displaying an image on your calibrated screen that has been saved in a standard color space such as AdobeRGB, the color management system will convert the colors from AdobeRGB into XYZ (called the “profile connection space”) and then from there to the color space defined by the color profile of your screen.

Sensor native color

---

Working color space

The working color space is internal to software, and not something you’ll experience visually when you edit an image. It’s the space within which the processing takes place. To be displayed on a monitor, the image needs to be converted from the working color space to the color space of the monitor.

In DxO PhotoLab, denoising and lens corrections are performed within sensor native color in order to achieve the best results. However, the image is then converted into our working color space, where red, green, and blue are defined according to human perception, rather than the color filters of the sensor. This ensures that editing tools such as HSL sliders, ColorWheel, or ClearView Plus are consistent, regardless of which camera has been used to take an image.

Output color space

Finally, the output color space defines how you save your images and is typically governed by what monitors can display. The most common is sRGB, and this is the standard for the World Wide Web, which uses the primaries as defined by the television standard Rec. 709. Other typical output color spaces include AdobeRGB, which is widely used in the prepress community, and Display P3, which is used for the displays in Apple’s recent desktop and notebook computers.

Fig 3: The different color spaces used in DxO’s RAW conversion pipeline

Why DxO PhotoLab ⁶ has moved to a wide gamut

Wide gamut monitors can display more vivid color than those with a standard gamut like sRGB. Whether this is useful depends on the content of an image, because under normal lighting conditions, even objects that we perceive as very colorful – for example red tomatoes or a blue sky — fit within sRGB.

However, there are a lot of colors that do not fit into sRGB. These are usually encountered on artificial objects such as brightly colored sportswear or from artificial lighting such as laser stage lights. On a wide-gamut monitor, these colors can be reproduced more accurately than on a regular monitor.

Fig 4: Comparison between wide gamut and sRGB monitors. Notice how, due to some of the values being out of gamut on the sRGB monitor, detail in the poppies’ petals is lost.

To fully exploit the capacity of a monitor, photo editing software should use a working color space with a gamut which at least matches that of the monitor. When we created our first RAW converter almost two decades ago, it was safe to assume that monitors would be either sRGB or – for the high-end, color-critical models – AdobeRGB. Choosing AdobeRGB as our working color space seemed to cover all needs, so that is what we did.

Since then, technology has evolved and monitors have improved. With Display P3 monitors used in recent Apple computers, their native red is “redder” than the “reddest red” that DxO PhotoLab ⁵ could produce. In order to simulate pure AdobeRGB red on such a monitor, the color management system must dilute it slightly and make it less intense by adding a small amount of blue. The much wider working color space of DxO PhotoLab ⁶ — which comprises both AdobeRGB and Display P3 — solves this and can produce pure, native color on such a display.

The same applies to printing. Certain printers and printing services can produce colors that are outside of AdobeRGB, and DxO PhotoLab ⁶ allows you to harness their full potential.

Fig 5: The gamut of WhiteWall’s Ultra HD photo prints (in color) in different RGB color spaces (in gray): sRGB (left), AdobeRGB (center) and DxO’s new working color space (right).

At the other end of the imaging workflow is the camera. Camera sensors do not actually have a gamut. Instead, they’re sensitive to every wavelength in the visible part of the spectrum and high-end sensors only differ from low-end models in that they better approximate the spectral sensitivity of the human eye. Thus, every color in a scene can be observed and recorded in the sensor native color space.

However, when converting from sensor native color into a working color space, as you do when developing RAW files, it may happen that a color cannot be represented. Essentially it has fallen outside of the working color space’s gamut. Having a working color space with a wider gamut therefore allows us to preserve more colors, just as they were recorded by a camera’s sensor. In combination with a wide gamut monitor and printer, the scene can then be captured, processed, and reproduced without losing its original intensity.

The problem of ‘clamping’ out-of-gamut colors

What does falling outside the color gamut mean precisely? Let’s start by going back to the idea of color values.

The simplest way to describe out-of-gamut colors and how they are managed is to think in terms of 8-bit images. In an 8-bit image, each of the red, green, and blue pixel values can range from 0 to 255. 255/0/0 would be the reddest possible red, while 128/128/128 is mid-gray.

Mathematically, a color would be out of gamut if at least one of the three RGB components had negative values. But obviously, this wouldn’t make sense as a monitor cannot emit negative light. A color could also be out of gamut if some of the values exceed the maximum. That, again, is not technically possible as a monitor cannot display values brighter than its limit.

One way of handling out-of-gamut colors is to simply clamp them to the closest allowed values, for example, setting them to 0 if they’re below the low limit, or to 255 if they’re above it. This is what many color management systems do, but they can produce unwanted results.

What do we mean by unwanted results? This ‘clamping’, whereby one of the RGB components is altered while keeping the others unchanged, means altering the hue. A more sophisticated method involves preserving the hue while accepting a reduction in saturation, and this generally yields better results. Unfortunately, even this approach can cause some problems. For instance, textures flatten as the contrasting color within those areas falls completely out of gamut.

How DxO’s reimagined color processing fixes the problem

For DxO PhotoLab ⁶, we’ve worked to ensure that all of the luminance details captured by the sensor are maintained throughout your workflow. For the best possible quality, our reengineered algorithm is designed to act in two stages: first when converting from sensor native color to working color, and then when converting from working color to output color.

As the image moves from sensor native color to working color, in order to avoid losing any of the details originally captured, the algorithm smartly analyzes the colors in each image and then desaturates – only if necessary – highly saturated colors by a small amount. This applies even to those inside the gamut, and is done in order to make headroom for those outside the gamut. Thanks to this algorithm, we can therefore produce images that contain all luminance details that were captured by the sensor — and although they appear less colorful than in the original scene, all of the tonality and detail is maintained.

The first stage (Protect saturated colors in the Color Rendering palette) has been reworked and improved compared to PhotoLab 5, the second stage (Protect color details’ in the Soft Proofing palette) is entirely new.

Fig 6: Comparison when converting to sRGB from original image

How DxO PhotoLab ⁶’s Soft Proofing mode keeps colors consistent for output

Most of the time, photographers use wide-gamut monitors which, in combination with software such as DxO PhotoLab ⁶, allow accurate reproduction of most of the colors contained in images. But when it comes to sharing images, either online or as physical prints, these output media have different gamuts that are typically a lot smaller.

A smaller gamut means that colors can look different between what you see on your monitor and what you get in print, or after exporting to other devices. Those changes in color also mean that delicate textures can be lost. Wouldn’t it be better to take that output gamut into account during editing? This is where soft proofing comes into play.

Soft proofing allows photographers to get an on-screen simulation of what an image will look like when displayed or printed on a certain device. It gives an overview of the outcome by emulating the less saturated primaries of a standard screen, or the inks of the printer and the way they physically react with paper.

How we designed our new working color space

As explained, the working color space used by a RAW converter determines the colors it can produce. The more, the better — correct?

At the same time, as its name suggests, the working color space should allow you to work with color and manipulate it in an intuitive way. For example, turning up the blue saturation in the DxO ColorWheel should produce some color that you perceive as very blue.

If it was only about reproducing color in the scene, the best working color space would be XYZ, as it contains every color that can be perceived by the human eye. However, while this color space was designed to express the color stimuli generated on our retina, it does not follow our perception of color as effectively as when using RGB color spaces. Deploying XYZ as a working color space would make our color adjustment tools behave strangely and make photo editing very challenging.

Among the RGB color spaces used today, the one that stands out is ProPhoto RGB. It covers a very large percentage of colors, and certainly all that are useful in practice. It achieves this through a trick: it uses imaginary colors for its blue and green primaries — points in the XYZ space that lie beyond spectral colors. Mathematically, this trick allows ProPhoto RGB to obtain certain spectral colors as a blend of the ProPhoto RGB primaries. The downside is that pure blue or pure green in this color space do not correspond to anything that physically exists or that any human has ever perceived.

When experimenting with our new algorithms for DxO PhotoLab ⁶ we found that even when you turn all saturation sliders up to 11, it is more intuitive to obtain results that tend toward spectral colors — and not beyond.

The diagram below shows a comparison of the hue and saturation between sRGB and ProPhoto RGB. The circles show all of the RGB values contained in each color space. The looping line symbolizes spectral colors, from a wavelength of 380 nanometers through to 750 nanometers — the most saturated colors that exist.

The gray dots represent all of the surface colors that can be observed in the real world. Since there are no surfaces that reflect light at a single wavelength, these surface colors are far less saturated than spectral colors. The sRGB color space on the left does not contain any spectral colors and you can see that some surface colors also lie outside of this color space.

The ProPhoto RGB color space on the right is far larger and easily contains all of the surface colors. However, while every RGB value in sRGB corresponds to some color, part of ProPhoto RGB lies outside of the spectral colors and corresponds to something that doesn’t exist. While fully saturated magenta, red, yellow, and cyan correspond to actual colors, fully saturated green and blue correspond to imaginary colors. This can make ProPhotoRGB counterintuitive when it comes to editing photographs.

For this reason we decided to design an RGB color space with the widest possible gamut that can be achieved utilizing spectral colors as primaries. The result is a color space that includes close to every color that can be reproduced on the best monitors and printers available today, and encompasses all of Pointer’s Gamut, the 4089 real-world surface color samples collected by scientist Doctor Michael R. Pointer at Kodak Research in 1980. link to https://onlinelibrary.wiley.com/doi/abs/10.1002/col.5080050308

Fig 9: DxO Wide Gamut (the green triangle) encompasses every possible color that a photographer might encounter in nature

The DxO PhotoLab ⁶ working color space uses spectral colors as its primaries. It is big enough to contain all real-world surface colors, and it achieves this without imaginary colors — i.e., every combination of R, G, and B in this color space represents an actual color.

Fig 10: DxO Wide Gamut (DPL6) vs AdobeRGB (DPL5 and older) vs sRGB, DisplayP3, and ProPhoto

DxO Wide Gamut: An intelligent compromise

We believe that this color space, which is quite similar to the television standard Rec. 2020, provides the best possible trade-off between preserving as much color as needed and allowing users to manipulate color in a way that feels natural and intuitive. Combined with our gamut-squeezing algorithm and soft proofing tools, it allows photographers to reproduce any color they may encounter, as closely as possible to the original, without ever losing details.