A custom tonemapper and eye adaptation

August 11, 2021
A custom tonemapper and eye adaptation

When you start working with pixel art in UE4 y'll notice very soon that something is off. The colours of the sprites, most likely picked from a well-defined palette, look brighter than they should be and this is a big turn-off for any artist working in the engine. The culprit is UE4's default tonemapper, which is aimed at portraing light and colors in the most realistic way possible, and certainly not to display textures as they are. The quick solution is to create a post-process material that will replace the default tonemapper, giving us more control over how the colours are actually handled.

Base tonemapper material

Create a new material like the one above, and make sure to set its domain to "Post process" and its blendable location to "Replace tonemapper". The "V3LinearTosRGB" node is a simple function to convert each channel in a linear Vector3 to its sRGB counterpart. Nothing crazy.

After you created the material, add a new PostProcessVolume to your scene, be sure to check "Infinte Extent" so the volume will affect the entire scene, and under "Rendering Features", then "Post Process Materials" you can add the material you just created.

Running the game now should display the sprites correctly.

Normal scene with tonemapper

However, the tonemapper in UE4 is an essential part of the graphics pipeline, and replacing it means losing access to other effects you might want in your game, like anti-aliasing or bloom. Unfortunately, you'll have to decide for yourself if you'd rather have color accuracy or the full extent of UE4 graphical features. For Nexatli I wanted to find a middle-ground, rebuilding the effects that are unavailable with a custom tonemapper.

The first one was eye adaptation, or camera exposure, which is fairly easy to implement, but needs extra care to get right for pixel art. Here's the updated tonemapper:

Normal scene with tonemapper

What I wanted to achieve is really simple: in normal conditions the colours should remain intact. As soon as the scene contains pixels brighter than 1 (rgb 255-255-255) the brightest pixel should become bright as white, and all others should become darker. UE4 already provides the tools for this, but I needed to set some boundaries. The extensive documentation for what I'm about to explain can be found in the following links

https://www.unrealengine.com/en-US/tech-blog/how-epic-games-is-handling-auto-exposure-in-4-25
https://docs.unrealengine.com/4.26/en-US/RenderingAndGraphics/PostProcessEffects/AutomaticExposure/

In the PostProcessVolume, under "Lens" then "Exposure" you'll find all the necessary settings.

Exposure settings in the post process volume

The main parameter you need to tweak is the "Exposure Compensation". The documentation contains a handy formula to get the compensation needed for the eye adaptation to be based on pure white, instead of gray 18%.

log2(target white / gray point) = exposure compensation

The gray point in UE4.25 and after is 0.18, therefore:

log2(1 / 0.18) = 2.473931

If you set the exposure compensation to 2.473931 the colors will still look off. That's because the engine is still making the scene brighter or darker according to the luminance of the elements in the scene. This parameter is the EV100 value, and that is what drives the eye-adaptation process.

We can restrict the EV100 with the two parameters "Min EV100" and "Max EV100". Set the Min value to 2.473931 and the Max value to something really high like 20, for now.
The reason for this is simple: since every pixel in the textures or in your pixel art palette is between 0 and 255 in brightness, the EV100 will likely be very low and it will make everything brighter as a result.
Running the game now will display the proper colours, until we run into some luminous pixels.

There are to reasons behind this behavior:

  1. The exposure compensation is still calibrated for a value of 1. The background has a brightness of 10 (a little less, since the trees and sky are not purely white).
  2. The EV100 can rise up to 20 and we're not controlling the brightness in this situation.

This is where the Exposure conpensation curve comes into play.

Create a new curve asset and add a key at (2.473931, 2.473931). This will tell the engine that when the EV100 is less or equal than 2.473931 we want the exposure to be exactly 2.473931.

The next step is adding another key, preferably one with a very high value, to handle all EV100 in between. To calculate a new one you'll need the debug tools.
Create a new material that's only emissive with a high value, like 100, then add it to a cube in your scene.

Emissive material

Open the console and type ShowFlag.VisualizeHDR 1. This will enable a handy tool to debug the eye adaptation process. When you point the camera at the bright cube, the third value on the right of the pointer should tell you the EV100 of that colour. In this case, a colour that is (100, 100, 100) produces an EV of 9.122.

Eye adaptation debug tool

The last step involves using the Pixel inspector tool. It can be enabled under the Window -> Developer tools menu.

While pointing the mouse at the cube with the Pixel inspector active we can determine the exact color output and at the same time change the exposure compensation. Through a short trial-and-error, we can find that the value 6.307 makes the cube a perfect white (RGB 1-1-1).
We can now add a new key to the curve, this time at (9.122, 6.307).

Base tonemapper material

It's time to apply the curve to the PostProcessVolume. Set the Exposure compensation to 0 and load the curve asset right below. Dont' forget to set the Max EV100 to 9, since we know that beyond 9.122 the curve has no data, but you can repeat the process to add more points for higher EVs if you need them. The resulting effect will keep the natural colours of the pixel art, while applying the eye adaptation only when there are bright elements in the scene.