OK, OK, many of you will know this already, but for those that don’t understand what raw is all about I’m going to try to explain.
First lets consider how conventional video is recorded. When TV was first invented back in the late 1930’s a way was needed to squeeze a signal with a large dynamic range into a sensible sized signal. One important thing to consider and remember (if this article is going to make any sense) is that each additional stop of exposure has double the brightness of the previous stop. This doubling of brightness translates into a doubling of the bandwidth or data required to transmit or store it. With a limited bandwidth system like TV broadcasting, if nothing was done to reduce the bandwidth required by ever brighter stops then you would only be able to broadcast a very narrow brightness or dynamic range.
Our own visual system is tuned to pay most attention to shadows and mid tones. After all, if anything was going to eat our ancient ancestors it was most likely going to come out of the shadows. In addition the things most important to us tend to be faces, plants and other things that are visually in the mid range. As a result we tend not to notice highlights and brighter parts of the world. So, if you take a picture or a video and reduce the amount of bandwidth or data used for the highlights we don’t tend to notice it in the same way that we would notice a reduction of data in the mid range. In order to keep video transmission and storage bandwidths under control something called a gamma curve is applied to recordings and broadcasts. This gamma curve gradually reduces the amount of bandwidth/data used as the brightness of the image increases. Gamma is a form of video compression and as with almost all types of video compression you are throwing away picture information. For the darker parts of the picture there is almost no compression, while the brighter parts, especially the highlights are highly compressed. For more info on Gamma take a look at Wikipedia.
So that’s gamma and gamma is used by all conventional video cameras. The problem with gamma is that if you overexpose an image, lets say a face, you push that face up into the more compressed part of the exposure range and this starts to get quite noticeable. Even though in post production you can reduce the brightness of the overexposed face it will still often not look right because of the extra compression imparted on the face and subsequent loss of data due to the over exposure.
To make matters worse, when your working with conventional video you have a very limited amount of bandwidth (think of it as a fixed size bucket) within which you must store all you picture information. Try to put too much information into that bucket and it will overflow. As the dynamic range of modern cameras increases we end up trying to squeeze ever greater amounts of picture information into that same sized bucket. The only way we can fit more stops into our fixed size bucket is by compressing the highlights even more. This means that the recording system becomes even less forgiving of over exposure. It’s a bit of a catch 22 situation: A camera with a greater dynamic range will often be less tolerant of incorrect exposure than a camera with a smaller dynamic range (and thus less highlight compression).
But what if we could do away with gamma curves altogether? Well if we could do away with gamma curves then our exposure would be less critical. We could over expose a face and provided it wasn’t actually clipped (exceeding peak white) it could be corrected down to the right brightness in post production and it would look just fine. This would be fantastic, but the amount of data you would need to record without gamma would be massive.
Enter the Bayer Pattern sensor! Raw can work with any type of sensor, but it’s Bayer type sensors that we normally associate with raw. A Bayer sensor is a single sensor with a special array of coloured filters above the pixels that allow it to reproduce a colour image. It’s important to remember that the pixels themselves are just light sensitive devices. They do not care what colour light falls on them. They just output a brightness value depending on how much light falls on them. If we take a pixel with a green filter above it, only green light will fall on the pixel and the pixel will output a brightness value. But the signal is still just a brightness value, it is not a colour signal. It does not become a colour signal until the output from the sensor is “De-Bayered”. De-Bayering is the process of taking all those brightness values from the pixels and converting them into a colour video signal. So again taking green as an example, we read out the first pixel (top left) and as this was behind a green filter we know that it was seeing green light. For the next pixel we know it was under a blue filter, but we still need a green value for our final picture. So we use the green pixels adjacent to the blue one to calculate an estimated green value for that location. This process is repeated for all 3 primary colours for every pixel location on the sensor. This gives us a nice colour image, but also creates a lot of data. If we started off with 4096×2160 pixels (4K sensor) we would initially have 8.8 Million data samples to record or store. However when we convert this brightness only information to RGB colour we get 4096×2160 of green, 4096×2160 of blue and 4096×2160 of red. A whopping 26.5 Million data samples. A traditional video camera does all this De-Bayering prior to recording, but what if we skipped this process and just recorded the original sensor brightness samples? We could save ourselves a huge amount of data.
The other thing that normally happens when we do the De-Bayering etc is that we make adjustments to the De-Bayered signal levels to allow for things like white balance and camera gain. Adjusting the gain or white balance of a camera does not change the way the sensor works. The same amount of light falls on the same pixels and the output of the sensor does not change. What we change is the proportions of Red Green and Blue that we mix together to get the correct white balance or we add additional amplification to the signal (like turning up the volume on an audio amplifier) adding gain, to make the picture brighter.
Raw just records the unprocessed sensor output.
So if we just record the raw data coming off the sensor we dramatically reduce the amount of data we need to record. As the recorded signal won’t be immediately viewable anyway (it will need to be de-Bayered first), we don’t need to use a gamma curve. As the amount of data is lower than it would be for a fully processed colour image we can actually record the data linearly without image compression. The downside is that to view the recorded image we must process and De-Bayer the image while it’s playing back. The plus side is that at the same time as De-Bayering we can add our colour balance adjustment and any gain we need, all of this can be done in the edit suite giving much finer control and the ability to correct it and re-do it if you want. What we are doing is moving the in image processing from in the camera, to in the edit suite. In addition there is the fact that the picture is linear, without gamma compression which makes it incredibly forgiving of overexposure.
If you have never worked with raw then I suggest you give it a try. Many stills cameras can shoot in raw and it’s essentially exactly the same process with a stills camera as a video camera. If you have a camera that will do both Jpeg and raw at the same time have a go at shooting with both modes and then adjusting both in a paint package like photoshop. The difference in post production flexibility is astounding.
Of course as with all these things there is no free lunch. Your still recording a lot of data with linear raw so your recorded files will be much larger than traditional compressed video. In addition De-Bayering and processing the images takes time. Modern computers are getting faster and storage is getting cheaper, working with raw is easier now than it’s ever been. I can work with the 4K raw files from my laptop (Retina MacBook Pro) in real time by using 1/4 resolution playback. The final renders from Resolve do take a little bit of time, but once you’ve taken a bite from the raw apple it will keep tempting you back for more!