When is 4k really 4k, Bayer Sensors and resolution.
First lets clarify a couple of term. Resolution can be expressed two ways. It can be expressed as pixel resolution, ie how many individual pixels are there on the sensor. Or as TV lines or TVL/ph, or how many individual lines can I see. If you point a camera at a resolution chart, what you talking about is at what point can I no longer discern one black line from the next. TVL/ph is also the resolution normalised for the picture height, so aspect ratio does not confuse the equation. TVL/ph is a measure of the actual resolution of the camera system. With video cameras TVL/ph is the normally quoted term, while pixel resolution or pixel count is often quoted for film replacement cameras. I believe the TVL/ph term to be prefferable as it is a true measure of the visible resolution of the camera.
The term 4k started in film with the use af 4k digital intermediate files for post production and compositing. The exposed film is scanned using a single row scanner that is 4,096 pixels wide. Each line of the film is scanned 3 times, once each through a red, green and blue filter, so each line is made up of three 4K pixel scans, a total of just under 12k per line. Then the next line is scanned in the same manner all the way to the bottom of the frame. For a 35mm 1.33 aspect ratio film frame (4×3) that equates to roughly 4K x 3K. So the end result is that each 35mm film frame is sampled using 3 (RGB) x 4k x 3k, or 36 million samples. That is what 4k originally meant, a 4k x 3k x3 intermediate file.
Putting that into Red One perspective, it has a sensor with 8 Million pixels, so the highest possible sample size would be 8 million samples. Red Epic 13.8 million. But it doesn’t stop there because Red (like the F3) use a Bayer sensor where the pixels have to sample the 3 primary colours. As the human eye is most sensitive to resolution in the middle of the colour spectrum, twice as many of these pixel are used for green compared to red and blue. So you have an array made up of blocks of 4 pixels, BG above GR.
Now all video cameras (at least all correctly designed ones) include a low pass filter in the optical path, right in front of the sensor. This is there to prevent moire that would be created by the fixed pattern of the pixels or samples. To work correctly and completely eliminate moire and aliasing you have to reduce the resolution of the image falling on the sensor below that of the pixel sample rate. You don’t want fine details that the sensor cannot resolve falling on to the sensor, because the missing picture information will create strange patterns called moire and aliasing.
It is impossible to produce an Optical Low Pass Filter that has an instant cut off point and we don’t want any picture detail that cannot be resolved falling on the sensor, so the filter cut-off must start below the sensor resolution. Next we have to consider that a 4k bayer sensor is in effect a 2K horizontal pixel Green sensor combined with a 1K Red and 1K Blue sensor, so where do you put the low pass cut-off? As information from the four pixels in the bayer patter is interpolated, left/right/up/down there is some room to have the low pass cut off above the 2k pixel of the green channel but this can lead to problems when shooting objects that contain lots of primary colours. If you set the low pass filter to satisfy the Green channel you will get strong aliasing in the R and B channels. If you put it so there would be no aliasing in the R and B channels the image would be very soft indeed. So camera manufacturers will put the low pass cut-off somewhere between the two leading to trade offs in resolution and aliasing. This is why with bayer cameras you often see those little coloured blue and red sparkles around edges in highly saturated parts of the image. It’s aliasing in the R and B channels. This problem is governed by the laws of physics and optics and there is very little that the camera manufacturers can do about it.
In the real world this means that a 4k bayer sensor cannot resolve more than about 1.5k to 1.8k TVL/ph without serious aliasing issues. Compare this with a 3 chip design with separate RGB sensors. With a three 1920×1080 pixel sensors, even with a sharp cut-off low pass filter to eliminate any aliasing in all the channels you should still get at 1k TVL/ph. That’s one reason why bayer sensors despite being around since the 70s and being cheaper to manufacture than 3 chip designs (with their own issues created by big thick prisms) have struggled to make serious inroads into professional equipment. This is starting to change now as it becomes cheaper to make high quality, high pixel count sensors allowing you to add ever more pixels to get higher resolution, like the F35 with it’s (non bayer) 14.4 million pixels.
This is a simplified look at whats going on with these sensors, but it highlights the fact that 4k does not mean 4k, in fact it doesn’t even mean 2k TVL/ph, the laws of physics prevent that. In reality even the very best 4k pixels bayer sensor should NOT be resolving more than 2.5k TVL/ph. If it is it will have serious aliasing issues.
After all that, those that I have not lost yet are probably thinking: well hang on a minute, what about that film scan, why doesn’t that alias as there is no low pass filter there? Well two things are going on. One is that the dynamic structure of all those particles used to create a film image, which is different from frame to frame reduces the fixed pattern effects of the sampling, which causes the aliasing to be totally different from frame to frame so it is far less noticeable. The other is that those particles are of a finite size so the film itself acts as the low pass filter, because it’s resolution is typically lower than that of the 4k scanner.
Interesting, why do heavy VFX shows seem to like 4K Red images for post, if it is in reality a 2k image? There must still be some advantage to having that size of aspect ratio image?
Red produces a sub 4k image, around 3.5k, so it is more than 2k/HD, so it does allow for re-sizing etc without image loss if the final output is 2k/HD.
Interesting, why do heavy VFX shows seem to like 4K Red images for post, if it is in reality a 2k image? There must still be some advantage to having that size of aspect ratio image?
Red produces a sub 4k image, around 3.5k, so it is more than 2k/HD, so it does allow for re-sizing etc without image loss if the final output is 2k/HD.
This article fails to take into account the fact that aliasing does not equal resolution. There are exactly zero sensors on the market that process information from photosites as quads (rgbg). The industry has known about this for over 20 years.
http://en.m.wikipedia.org/wiki/Super-resolution
Your correct that Aliasing does not equal resolution, I never said that. But in respect of current technology video cameras, aliasing does limit real world useable resolution. Any resolution above nyquist will likely contain undesirable artefacts. One of the biggest issues is that aliases on moving image data travel in the opposite direction to the true image information, this is distracting to view but more importantly it causes problems for the interframe codecs commonly used for distribution and broadcast. So while it is possible to have resolution beyond aliasing it contain highly undesirable aliasing artefacts that degrade the image quality by more than the benefit gained by any small resolution increase. In practical, real world terms aliasing will limit the useable resolution of a video camera. As I said bayer sensor pixels are arranged in 2×2 blocks, I did not say that that is how they are read or processed. From an aliasing point of view the readout method only makes a small difference to aliasing. The aliasing comes from the inevitable gaps between the different coloured photo sites. When the resolution of the image falling on the sensor is greater than the pitch of the pixels some information falls between samples and is lost, this causes aliasing. The wide spacing of the R and B samples in a Bayer sensor is particularly problematic as is the diagonal resolution of the green channel. Bayer sampling does attempt to mitigate this by averaging what might be between samples and cross colour leakage is used to help as well. But the structure and layout of a Bayer sensor means that it is impossible to fully sample the incoming image in each of the R, G and B channels at the full pixel count without considerable, undesirable aliasing. The bottom line is that there is a very big difference between a 4K R, 4K G, 4K B image as obtained with a film scanner scanning each photosite in R, G and B and the image that comes from a sensor with 4K horizontal bayer pattern pixels. 4K used to mean a 4K RGB film scan. However we are now being sold cameras pertaining to be 4K that do not provide 4K RGB images. Manufactures like to tell us that this or that camera is a 4K or 5K camera when it is not. It is a camera with 4K of horizontal pixels and the resolution will be less than 4K,typically around 0.8 x the pixel count for G, even less for R and B.
I’m not sure of the relevance of the Superresolution link as this largely deals with static subjects and the use of pixel shifts and multiple image stacking to gain a greater resolution than that of the imaging system itself. I’ve used this technology for years for astrophotography where it is know simply as stacking or drizzle, It relies on the subject having a minute amount of movement and it is this sub pixel movement from frame to frame over many, many frames, typically at least 10, that allows you to gain a resolution increase plus a noise decrease. This has no practical application in moving video. The single image method has to guess at what the magnified image should look like and then creates a magnified image based on estimations, it does not mitigate aliasing and if the original image is aliased it is of little benefit.
The post was written following a sales person from a well known manufacturer trying to convince me that his 2.4K camera was better than an HD camera because it’s 2.4K. The reality was that his 2.4K camera had a 2.4K bayer sensor, so the actual resolution would be roughly the same as most HD camera cameras. But because he didn’t know any different his assumption was that a 2.4K sensor meant higher than HD resolution.