TEST TO COMPARE A PMW-F3 RECORDING ON ATAMOS SAMURAI, CONVERGENT DESIGN GEMINI and INTERNALLY.
Now I thought this would be simple. Set up my PMW-F3 with a Convergent Design Gemini and an Atamos Samurai and shoot a few shots so that I could compare the output from each recording codec. Well, the shoot side of things went just fine, a couple of different shots recording to DPX on the Gemini, ProRes on the Samurai and the internal recordings on the F3. The shots were exposed using a grey card exposed at 38%. The F3 was set to output 444 Dual Link S-Log.
However all sorts of strange things are happening when I bring the clips into my edit applications. What’s worse I get different results with different apps. For example bringing in all the material in to Adobe CS5.5 results in different luma levels for all the clips. Not only that, I also created some ProRes 4444 clips from the uncompressed DPX files and these are different again. This makes it really hard to analyse the footage when black levels are different or brightness and saturation are different. Heck all the clips were shot at exactly the same moment in time or at least within a few seconds of each other. The ONLY difference should be between the quality of the recordings. This is just more Apple Quicktime weirdness making my life difficult.
Anyway, after much head scratching and trying FCP, Avid and Premier to view the clips I was able to draw some useful conclusions which I’ll share with you here. Now the first, which may seem obvious, is that of the bunch, the Convergent Design Gemini DPX files are without doubt the best. They have zero artefacts and no added noise. After that though things become much more confusing!
Now you would expect (or at least I expected) the F3’s internal 8 bit recordings to be the worst of the bunch as all the others are 10 bit and much higher bit rates, but you know what the F3 recordings stand up really well. Why? Well in the tests I did the S-Log footage really didn’t need much of a grade to produce a nice looking image, so the clips were not being pushed very hard. One thing I’ve noticed about ProRes in the past and once again noticed in these tests, is that it’s noisy, or at least adds noise on the first encode. When you look at the Grey card in the frame grabs you can see that all of the ProRes clips show more noise than the DPX files or the F3 internal recordings, I conclude from this that ProRes is adding noise.
So in a low motion, fairly static scene the F3 recordings hold up impressively well against the ProRes ones. However I also know from experience that when you start adding in a lot of motion to a scene that 35Mb/s Mpeg can start to soften a little and in extreme circumstances really struggle and this is where ProRes starts to win as the extra data the HQ version uses really helps.
Going back to the clips from this test and looking more closely at the my wife’s face there is a difference in the way the EX codec and the ProRes codecs look. The ProRes is showing a little less blockiness and this would be a big help if you were harding the material harder. Again the clear winner is the DPX files from the Gemini, but then what do you expect!
So these are my conclusions from this test:
The Gemini DPX files produce the cleanest most gradeable images.
ProRes is noisy compared to the original source.
F3 S-Log recordings using the internal codec can be graded successfully by small amounts without obvious degradation, but I would not recommend it as you never know how much the colourist may want to push the material.
I need to shoot more challenging scenes for this type of test.
Registered users can download a 45MB zip file containing full resolution frames, graded and un-graded recorded using the Gemini, Samurai and F3. If you are not a registered user you will get the option to create an account.
[downloads_box title=”Download The Frame Grabs”]
Codec Tests Frame Grabs, 45MB
[/downloads_box]
I have a tangential question, having just purchased the Samurai for the F3. Do you think that this camera has a native color temperature that is more optimal (5600 or 3200) all other things being equal? Also, I do not currently have S-Log so how would these findings compare in REC 509 or programmable hypergammas?
Hey I can’t download the files? Any way to see the results?
Nevermind. It finally worked.
Hey Alister,
So do you notice the prores grain when you transcode to ProRes 444 from the Gemini as well or only when you record it to the Samurai?
The grain increases in all cases. The Samurai looks to be the worst.
5600 should be optimal as there is less gain in the blue channel which should reduce noise. The findings of this test would also apply to any other gamma. The test was of the recording codec more than anything else.
The download link is fixed now.
Also, if so then I have trouble understanding the point of using the Gemini if the project calls for a transcode to a compression codec anyway? I guess if pro-res is truly noisy then the next test to do, is PIX 240 at DnX 422 and a Trancoded Gemini at the same codec or DnX444.
I’m personally having alot of trouble deciding the better purchase decision. On the one hand, the Gemini is way more future proof, because these compression options will continue to get better. But is it worth it to then have to transcode for every job?
To me the largest selling point of the Gemini for the F3 is the future ability to record 1080/60p (which is huge). Sorry for the somewhat lengthy rant.
Have you looked at a Cinedeck unit? as they do all codecs.
The Gemini is aimed fair and square at high end production. It rivals and may even be better than the long standing defect hi-end standard which is HDCAM SR but at a fraction of the cost. I think that for movies, drama, 3D and any other high end productions then the Gemini must be the one to choose, but if that’s not the arena you work in the Gemini may not be right for you. It’s a tough decision. All the current recorders have pro’s and con’s, I don’t think there is a one fits all solution.
You have got me thinking. I am going to setup my EX3 with 5 Cinedeck units and I am going to record it all at the same time though a SDI matrix.
I will get back to you with my results. Really wish I had a F3 with S-Log to test it with.
Please do and let us know your findings. There is certainly a difference between the amount of noise from ProRes encoded on my Mac’s to that from the Samurai. I have not tested a cinedeck in depth but I have used them, although never taken away before and after tests to compare.
I am Busy this week but will over the next few to get the footage. I will send you the footage when done.
If you need a cinedeck to test then speak to OxygenDCT and they have demo units.
I too have the Gemini and it is incredible of the images you get. I am a little disappointed that the Gemini doesn’t have or will (as far as I know) record in a Prores HQ codec. BUT… here’s what I have done to lessen the huge file. From the SSD card I transcoded it (yes, I took the risk without backing it up first) to Prores HQ and did a difference check on the original DPX in After Effects and exposed it at +40… all black! That means that the color spacing is exactly like the DPX. I’m amazed!
If you think I’m wrong or should be questioned or if you’ve done this test and concur… then let me know.
I cannot recommend ProRes for Gemini and F3 footage. Pro Res is very noisy and taking your pristine F3 footage and adding un-necessary noise to it just does not make sense. Just try shooting a grey card with the Gemini and then encode it to ProRes. Zoom in by 300% and look at the noise. I’ve gone over this several times checking my workflow, trying different encoders etc yet every encode to proRes results in extra noise. Avid’s DNxHD 444 does not do this. I can encode, like you direct from the DPX files to DNxHD and the results really are perfect.
It’s well known and well documented that ProRes adds noise, it’s just until the advent of a true low noise camera like the F3 the extra noise went largely un noticed.
Hmmmmm… I’ll do that test myself. I’m so used to making a Prores file and we’ve never seen the noise. I am using Prores 422 HQ… is this the grade you used to see the grain?
I’m surprised that Avid’s DNxHD 444 is good for you. I’ll try this out too.
Question: Can you pump a 3G signal to the Gemini using a single HD-SDI? I can’t get mine to work?
Is your files still up for download?
3G on the Gemini is not working yet. Yes, ProRes 422 HQ is what I’ve been testing as well as ProRes 444.
Why are you surprised about DNxHD 444? it’s a 440Mb/s codec (same as SR-SQ), so it should be good.
Hy Alister,
comparison download does not work
update,
sorry, it works
Thanks Alsiter for the comparison. I is astonished how bad Samurai looks and i am also very impressed how EX internal S-log without recorder looks. EX internal is not far away from the others, hardly to see a difference.
Hey Allister
First of all thanks so much for doing these tests and bringing up this discussion! I feel like so many lower budget productions work in Pro-Res these days because in Final Cut its the easiest most accessible format. I have too recommended to “just convert everything to ProRes” because i never noticed the noise issue. This post has made me reconsider my whole workflow moving forward.
So I was wondering if you would be willing to share some of the video files from these tests in addition to the stills? I’d really love to do a test myself but I don’t have SLOG and I’m trying to decide if it makes sense to rent SLOG for this low light project I want to shoot where i don’t think I’ll be able to get my hands on an uncompressed recorder.
I’d love to share the files, but I’ve shared large video files in the past and it ends up costing me a fortune in web bandwidth. The site is paid for out of my own pocket and when I ask for donations in return for the downloads I get several hundred people downloading the files and maybe only a dozen (very gratefully received) donations. Just a 10 seconds of DPX would be 250MB plus all the other files, it would probably add up to a gig. Half a terabyte of downloads or more in a couple of days would freak out my hosting company!
I suppose I could use dropbox or similar. I’ll consider it.
Login method to get the files changed. Please sign in or register to download the files.
I can’t download the files or register?
Please try registering again.