Technically, this post should be called noise "redirection" on HDSLR footage but we'll get to that in a bit. About a year and a half ago I shot a documentary with the Canon 5D mark II firmware in it's infancy. No 24p, no manual exposure, no manual audio, obviously not the best way to do it. On top of that I was in Washington State in September/October with cloudy light and no light kit. The answer...dah tah dah...ISO 3200/6400! My footage still has those rainbows of random CMOS imager noise ruining an otherwise nice image.
So you now know my dilemma. I want to use the footage but it looks like it was shot on a minio flip with a DoF adapter. After Effects has a noise reduction effect, but it tends to smooth over details and create "shiny plastic" looking video - nasty. There's also Neat Video, but that uses a similar low pass filter algorithm causing detail loss and it's currently $99 - I wanna be cheap here.
I've said it before here; film grain is supposedly "artistic" right? I mean you see it in just about every projected film. So what I decided to do was to turn that grotesquely colorful CMOS imager noise into film grain. How, you may ask? Our eyes see image details in luma (brightness), not chroma (color). Every image compression algorithm takes advantage of that and throws away lots of color information. So the first step to redirecting noise is to convert the image to a color space that separates luma information from chroma information, so I can keep the detail while operating on the CMOS characteristic colorful noise. After Effects can do a conversion from RGB to YCbCr using the "Channel Combiner" Effect (click on picture below). One thing I found out recently on an Adobe blog post is that when Adobe says 'YUV' they really mean YCbCr. For clarity I'm going to talk about the YUV conversion just so the overall topic doesn't get lost and the process is easier to follow in After Effects. The 'Y' channel represents brightness or luma. We don't want to touch that because that's our sharpness and detail. The 'U' and 'V' channels represent chroma, or color information. So for the "From" parameter select "RGB to YUV" in the drop down box. Now we are in a working space where we can operate on chroma separately from luma we can start chroma noise redirection to somthing that looks like film grain.
As with any noise reduction algorithm we're going to apply a filter to filter out the noise. I'm not an image processing expert, but I do know from raw image conversion that if I blur the chroma channels my remaining luma noise tends to look monochromatic (the holy grail of film grain). So the next effect I'm going to apply is "Channel Blur." Since we are already in "YUV", 'Y' equates to the "Red" Channel and 'U' and 'V' equate to the "Blue" and "Green" channels. We can now look at the noise in each of the chroma channels by using the little button with the three RGB circles below the preview window (see first picture in sequence below). I know it's next to impossible to see the noise with the small pictures in the main section of this blog post, so I recommend you click on the pictures to enlarge them. What you will see with the larger pictures is that the noise on the "Green" (i.e. 'U' or Cb) channel looks reasonably OK. The noise on the "Blue" (i.e. 'V' or Cr) channel looks terrible. Let's low pass filter that noise using the Channel Blur effect. This is done to taste, so to speak. I found a reasonable value for this footage is a "Blue Blurriness" of 30.
The final step is to convert YUV back to RGB. I just used that same Channel Combiner effect, but selected "YUV to RGB" as the "From" parameter. Nothing special otherwise. The image below shows the before and after. Again, you'll likely need to click on the picture to enlarge it enough to see the effect. The noise is still there in the "After" image, but it's been filtered and redirected to look like monochromatic film grain instead of colorful CMOS noise. Notice that unlike the "Reduce Noise" effect there is still reasonable detail in the fabric of his sweatshirt because we never touched the luma channel.
That mostly saved the footage. My half-ass first attempts at audio actually required more fixing. What's left is a conversion from 30.0 frames/sec to 23.976 frames/sec (i.e. 24p conversion). Turns out that After Effects can be quite adept at that as well, but more on the frame rate conversion topic later.