Visual Effects on Act of Valor

At Bandito Brothers, we have always prided ourselves in being authentic and real.  This is one of the values that led Scott and Mouse to cast real Navy SEALS for the main roles in Act of Valor, and doing things like using live ammo in certain scenes.  So at first glance, it would seem surprising that we had hundreds of visual effects shots in the movie.  About half of them were in response to issues that resulted from shooting with the 5D, but we also did have to add things like blood hits, and do fixes like painting out crew members.  Additionally, we had a fair bit of motion graphics work, both overlaid graphics for the audience, and screen replacement for briefing information.

The film based work was very straight forward, you got a stack of 24p 2K DPX film scans, and you rendered out a stack of fixed DPXs when finished.  Film didn’t have to be converted from 30p to 24p, or exhibit rolling shutter artifacts, or Twixtor issues.  But compositing a 30p Canon files with a 24p film scan does take some work.

Twixtor is an amazing tool, but it is not perfect.  Whenever there is too much motion in a scene for the software to accurately track what is going on, it gives some strange results.  Since Act of Valor is a fairly dynamic movie, to put it lightly, we had our share of issues.  When issues with Twixtor were encountered, we had a fairly methodical, but time consuming solution.  We would bring three versions of the shot into the Premiere sequence: the original 30p version, which would drop frames to play at 24p, another copy reinterpreted at 24p, (so it was longer, but every frame was accessible) and the Twixtor interpolated render.  We would start by examining the Twixtor version, and replacing the bad frames with ones from the original file.  If there was enough action to make Twixtor totally useless, then that chaos was usually enough to hide the dropped frames anyway.  On shots that were smoother, where that would be too noticeable, we would sometimes have to use the reinterpreted version, and slow the shot down for a couple frames here and there to get through the rough spots.  This obviously took awhile, and I got really good at moving single frame clips around on the Premiere timeline.  Smooth playback with lots of layers and micro-cuts was critical for this type of work, so the fact that CS5 played back everything in real-time was critical, especially for rapidly comparing different variations.

For complex visual effects shots, our artists had to do this cleanup process on nearly every 5D source layer, before they could even start doing any regular VFX work.  Brett Novak did a lot of the early experimentation with compositing 30p Canon files with 24p film footage.  Andrew Furlong spent the better part of two years working on shots to keep pace with Scott’s editorial process.  Besides the 30p conversion issue, the visual effects were processed in a fairly traditional way, and exported to Cineform when complete.

We also dealt with a number of different types of rolling shutter artifacts.  My favorite ones were the muzzle flashes at night.  Because of the way CMOS sensors record light data sequential from top to bottom, a bright flash results in horizontal lines of varying brightness in the recorded image.  There is a lot of gunfire in Act of Valor, and most of it is in the dark, which exaggerated the issue.  This is one case where shooting 30p actually worked in our favor.  Since we had extra frames available, we would just reinterpret the footage, and manually cut out the bad frames we wanted to remove, and enough other ones to get it down to 24p.  If this didn’t result in smooth playback, someone would have to manually stitch together the good parts of consecutive frames, where once again, it was helpful to have 30 fps to start with.

We also had over a hundred subtitles, due to the Spanish and Russian dialog of the bad guys. (Some might argue that we should have subtitled the SEAL’s jargon)  These weren’t necessarily complicated, but had to be inserted into the workflow AFTER the color and grain passes, and the reframing for different aspect ratios.

We didn’t have much in the way of motion graphics, but treating them the same as visual effects was probably a mistake.  They were giving us issues right to the end, primarily because of all the revisions that graphics usually get, and because the original templates weren’t designed with our final 2K online process in mind.  Most of them were rendered as overlay layers with alpha channels, and composited over the backgrounds directly in the Premiere Pro online sequence, which gave us more flexibility to adjust them than rendering the backgrounds into the shots would have.

It was important to fix all of the visual effect, frame rate conversion artifacts, and rolling shutter issues before we further processed the image with color correction and our texture pass.  Once those modifications were rendered into the footage, pulling up source frames to cover up problems would no longer match.  Simple dust busting fixes on the other hand could easily be done on the corrected image.  So that was one of the last steps I did, pulling up individual DPX files in Photoshop for repair with the Clone-Stamp tool.

Tags: 2K I/O, ActOfValor, Adobe, Canon, Cineform, Compression, DSLR