5 Artificial Intelligence Applications for Editorial Post Teams

Artificial intelligence applications in video post production come in many flavors: automatic transcription and metadata entry, speeding up lengthy color correction and VFX processes, and so much more.

Fears of computational overreach, though, have become common in film and TV. Usually, these worries are a reaction to news of studios using performance prediction algorithms to make decisions about which films or TV series get made.

These fears are overblown—at least where film and TV post production is concerned. We are in no danger of artificial intelligence replacing the most vital post-production creative processes. AI cannot select the best take out of a pool of options, it doesn't know exactly when to let a scene breathe and when to make a particular cut for comedic effect, and it likely won't make an unorthodox scoring choice that elevates a scene.

5 Ways to Integrate AI into Your Post Workflow

What these artificial intelligence applications can do is free up your video post production teams, allowing them to concentrate on the creative work that is so valuable to the art of TV and film. Here are five artificial intelligence applications you can integrate into your post workflow to help your creatives stay creative.

1. Media Management

With AI, your DIT and AEs no longer have to spend hours entering metadata. Facial recognition software can automatically detect who is in a scene and create metadata from that information. A robust AI solution will even be able to sync your script with the associated dialogue spoken in footage and incorporate that information into the metadata attached to each clip.

You can take that scenario even further: implementing an artificial intelligence application with phonetic search or the ability to automatically transcribe footage can remove a huge amount of time and effort from productions such as documentaries, which may not have a script for large portions of footage. Rather than days spent performing hour-long searches for specific clips or takes that deviate slightly from the script, the footage is just a simple search away—no painstaking metadata entry needed.

2. VFX

Since the days of Peter Jackson's The Lord of the Rings trilogy, AI has been a tool for simulating independent movement amidst massive computer-generated crowds. The VFX applications for AI today are nearly endless, and they don't necessarily require huge amounts of computing power.

Imagine all the time your editors and VFX artists could save if rotoscoping were entirely outsourced to AI. Or if a 3D model could be generated automatically from a 2D drawing.

Those applications already exist and are very much in use today. Green screen use is fading now that computers can be trained to select human beings as objects and either lift them out of frame or change their environments. Cumbersome motion capture suits may soon be a thing of the past as computers streamline how they use performances captured on camera to generate on-set CGI renders.

Further advances in tech that can automatically replace faces may also eventually remove the need in some cases for prosthetics when they are costly, uncomfortable, or time-consuming for actors.

3. Sound

When constructing the sound design of a film, options are key. If, for example, a director finds a sound they want to keep in the film—but it doesn't sound quite right—a good sound artist will want to present them with options. Artificial intelligence applications include finding similar sounds within a given library or even generating similar sounds on their own.

In a similar vein, composers with this kind of tool can gather a wide array of potential sounds to integrate into their scores, creating innovative soundscapes. Editors, for their part, can use AI-generated scores to help the creative process along—instead of lumps of static stock music, AI-generated scores dynamically adapt to what the editor does.

Or, to solve more mundane problems, current AI tools can remove wind sounds from audio tracks, solving an issue that has plagued every production since the dawn of sound in film.

These programs are unlikely to spontaneously generate Hans Zimmer-quality scores or create the full soundscape of a film. However, they will solve some persistent problems that sound designers have had to deal with for decades—ultimately allowing them more mental bandwidth to focus on creation rather than creative problem-solving.

4. Color Correction

Colorists won't be out of a job with the entry of AI, but what the technology should do is let colorists spend more time getting a key image exactly right. An AI program can automatically match the rest of the scene to that key image or shot and do it much more quickly than a colorist could do on their own—i.e., in seconds rather than 30 minutes.

This kind of automation reduces the time it takes to grade an entire film, potentially from weeks to mere days. While the options currently on the market aren't perfect, they're a massive leap forward from even just a few years ago; any errors are easily smoothed out in the polishing phase.

5. Quality Assurance

AI offers a way out of hours spent manually dustbusting. The right tech is built to deal with those unwanted pixels, even if they belong to a boom in the shot. With solutions' current ability to erase booms, directors and editors get the freedom to choose the best take even if the boom is showing.

Upscaling old or lower-resolution footage can be handled through AI as well, which unlocks older films for commercial purposes. Future applications may even be able to create new frames from existing ones, upping the frame rate for films.

Putting the Art in Artificial Intelligence

Artificial intelligence applications are just that—applications of a particular tool to the creative process. AI will always need a human intellect to make the creative decisions it can't. By leveraging AI to create workflow efficiencies and free up mental bandwidth to make those decisions, post-production teams have the breathing room to produce their best work.

More Like This