How Peter Jackson Used Artificial Intelligence to Restore the Video & Audio Featured in The Beatles: Get Back

Much has been made in recent years of the “de-aging” processes that allow actors to credibly play characters far younger than themselves. But it has also become possible to de-age film itself, as demonstrated by Peter Jackson’s celebrated new docu-series The Beatles: Get Back. The vast majority of the material that comprises its nearly eight-hour runtime was originally shot in 1969, under the direction of Michael Lindsay-Hogg for the documentary that became Let It Be.

Those who have seen both Linday-Hogg’s and Jackson’s documentaries will notice how much sharper, smoother, and more vivid the very same footage looks in the latter, despite the sixteen-millimeter film having languished for half a century. The kind of visual restoration and enhancement seen in Get Back was made possible by technologies that have only emerged in the past few decades — and previously seen in Jackson’s They Shall Not Grow Old, a documentary acclaimed for its restoration of century-old World War I footage to a time-travel-like degree of verisimilitude.

“You can’t actually just do it with off-the-shelf software,” Jackson explained in an interview about the restoration processes involved in They Shall Not Grow Old. This necessitated marshaling, at his New Zealand company Park Road Post Production, “a department of code writers who write computer code in software.” In other words, a sufficiently ambitious project of visual revitalization — making media from bygone times even more lifelike than it was to begin with — becomes as much a job of traditional film-restoration or visual-effects as of computer programming.

This also goes for the less obvious but no-less-impressive treatment given by Jackson and his team to the audio that came with the Let It Be footage. Recorded in large part monaurally, these tapes presented a formidable production challenge. John, Paul, George, and Ringo’s instruments share a single track with their voices — and not just their singing voices, but their speaking ones as well. On first listen, this renders many of their conversations inaudible, and probably by design: “If they were in a conversation,” said Jackson, they would turn their amps up loud and they’d strum the guitar.”

This means of keeping their words from Lindsay-Hogg and his crew worked well enough in the wholly analog late 1960s, but it has proven no match for the artificial intelligence/machine learning of the 2020s. “We devised a technology that is called demixing,” said Jackson. “You teach the computer what a guitar sounds like, you teach them what a human voice sounds like, you teach it what a drum sounds like, you teach it what a bass sounds like.” Supplied with enough sonic data, the system eventually learned to distinguish from one another not just the sounds of the Beatles’ instruments but of their voices as well.

Hence, in addition to Get Back‘s revelatory musical moments, its many once-private but now crisply audible exchanges between the Fab Four. “Oh, you’re recording our conversation?” George Harrison at one point asks Lindsay-Hogg in a characteristic tone of faux surprise. But if he could hear the recordings today, his surprise would surely be real.

Related Content:

Watch Paul McCartney Compose The Beatles Classic “Get Back” Out of Thin Air (1969)

Peter Jackson Gives Us an Enticing Glimpse of His Upcoming Beatles Documentary The Beatles: Get Back

Lennon or McCartney? Scientists Use Artificial Intelligence to Figure Out Who Wrote Iconic Beatles Songs

Artificial Intelligence Program Tries to Write a Beatles Song: Listen to “Daddy’s Car”

Watch The Beatles Perform Their Famous Rooftop Concert: It Happened 50 Years Ago Today (January 30, 1969)

How Peter Jackson Made His State-of-the-Art World War I Documentary They Shall Not Grow Old: An Inside Look

Based in Seoul, Colin Marshall writes and broadcasts on cities and culture. His projects include the book The Stateless City: a Walk through 21st-Century Los Angeles and the video series The City in Cinema. Follow him on Twitter at @colinmarshall or on Facebook.

by | Permalink | Comments (5) |

Support Open Culture

We’re hoping to rely on our loyal readers rather than erratic ads. To support Open Culture’s educational mission, please consider making a donation. We accept PayPal, Venmo (@openculture), Patreon and Crypto! Please find all options here. We thank you!

Comments (5)
You can skip to the end and leave a response. Pinging is currently not allowed.
  • Peter Conheim says:

    I must correct the record here regarding the picture “restoration” (sic) of the footage that comprises GET BACK. The image is not “sharper” – quite the opposite. It is many, many times softer and has had much of the actual picture detail REMOVED. This was the unfortunate choice that the restoration team made: the “de-grain” all the footage, digitally, and then “re-grain” it. This is not an acceptable archival practice. The result is that so much of the original detail in the image has literally been sanded away that subjects look waxy, soft and dull. This affects the _entire_ running time of GET BACK (with the exception of the majority of the older archival footage inserts).

    There are times when this process is so egregious that the “machine learning” (sic) literally breaks down. Paul’s face appears with sudden “crows’ feet” around the corners of his eyes which look like tattoos because the rest of his features are flattened entirely, but the processor couldn’t deal with some face lines. John’s fur jacket loses each strand and becomes a moving set of flat globs. And deep focus shots suffer the worst throughout, where the viewer sees a sharp object in the foreground and slushy goo in the background.

    It’s an appalling result. DNR (digital noise reduction) and de-graining techniques must be used sparingly, if at all, not as a sandblaster.

  • Michael Helm says:

    That’s interesting – to me, the look of the show is amazing – it looks like it was filmed Yesterday.
    That is it looks like it was shot with modern equipment right now & doesn’t have the muddy look I now associate with the ’60’s. I have seen “Let It Be” but it was a long, long time ago & it’s unavailable except for the low-res snippets on youtube of the rooftop show – I wouldn’t consider that a safe comparison. I don’t trust my memory of the original after that stretch of time.

    Can you point to one (or more) specific scenes that I should look at more closely? I’d like to see if I can learn to see what you’re seeing.

    I have had similar situations develop with digital photos. I took some pictures of a person once & used a lot of restorative techniques in Photoshop to clean up the image & remove shadows, disagreeable stuff in the picture, and blur out some distractive junk in the background. I liked it. Some people liked it. But the person in the photo, for whom it was taken, hated it.

  • Peter Conheim says:

    (And I had line breaks in that screed above, but they didn’t seem to make it to the post!)

  • Peter Conheim says:

    I’m going to try to re-submit this so it is more readable:

    Absolutely, though to be clear, the digital scrubbing the film took is evident across about 99% of its entire running time! I gradually observed shots here and there which survived better, most likely because they had less complex material for the software to sort out: static shots of people with non-textured clothing, for instance.But all you have to do is look at the grilles on the various Fender amps at Twickenham. Take a look at the 8×10 (I believe) bass speaker stack that sits behind Paul as he is birthing “Get Back”. You will observe that much of the time, the front of the amp appears like a gray, solid monolith. It isn’t. It’s a sparkly, sculpted grille cloth with lots of texture. Here’s an example of (more or less) what details the image once held… and this is actually not that sharp a picture, but it STILL shows details: or, sharper: This isn’t the best example I can find for comparison, but it will suffice. Stop your playback around 0:20. Note how the grille on the amp is flattened, sure, but look at ALL the faces. Everything has lost natural detail. Everyone looks made of wax. Worse, look at the same clip and watch Paul’s face (and eyes) around 45-50 seconds in: there’s a (brief, it’s way worse elsewhere!) observance of his face having been so scrubbed of detail that when the crows’ feet appear in the corners of his eyes, he looks like a transforming space alien! If the whole show were online, it would be much easier to point at the most egregiously bad moments… but, as said, the ENTIRE thing has been processed. Actually, here’s a better way to see the larger detail loss. Start at 1:15. There’s a further note of clarification which needs to be made regarding how the original LET IT BE looked for most people, which has been lost in all the noise: most people know the film from the short-lived 1980s video release (on tape, LaserDisc and CED). It is hard to find words to describe how abominable that transfer looked, but the short version is: movie was shot 16mm “Academy” ratio, or a square. When prints of the film were made, they were HARD MATTED into a 1.85 widescreen (top and bottom were painted out with a black bar). When the video transfer was done, it used one of these “cropped”/”matted” prints, and then FURTHER zoomed into the image by lopping the left and right off for television screens. Tons of picture was lost and the image was unnaturally enlarged. Furthermore, the original 16mm-to-35mm blow-up was simply not great. Lab work wasn’t that well done – plus, the original 35mm prints were IB Technicolor, which, while featuring highly saturated colors, could impart a certain additional softness if they weren’t extremely well printed. So, the deck was stacked against the film looking good for 50 years. When Jackson and his team went back to the nice, sharp 16mm negative to start all over again, they made choices. One was to matte it back into widescreen again (controversial, but there IS a precedent for it), and the other was to decide no one wanted to see the original film grain, and brutally remove it. This latter practice is what resulted in the mess we have now.

  • Peter Conheim says:

    Michael, I wrote a (bloated) reply to you here and cited some examples, but it didn’t seem to get accepted as a post. Check out the “McCartney writes ‘Get Back’ clip that is posted everywhere and take a look at the bass speaker cabinet. Then look at any older posted photos from the same basic shot at Twickenham…

Leave a Reply

Open Culture was founded by Dan Colman.