Artificial Intelligence Brings to Life Figures from 7 Famous Paintings: The Mona Lisa, Birth of Venus & More

Denis Shiryaev is an AI wiz­ard who has lib­er­al­ly applied his mag­ic to old film—upscaling, col­oriz­ing, and oth­er­wise mod­ern­iz­ing scenes from Vic­to­ri­an Eng­land, late Tsarist Rus­sia, and Belle Époque Paris. He trained machines to restore the ear­li­est known motion pic­ture, 1888’s Round­hay Gar­den Scene and one of the most mythol­o­gized works of ear­ly cin­e­ma, the Lumière Broth­ers 50-sec­ond Arrival of a Train at La Cio­tat Sta­tion.

Shiryaev’s casu­al dis­tri­b­u­tion of these efforts on YouTube can make us take for grant­ed just how extra­or­di­nary they are. Such recre­ations would have been impos­si­ble just a decade or so ago. But we should not see these as his­toric restora­tions. The soft­ware Shiryaev uses fills in gaps between the frames, allow­ing him to upscale the frame rate and make more natur­is­tic-look­ing images. This often comes at a cost. As Ted Mills wrote in an ear­li­er Open Cul­ture post on Shiryaev’s meth­ods, “there are a lot of arti­facts, squooshy, mor­ph­ing moments where the neur­al net­work can’t fig­ure things out.”

But it’s an evolv­ing tech­nol­o­gy. Unlike wiz­ards of old, Shiryaev hap­pi­ly reveals his trade secrets so enter­pris­ing coders can give it a try them­selves, if they’ve got the bud­get. In his lat­est video, above, he plugs the NVIDIA Quadro RTX 6000, a $4,000 graph­ics card (and does some grip­ing about rights issues), before get­ting to the fun stuff. Rather than make old film look new, he’s “applied a bunch of dif­fer­ent neur­al net­works in an attempt to gen­er­ate real­is­tic faces of peo­ple from famous paint­ings.”

These are, Shiryaev empha­sizes, “esti­ma­tions,” not his­tor­i­cal recre­ations of the faces behind Leonardo’s Mona Lisa and Lady with an Ermine, Botticelli’s mod­el for The Birth of Venus, Vermeer’s for Girl with a Pearl Ear­ring, or Rembrandt’s The Night Watch. In the case of Amer­i­can Goth­ic, we have a pho­to of the mod­el, artist Grant Wood’s sis­ter, to com­pare to the AI’s ver­sion. Fri­da Kahlo’s Self-Por­trait with Thorn Neck­lace and Hum­ming­bird gets the treat­ment. She left per­haps a few hun­dred pho­tographs and some films that prob­a­bly look more like her than the AI ver­sion.

The GIF-like “trans­for­ma­tions,” as they might be called, may remind us of a less fun use of such tech­nol­o­gy: AI’s abil­i­ty to cre­ate real­is­tic faces of peo­ple who don’t exist for devi­ous pur­pos­es and to make “deep fake” videos of those who do. But that needn’t take away from the fact that it’s pret­ty cool to see Botticelli’s Venus, or a sim­u­la­tion of her any­way, smile and blink at us from a dis­tance of over 500 years.

Relat­ed Con­tent:

Watch Scenes from Czarist Moscow Vivid­ly Restored with Arti­fi­cial Intel­li­gence (May 1896)

Watch AI-Restored Film of Labor­ers Going Through Life in Vic­to­ri­an Eng­land (1901)

Icon­ic Film from 1896 Restored with Arti­fi­cial Intel­li­gence: Watch an AI-Upscaled Ver­sion of the Lumière Broth­ers’ The Arrival of a Train at La Cio­tat Sta­tion

The Ear­li­est Known Motion Pic­ture, 1888’s Round­hay Gar­den Scene, Restored with Arti­fi­cial Intel­li­gence

Josh Jones is a writer and musi­cian based in Durham, NC. Fol­low him at @jdmagness

by | Permalink | Comments (6) |

Sup­port Open Cul­ture

We’re hop­ing to rely on our loy­al read­ers rather than errat­ic ads. To sup­port Open Cul­ture’s edu­ca­tion­al mis­sion, please con­sid­er mak­ing a dona­tion. We accept Pay­Pal, Ven­mo (@openculture), Patre­on and Cryp­to! Please find all options here. We thank you!

Comments (6)
You can skip to the end and leave a response. Pinging is currently not allowed.
  • Sandy Rutger says:

    Inter­est­ing that you choose a Fri­da Kahlo self-por­trait and then gen­er­ate a face. Ummm…it’s a Fri­da Kahlo self-por­trait, so we KNOW exact­ly what face that is.

  • Eleanor Bailey says:

    Most excit­ing that this can be done!!!

  • David Longstreet says:

    The AI gen­er­at­ed sub­sti­tutes don’t look at all the the orig­i­nal.

  • Sarah Crocker says:

    Yup, this is the artist that can enhance the gam­ing design, CGI tech­nol­o­gy, and VR to make the Mag­ic the Gath­er­ing trail­er as close to real as I’ve ever seen. Nice job and use of A.I.! Very, very impres­sive work! The A.I. Artist…I like that a lot bet­ter than hear­ing about the Robot that wants to destroy the human race. ARTISTS ARISE!

  • Denis says:

    Very bad­ly done. The arti­fi­cial faces don’t look at all like the orig­i­nals

  • Barbara says:

    Not appre­ci­at­ed..
    cease and desist
    cre­ate your own art

Leave a Reply

Open Culture was founded by Dan Colman.